base system installation is not robust against transient network failures

Bug #954197 reported by Colin Watson
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
base-installer (Debian)
New
Unknown
base-installer (Ubuntu)
Triaged
Medium
Unassigned
debootstrap (Debian)
Fix Released
Unknown
debootstrap (Ubuntu)
Fix Released
Medium
Unassigned

Bug Description

[This bug was originally reported by Gary Potwin in https://answers.launchpad.net/ubuntu/+source/ubiquity/+question/158404]

I have a Supermicro P6SBA motherboard with a 700 MHz Pentium III, 512 M of ram, 20G and 120G hard drives, and a DSL Internet connection.

This system has been running Windows 98 for years, and I wanted to try Ubuntu 10.04.

Using the network kernel and initrd, the systems boots OK and downloads the rest of the installer OK from the default mirror (us.archive.ubuntu.com) except for one small problem (see below).

Everything seems OK until I try to load the base system.

After downloading files for about 3.5 minutes (often when it is trying to get the file libklibc), I see the network activity stop, and soon after I get an error message stating that it has failed to load that file (all others up to that point were OK).

After about 2 more minutes, during which time one or more additional files fail to load, the network activity goes back to normal, and all the remaining files for the base system download OK.

Due to the failed files, I get the error message that the base system has failed to install.

I did successfully download the failed files using wget into /target/var/cache/apt/archives while the automatic download was still in progress using a console, but after the above failure.

The system still thinks that the files were not successfully downloaded, and I don't know how to tell the system that they are there and OK. I have tried using many different mirrors at different times of the day, and both http and ftp, all fail as above.

Using a similar technique, I was able to successfully load Debian 5.08, so I think the hardware is OK, but I would really like to try the Ubuntu.

To try to rule out any problem with the DSL, I downloaded a very large file that took 10 minutes of continuous running under Windows.

Then I went back to the 10.04 install and did the same thing using wget at a console, just after partitioning the hard drive, and just before starting the base install, and it worked fine. While loading the installer, one file did fail to load, but you were given the opportunity to retry, which took care of the problem.

 I wish the base install allowed retries instead of just "go back" and "continue", which don't seem to make any additional attempt to retry.

Any help would be appreciated.

Gary

Colin Watson (cjwatson)
description: updated
affects: ubiquity (Ubuntu) → debootstrap (Ubuntu)
Revision history for this message
Colin Watson (cjwatson) wrote :

This is a complex problem that has been known in the Debian installer since at least 2004. I'm going to try to break it down here in the hope of making some progress on it.

1. Download error handling in debootstrap is arranged wrongly

  In particular, it doesn't deal correctly with corrupted files, and will tend to muddle on until something fails as a consequence of the corruption. In some cases it's possible for debootstrap to complete successfully despite a corrupted download! There's a patch in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=618920 that improves things, although I've been working on a better version of it.

2. No retry option

  As Joey notes in http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=283600, there's only a fairly limited communication channel between debootstrap (which is a separate tool invoked by the installer to do the hard work) and the parts of the installer that can actually interact with the user. This means that it's hard to set up a "retry" option that just retries a single download, because debootstrap doesn't wait for user interaction on errors and it would be a substantial amount of work to rearrange it to do so.

  What we might be able to do is as follows: if debootstrap fails at the retrieval stage before it actually starts unpacking anything, then we could offer an option that simply tries the whole thing again, keeping the previous contents of /target (so that would also preserve anything you'd wgetted by hand, but it would also try to redownload any other missing files for itself). This is a little less neat, but would do the job. In fact, if we borrowed some ideas from net-retriever, we could even let you choose a different mirror.

Changed in base-installer (Ubuntu):
status: New → Triaged
importance: Undecided → Medium
Changed in debootstrap (Ubuntu):
status: New → Triaged
importance: Undecided → Medium
Revision history for this message
Colin Watson (cjwatson) wrote :

The attached stunt web proxy is useful for debugging this kind of thing.

Changed in base-installer (Debian):
status: Unknown → New
Colin Watson (cjwatson)
Changed in debootstrap (Ubuntu):
status: Triaged → In Progress
Changed in debootstrap (Debian):
status: Unknown → New
Revision history for this message
Colin Watson (cjwatson) wrote :

debootstrap (1.0.39) unstable; urgency=low

  * Retry corrupted downloads rather than carrying on almost regardless.
    Patch mostly due to Michael Gilbert, rearranged somewhat by me (closes:
    #618920).
  * Stop at the end of the retrieval phase if any packages failed to
    download.

 -- Colin Watson <email address hidden> Tue, 13 Mar 2012 17:21:13 +0000

Changed in debootstrap (Ubuntu):
status: In Progress → Fix Released
Changed in debootstrap (Debian):
status: New → Fix Released
Revision history for this message
Gary Potwin (gary-theworld) wrote :

Hello, Colin Watson. Thank you for the two emails that you sent to me on 3/13/2012 concerning this bug. Did you receive my responding email on 3/15/2012? Please let me know if I need to resend it.

Thank you,

Gary Potwin

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.