update downloads bad lists and disables update functions

Asked by Morris Jones on 2011-05-19

The upgrade process appears to be attempting to download package lists from a server, when I have not yet logged into the wireless proxy. Numerous files in /var/lib/apt/lists no longer contain XML, but contain the login proxy html. (The login proxy returns the login screen for any and all web accesses) I have repaired this by getting the files manually with wget, but it happens again about every three weeks. I cannot file a bug with ubuntu-bug because it depends on the update lists to find the package information.
Using Ubuntu 11.04

I am not sure which package is attempting to update /var/lib/apt/lists
I expected the updater to check the file and reject the downloaded file if it is not good xml.
What happens is the login proxy screen html is placed in various files in /var/lib/apt/lists (about 65 of them)

Question information

English Edit question
Ubuntu apt Edit question
No assignee Edit question
Last query:
Last reply:
Fabio Marconi (fabiomarconi) said : #1

Thank you for taking the time to report this issue and helping to make Ubuntu better. Examining the information you have given us, this does not appear to be a bug report so we are closing it and converting it to a question in the support tracker. We appreciate the difficulties you are facing, but it would make more sense to raise problems you are having in the support tracker at https://answers.launchpad.net/ubuntu if you are uncertain if they are bugs. For help on reporting bugs, see https://help.ubuntu.com/community/ReportingBugs#When%20not%20to%20file%20a%20bug.

Fabio Marconi (fabiomarconi) said : #2

Please run in a terminal
sudo apt-get clean
and run again update manager

Morris Jones (morris-jones) said : #3

That doesn't work. When you try to run the update, it crashes with a message about bad XML files. (and they are bad). I have written a perl script that searches for the html files, downloads new versions, and deletes any files containing the html which are not in the repository. (I don't know why you think this isn't a bug, it happens about every 3 days to me)

Perl script below: (Note, the grep for SJSU is how I find the clobbered files)

#!/usr/bin/perl -w

my @files= `grep -l SJSU /var/lib/apt/lists/*`;
foreach $file (@files) {
  print $file . "\n";
  my $ul = $file;
  $ul =~ s!/var/lib/apt/lists/!! ;
  print $ul."\n";
  $ul =~ s!_!/!g ;
  print $ul."\n";
  `rm fl.gz`;
  `rm junk`;
  `rm fl`;
  `rm fl.bz2`;
  my $failedFetch=0;
  `wget -O fl.gz http://$ul.gz` or $failedFetch=1;
  if($failedFetch eq 0) {
    `gzip -d fl.gz` or die("failed gzip -d");
    `sudo cp fl $file` or die("Failed to copy a file");
  } else {
    `wget -O fl http://$ul` or $failedFetch=1;
    if($failedFetch eq 0) {
      `sudo cp fl $file` or die("Failed simple copy");
    } else {
      `wget -O fl.bz2 http://$ul.bz2` or $failedFetch=1;
      if($failedFetch eq 0) {
        `bzip2 -d fl.bz2` or die("Failed bzip2 -d");
        `sudo cp fl $file` or die("Failed bzip2 copy");
      } else {
        print "Couldn't find file $file.\n";
        `sudo rm -rf $file` or print("Failed to remove file\n");
  `rm fl fl.gz fl.bz2 junk`;

Morris Jones (morris-jones) said : #4

The problem seems to be caused by the 802.11 proxy timing out in about 10 minutes of inactivity. (Even though the wireless is connected all day). The update downloads a new set of lists. This clobbers the existing set of lists.

mfauzirahman (mfauzirahman) said : #5

sudo apt-get clean
cd /var/lib/apt
sudo mv lists lists.old
sudo rm -rf lists.old
sudo mkdir -p lists/partial
sudo apt-get clean
sudo apt-get update

mfauzirahman (mfauzirahman) said : #6

Any update on this issue?
Please change to solved status if the problem is solved

Can you help with this problem?

Provide an answer of your own, or ask Morris Jones for more information if necessary.

To post a message you must log in.