files list file for package `*' is missing final newline

Bug #108189 reported by Jan Klopper
This bug affects 42 people
Affects Status Importance Assigned to Milestone
dpkg (Debian)
New
Undecided
Unassigned
dpkg (Ubuntu)
Invalid
High
Unassigned

Bug Description

APt-get, adept, and the restricted drivers manager might halt installing packages when one or more of the files in:

/var/lib/dpkg/info/

is corrupt.

The error shown will be either:
files list file for package `*' is missing final newline
Or:
files list file for package `*' contains an empty filename

Where * start is a random but steady "per install" package name.

Adding a newline char to the file changes the message form the first to the seccond error noting about the empty filename.

Since all these files seem to contain only a string of @^@^@^@^@^@^@^@^@^@^@^@^
it seems they are redundant, and i ventured on moving them from the folder, this solved the problem.

Two things need to be resolved here i guess,
1: dpkg should handle these files more gracefully, there's an abundance of forum posts and related bugs available from people who have no clue where to search, deleting them seems to work, so why not delete them when dpkg encounters this error.
2: Since all these files essentially contain the same data, why are they even here, there's 4200 of them in a default feisty install, the installer works fine without, so why are they here?

.:: WORKAROUND ::.
The reference is http://ubuntuforums.org/showpost.php?p=2635193&postcount=12 . I've rewritten some part of the post to adapt it to Ubuntu.

There is an easy way to fix this problem without having those serious warnings. I did it and it worked out fine. I found the solution in the following web site: http://finkproject.org/faq/usage-fink.php

I'll just copy it here:

Quote:
I can't install or remove anything, because of a problem with a "files list file".

A: Typically these errors take the form:

Quote:
files list file for package packagename contains empty filename
or

Quote:
files list file for package packagename is missing final newline
This can be fixed, with a little work. If you have the .deb file for the offending package currently available on your system, then check its integrity by running

Quote:
dpkg --contents full-path-to-debfile

e.g.

Quote:
dpkg --contents /var/cache/apt/archive/libgnome2-0_2.24.1-0ubuntu3_i386.deb
If you get back a listing of directories and files, then your .deb is OK. If the output is something other than directories and files, or if you don't have the .deb file, you can still proceed because the error doesn't interfere with builds.

If you have been installing from the binary distribution or you know for sure that the version in the binary distribution is the same as what you have installed (e.g. by checking the package database), then you can get a .deb file by running sudo apt-get install --reinstall --download-only packagename .

Once you have a valid .deb file, then you can reconstitute the file. First become root by using sudo -s (enter your administrative user password if necessary), and then use the following command:

Quote:
dpkg -c full-path-to-debfile | awk '{if ($6 == "./"){ print "/."; } \
else if (substr($6, length($6), 1) == "/")\
{print substr($6, 2, length($6) - 2); } \
else { print substr($6, 2, length($6) - 1);}}'\
> /sw/var/lib/dpkg/info/packagename.list
e.g.

Quote:
dpkg -c /var/cache/apt/archive/libgnome2-0_2.24.1-0ubuntu3_i386.deb | awk \
'{if ($6 == "./") { print "/."; } \
else if (substr($6, length($6), 1) == "/") \
{print substr($6, 2, length($6) - 2); } \
else { print substr($6, 2, length($6) - 1);}}' \
> /var/lib/dpkg/info/libgnome2-0.list
What this does is to extract the contents of the .deb file, remove everything but the filenames, and write these to the .list file.

I hope that helps.

Revision history for this message
Jan Klopper (janklopper-innerheight) wrote :

Here's the first offending file from my system.

Revision history for this message
Ian Jackson (ijackson) wrote : Re: [Bug 108189] files list file for package `*' is missing final newline

Killercow writes ("[Bug 108189] files list file for package `*' is missing final newline"):
> APt-get, adept, and the restricted drivers manager might halt installing
> packages when one or more of the files in:
> /var/lib/dpkg/info/
> is corrupt.

These files should not be corrupted. If they are then you can fix it
up manually if you think you can, or alternatively restore your system
from backups. (Just removing them is not good enough, as they affect
the way dpkg handles upgrades and removals.)

However, the corrupted error message is definitely a problem. If your
hardware is functioning properly this shouldn't happen.

> Since all these files seem to contain only a string of
> @^@^@^@^@^@^@^@^@^@^@^@^ it seems they are redundant, and i ventured
> on moving them from the folder, this solved the problem.

Your machine is misbehaving somehow. I have some questions:

* What kind of filesystem contains your /var/lib/dpkg ?
* What locale are you using ?

If you can reproduce the dpkg corrupted error message then please tar
up your /var/lib/dpkg and attach it to this bug report.

Ian.

Revision history for this message
Jan Klopper (janklopper-innerheight) wrote :

Hi,

The install was a brand new install from the 7.04 dvd,
It was installed onto a 36gb scsi disk connected to a compaq 5300 smartarray controller inside a quad opteron.
The filesystem for / is ext3
/home is mounted on a sata disk also with ext3.

I use the default english language and i guess also the default charset.

Il try to recreate the invalid files, and zip the folder.
The attachment contains the broken file, which has the same contents as the rest of the files in the folder.

I checked the md5 sums on the .deb files for the broken packages, but they are okay.

With regards
Jan

Revision history for this message
rambo3 (adnanba) wrote :

This is the error in swedish basicly reporst missing last line in file libwrap0.
Why is dpkg reading libwrap0 when i am just trying to uninstall hostapd.

dpkg: fel vid hantering av hostapd (--remove):
 files list file for package `libwrap0' is missing final newline
Fel uppstod vid hantering:
 hostapd
Hanteringen avbröts då för många fel uppstod.
E: Sub-process /usr/bin/dpkg returned an error code (1)

I am attaching files in RAR format .

cp /var/lib/dpkg/info/libwrap*

Revision history for this message
Jan Klopper (janklopper-innerheight) wrote :

Your list file is almost completely correct, the end however is missing.

If you know xml you can try to fix it by adding the end tags. (try opening it in firefox after renaming the extention to .xml)
But you can never be sure if you need to add just the closing tags, or if there's more information missing.

Revision history for this message
rambo3 (adnanba) wrote :

I know xml . end tag would be </xml> in that case.
I think i have a bigger problem then this. It might be the bad blocks on HD.
 I dont know its 5 th time i had to reinstalll ubuntu feisty, first missing new line from apt , then i run fsck on unmounted partition and then reinstall becouse nothing works . Extracting backup will take as much time as reinstalling .
 I had debian woody that worked before ubuntu with no problems.
 Can one run some kind of badblock utility before installing. I am out of ideas

Revision history for this message
Matteo Z (matteozandi) wrote :

I also was affected and it was fairly annoying. I managed to workaround following this thread http://ubuntuforums.org/showthread.php?t=12737, altough you end up with apt complaining with "serious warnings". I found out that you can pass the serious warnings if you reinstall each package affected by the serious warning. I'm not at home now, so I can't paste the warnings I'm talking about and the script I plan to use, but I plan to do i shortly (week end).

Revision history for this message
rambo3 (adnanba) wrote :

That is not realy a good solution . I've dont that couple of times. It will eventiualy break all there is to break.
I've installed debian and rebooted couple of times , so far with no problems.
But i still would like to know what happend and why .

Revision history for this message
Matteo Z (matteozandi) wrote :

Full story: I installed feisty on my desktop machine, I actually noticed that the installation process was slow.. I used a RW CD, so I thought that maybe the slowness was due to the media I used. The installation process ended up with an error such "files list file for package `*' is missing final newline" but when I rebooted the comp was going fine.

I customized it a bit, turned gl effects on, etc but then I noticed that apt was still throwing that error refusing to install any packages. I looked on the duplicates bugs and found the forum post mentioned above. I played a bit with apt files under /var, but then apt had another error

dpkg: serious warning: files list file for package `libgnome-media0' missing, assuming package has no files currently installed.
dpkg: serious warning: files list file for package `netcat' missing, assuming package has no files currently installed.
[...]

so I wrote a little script that reinstalls every packaged mentioned in those warnings. Now apt works without complaining.

Rmabo3, you are right, I also would like to know how it happened, but I really wanted feisty and did my best to fix this thing.

Revision history for this message
Ian Jackson (ijackson) wrote : Re: [Bug 108189] Re: files list file for package `*' is missing final newline

rambo3 writes ("[Bug 108189] Re: files list file for package `*' is missing final newline"):
> I know xml . end tag would be </xml> in that case.
> I think i have a bigger problem then this. It might be the bad blocks on HD.
> I dont know its 5 th time i had to reinstalll ubuntu feisty, first missing new line from apt , then i run fsck on unmounted partition and then reinstall becouse nothing works . Extracting backup will take as much time as reinstalling .
> I had debian woody that worked before ubuntu with no problems.
> Can one run some kind of badblock utility before installing. I am out of ideas

Yes, I think your hardware is broken. It may well be a bad
motherboard, CPU or memory, rather than a bad hard disk.

Ian.

Revision history for this message
Bert (rijmeer) wrote :

I'm having the exact same problem. In my case I get the error message about the file /var/lib/dpkg/info/openoffice.org-kde-integration.list. I had a look at this file and found out that it contains some text that has got nothing to do with openoffice.org. Instead it's about the package xman. I attached this file to this posting.

Revision history for this message
Ramin Nakisa (ramin-nakisa) wrote :

I get exactly the same problem with a fresh 7.10 install. The CD installation failed at about 95% through with this error message, but allowed me to complete the installation by clicking on continue. Now I've just spent a few hours working through the awk script method described here

   http://ubuntuforums.org/showthread.php?t=12737&page=2

Now I'm having the same problem again. This seems like a serious bug to me.

Revision history for this message
foox (fooxland) wrote :

hi,
i've going through the same problem for three days now. I've tried to install 6 or 7 times using different medias and even changed the memory. I also tried the methods described in the forum and a few other things.
it does look like a corrupted file is being downloaded during the installation and brings trouble.
i'm locked.
did anyone progress on this ?

Revision history for this message
foox (fooxland) wrote :

hi,

i just did a fresh clean install without any problems. i really don't get it.

Revision history for this message
Alex Mayorga (alex-mayorga) wrote :

I had this problem today.

The underlying problem seems to be that my HD had some blocks linked twice or something of that sort. I believe the hard drive might be too old already.

I forced a fsck and let it correct all the inodes and stuff.

I'm still wondering what's the best way I can I fix this error and update the system.

This is on Xubuntu 7.10

Revision history for this message
Philipp Bönhof (phboe-launchpad) wrote :

Hi,
I had similar problems after new installation of Ubuntu 8.04.

My error message was:
dpkg: serious warning: files list file for package `*' missing, assuming package has no files currently installed.

For * being several installed packages eg. python-conf, xserver-xorg-video-ati and so on.
In /var/lib/dpkg/info/ I really found that the *.list files for those packages were missing.

But the packages really seemed to be installed, because a "sudo apt-get install python-conf" would tell that it is installed.
For the same reason "sudo apt-get -f install" did not work as well.

The only option I had was to trick apt-get to spit out the tentatively missing packages again:
sudo apt-get --reinstall <some installed package>

With this you can build up a command to reinstall all such broken packages:

sudo apt-get --reinstall install $(echo y | sudo apt-get --reinstall install python-conf | grep "dpkg:" | grep -o "\`.*'" | sed "s/[\`']//g")

Unfortunately there are cyclic dependencies between some of these packages and also other conflicts while reinstalling.
A work around would be to add in a "| head -n20" before the ')' to only install the first 20 of the list. And then run the whole command repeatedly.

Changed in dpkg:
status: New → Triaged
description: updated
Changed in dpkg:
importance: Undecided → High
Revision history for this message
Emill (empann) wrote :

I am also getting this problem with Ubuntu 8.10
I can't install anything anymore...
This bug is more than two years old. Hasn't it been fixed yet??

gopi (gopalakrishnan29)
Changed in dpkg (Ubuntu):
assignee: nobody → gopi (gopalakrishnan29)
Revision history for this message
Sergey Mindar (sergey-mindar) wrote :

Workaround worked perfectly on 10.04 Lucid netbook remix, Acer Aspire One /1gb RAM/ 32GB supertalent SSD.
Corrupted package was "libc-bin"

Only one thing in workaround been amended:
use:
> /var/lib/dpkg/info/packagename.list
instead of:
> /sw/var/lib/dpkg/info/packagename.list

Revision history for this message
Len Tyree (len-tyree) wrote : Re: [Bug 108189] Re: files list file for package `*' is missing final newline
Download full text (4.8 KiB)

Hi, just received an update from Linux/Ubuntu to the libc-bin.
Let it update, nut there was no change in my system, still unable to read/see anything on my usbdrive? The update was for Ubuntu 7, but I thought it might work for the later versions of Ubuntu, silly me.

So I am still stuck. Thanks, Len.

Maybe they will update the one in Ubuntu 10.04? I can't be the only person having this problem. (At least I hope not.)

________________________________
From: Sergey Mindar <email address hidden>
To: <email address hidden>
Sent: Tue, May 25, 2010 3:16:26 AM
Subject: [Bug 108189] Re: files list file for package `*' is missing final newline

Workaround worked perfectly on 10.04 Lucid netbook remix, Acer Aspire One /1gb RAM/ 32GB supertalent SSD.
Corrupted package was "libc-bin"

Only one thing in workaround been amended:
use:
> /var/lib/dpkg/info/packagename.list
instead of:
> /sw/var/lib/dpkg/info/packagename.list

--
files list file for package `*' is missing final newline
https://bugs.launchpad.net/bugs/108189
You received this bug notification because you are a direct subscriber
of a duplicate bug.

Status in “dpkg” package in Ubuntu: Triaged

Bug description:
APt-get, adept, and the restricted drivers manager might halt installing packages when one or more of the files in:

/var/lib/dpkg/info/

is corrupt.

The error shown will be either:
files list file for package `*' is missing final newline
Or:
files list file for package `*' contains an empty filename

Where * start is a random but steady "per install" package name.

Adding a newline char to the file changes the message form the first to the seccond error noting about the empty filename.

Since all these files seem to contain only a string of @^@^@^@^@^@^@^@^@^@^@^@^
it seems they are redundant, and i ventured on moving them from the folder, this solved the problem.

Two things need to be resolved here i guess,
1: dpkg should handle these files more gracefully, there's an abundance of forum posts and related bugs available from people who have no clue where to search, deleting them seems to work, so why not delete them when dpkg encounters this error.
2: Since all these files essentially contain the same data, why are they even here, there's 4200 of them in a default feisty install, the installer works fine without, so why are they here?

.:: WORKAROUND ::.
The reference is http://ubuntuforums.org/showpost.php?p=2635193&postcount=12 . I've rewritten some part of the post to adapt it to Ubuntu.

There is an easy way to fix this problem without having those serious warnings. I did it and it worked out fine. I found the solution in the following web site: http://finkproject.org/faq/usage-fink.php

I'll just copy it here:

Quote:
I can't install or remove anything, because of a problem with a "files list file".

A: Typically these errors take the form:

Quote:
files list file for package packagename contains empty filename
or

Quote:
files list file for package packagename is missing final newline
This can be fixed, with a little work. If you have the .deb file for the offending package currently available on your system, then check its integrity by running

Quote:
dpkg --contents full...

Read more...

Revision history for this message
MarkMT (mark-thomson) wrote :

Also worked for me on 8.10 with corrupted 'cups' package. I had a
complication in that at some point some files from the dpkg package,
which was also pending an upgrade, disappeared. I had to download the
old .deb file and extract and copy the missing files manually, then
upgrade dpkg to the latest version, before updating the rest of the system.

Sergey Mindar wrote:
> Workaround worked perfectly on 10.04 Lucid netbook remix, Acer Aspire One /1gb RAM/ 32GB supertalent SSD.
> Corrupted package was "libc-bin"
>
> Only one thing in workaround been amended:
> use:
>
>> /var/lib/dpkg/info/packagename.list
>>
> instead of:
>
>> /sw/var/lib/dpkg/info/packagename.list
>>
>
>

gopi (gopalakrishnan29)
Changed in dpkg (Ubuntu):
assignee: gopi (gopalakrishnan29) → nobody
Revision history for this message
Aaron C. de Bruyn (darkpixel2k) wrote :

I am running into this fairly regularly on systems I have deployed for customers. The only thing in common with the systems is that they have the root partition on an SSD. All of these systems are using the 'KINGSTON SS100S2' SSD. I get no errors about disk problems in dmesg, but if I 'touch /forcefsck && shutdown -r now', the box will reboot and usually find filesystem corruption.

Out of roughly 45 machines there are 10 with SSDs. 9 of the SSD machines have suffered from this issue. My resolution was to delete /var/lib/dpkg/info/<package>.list and re-run 'apt-get update && apt-get upgrade'. None of the non-SSD machines have run into this issue. Lastly, the SSD machines are a mix of 10.04 and 10.10.

Revision history for this message
Raphaël Hertzog (hertzog) wrote :

dpkg does not produce invalid .list files. Data gets corrupted by bad hardware (disk/memory/filesystem bugs) and after that dpkg complains about the corrupted data.

All those .list files are important. If one gets corrupted, you can remove it but you should immediately reinstall the corresponding package to regenerate it (apt-get --reinstall install <package>). The format of those files is easy, one filename per line and nothing else. If you see garbage, then it's corrupted.

Changed in dpkg (Ubuntu):
status: Triaged → Invalid
Revision history for this message
Eliah Kagan (degeneracypressure) wrote :

If data corruption is common enough that there enough people experiencing it that there are 41 people indicating they are experiencing it here (which is probably a small fraction), then perhaps this bug should be reopened as a feature request for redundancy in .list files to be implemented. (Example fix: when a .list file changes, the new version is used and backed up if it is valid, but the user is notified and given the opportunity to use an automatic backup if it is invalid.)

Revision history for this message
Raphaël Hertzog (hertzog) wrote :

(Eliah I get the mail by a group subscription, no need to subscribe me)

It's a wrong good idea IMO. A computer where you can't trust data stored in your filesystem is not very useful. That said I agree that dpkg could give more clue as to what can be done to recover. But then it's merely a wishlist bug.

I'd rather open a new bug requesting "dpkg --verify-database" and "dpkg --repair-database" rather than hijack this one as changing the subject means people will no longer find it. Feel free to submit it.

Revision history for this message
Klonos (klonos) wrote :

Years later I still got tormented by this issue. Just by landing in this page I was amazed by the amount of duplicate bug reports filed for this very same issue!! ...well? Has there been any follow-ups on this one? Any feature requests for "dpkg --verify-database" and "dpkg --repair-database" as Raphaël suggests in the previous message?

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.