Ubuntu
apt package

apt-get update fails with "The package lists or status file could not be parsed or opened."

Asked by Andrew Havens

I followed the directions in the troubleshooting document: https://help.ubuntu.com/community/PackageManagerTroubleshootingProcedure

However, I'm still having issues. Here's the scroll back:

root@myserver:~# ubuntu-support-status
-bash: ubuntu-support-status: command not found
root@myserver:~# cat /etc/dpkg/dpkg.cfg.d/multiarch
cat: /etc/dpkg/dpkg.cfg.d/multiarch: No such file or directory
root@myserver:~# dpkg --print-foreign-architectures

root@myserver:~# sudo grep -R proxy /etc/apt/*
root@myserver:~# grep proxy /etc/environment
root@myserver:~# echo $http_proxy

root@myserver:~# echo $ftp_proxy

root@myserver:~# grep proxy /etc/bash.bashrc
root@myserver:~# grep proxy ~/.bashrc
root@myserver:~# cat /etc/apt/apt.conf
cat: /etc/apt/apt.conf: No such file or directory
root@myserver:~# sudo fuser -vvv /var/lib/dpkg/lock
root@myserver:~# sudo fuser -vvv /var/cache/apt/archives/lock
root@myserver:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS"
root@myserver:~# uname -a
Linux myserver 3.2.0-24-virtual #37-Ubuntu SMP Wed Apr 25 12:51:49 UTC 2012 i686 i686 i386 GNU/Linux
root@myserver:~# sudo rm /var/lib/apt/lists/lock
root@myserver:~# sudo rm /var/cache/apt/archives/lock
root@myserver:~# sudo rm /var/lib/dpkg/lock
root@myserver:~# sudo touch /var/lib/dpkg/lock
root@myserver:~# sudo cp -arf /var/lib/dpkg /var/lib/dpkg.backup
root@myserver:~# sudo mv /var/lib/dpkg/status /var/lib/dpkg/status-bad
root@myserver:~# sudo cp /var/lib/dpkg/status-old /var/lib/dpkg/status || sudo cp /var/backups/apt.extended_states.0 /var/lib/dpkg/status
root@myserver:~# sudo mv /var/lib/dpkg/available /var/lib/dpkg/available-bad
root@myserver:~# sudo cp /var/lib/dpkg/available-old /var/lib/dpkg/available
root@myserver:~# sudo rm -rf /var/lib/dpkg/updates/*
root@myserver:~# sudo rm -rf /var/lib/apt/lists
root@myserver:~# sudo rm /var/cache/apt/*.bin
root@myserver:~# sudo mkdir /var/lib/apt/lists
root@myserver:~# sudo mkdir /var/lib/apt/lists/partial
root@myserver:~# LANG=C;sudo apt-get clean
root@myserver:~# LANG=C;sudo apt-get autoclean
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.
root@myserver:~# LANG=C;sudo apt-get --purge autoremove
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.
root@myserver:~# LANG=C;sudo apt-get --fix-missing update -o APT::Cache-Limit=100000000
Get:1 http://archive.ubuntu.com precise Release.gpg [198 B]
Get:2 http://archive.ubuntu.com precise-updates Release.gpg [198 B]
Get:3 http://archive.ubuntu.com precise Release [49.6 kB]
Get:4 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get:5 http://archive.ubuntu.com precise-updates Release [49.6 kB]
Get:6 http://security.ubuntu.com precise-security Release [49.6 kB]
Get:7 http://archive.ubuntu.com precise/main i386 Packages [1274 kB]
Get:8 http://security.ubuntu.com precise-security/main i386 Packages [402 kB]
Get:9 http://archive.ubuntu.com precise/universe i386 Packages [4796 kB]
Get:10 http://security.ubuntu.com precise-security/main TranslationIndex [74 B]
Get:11 http://archive.ubuntu.com precise/main TranslationIndex [3706 B]
Get:12 http://archive.ubuntu.com precise/universe TranslationIndex [2922 B]
Get:13 http://archive.ubuntu.com precise-updates/main i386 Packages [788 kB]
Get:14 http://security.ubuntu.com precise-security/main Translation-en [175 kB]
Get:15 http://archive.ubuntu.com precise-updates/universe i386 Packages [244 kB]
Get:16 http://archive.ubuntu.com precise-updates/main TranslationIndex [3564 B]
Get:17 http://archive.ubuntu.com precise-updates/universe TranslationIndex [2850 B]
Get:18 http://archive.ubuntu.com precise/main Translation-en [726 kB]
Get:19 http://archive.ubuntu.com precise/universe Translation-en [3341 kB]
Get:20 http://archive.ubuntu.com precise-updates/main Translation-en [341 kB]
Get:21 http://archive.ubuntu.com precise-updates/universe Translation-en [139 kB]
Fetched 12.4 MB in 10s (1196 kB/s)
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.
root@myserver:~# sudo dpkg --configure -a
dpkg: error: parsing file '/var/lib/dpkg/status' near line 0:
 EOF after field name `'
root@myserver:~# sudo dpkg --clear-avail
root@myserver:~# LANG=C;sudo apt-get -f install
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.
root@myserver:~# LANG=C;sudo apt-get --fix-missing install
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.
root@myserver:~# LANG=C;sudo apt-get update -o APT::Cache-Limit=100000000 && sudo apt-get dist-upgrade
Hit http://archive.ubuntu.com precise Release.gpg
Hit http://archive.ubuntu.com precise-updates Release.gpg
Hit http://archive.ubuntu.com precise Release
Hit http://archive.ubuntu.com precise-updates Release
Hit http://archive.ubuntu.com precise/main i386 Packages
Hit http://archive.ubuntu.com precise/universe i386 Packages
Hit http://archive.ubuntu.com precise/main TranslationIndex
Hit http://archive.ubuntu.com precise/universe TranslationIndex
Hit http://security.ubuntu.com precise-security Release.gpg
Hit http://archive.ubuntu.com precise-updates/main i386 Packages
Hit http://archive.ubuntu.com precise-updates/universe i386 Packages
Hit http://archive.ubuntu.com precise-updates/main TranslationIndex
Hit http://archive.ubuntu.com precise-updates/universe TranslationIndex
Hit http://archive.ubuntu.com precise/main Translation-en
Hit http://archive.ubuntu.com precise/universe Translation-en
Hit http://archive.ubuntu.com precise-updates/main Translation-en
Hit http://archive.ubuntu.com precise-updates/universe Translation-en
Hit http://security.ubuntu.com precise-security Release
Hit http://security.ubuntu.com precise-security/main i386 Packages
Hit http://security.ubuntu.com precise-security/main TranslationIndex
Hit http://security.ubuntu.com precise-security/main Translation-en
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.
root@myserver:~# find /etc/apt -name '*.list' -exec bash -c 'echo -e "\n$1\n"; cat -n "$1"' _ '{}' \;

/etc/apt/sources.list

     1 deb http://archive.ubuntu.com/ubuntu precise main
     2 deb http://archive.ubuntu.com/ubuntu precise-updates main
     3 deb http://security.ubuntu.com/ubuntu precise-security main
     4 deb http://archive.ubuntu.com/ubuntu precise universe
     5 deb http://archive.ubuntu.com/ubuntu precise-updates universe
     6 # deb http://apt.newrelic.com/debian/ newrelic non-free
     7
root@myserver:~#

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu apt Edit question
Assignee:
No assignee Edit question
Solved by:
Andrew Havens
Solved:
Last query:
Last reply:
Revision history for this message
actionparsnip (andrew-woodhead666) said : 		#1

What is the output of:

ls -la /var/lib/dpkg/status

Thanks

Revision history for this message
Andrew Havens (z-email) said : 		#2

# ls -la /var/lib/dpkg/status
-rw-r--r-- 1 root root 444535 Apr 11 19:18 /var/lib/dpkg/status

Revision history for this message
actionparsnip (andrew-woodhead666) said : 		#3

Ok and the output of:

ls /var/lib/dpkg/* | grep -i status

Thanks

Revision history for this message
Andrew Havens (z-email) said : 		#4

# ls /var/lib/dpkg/* | grep -i status
/var/lib/dpkg/status
/var/lib/dpkg/status-bad
/var/lib/dpkg/status-old
/var/lib/dpkg/status.bad

Revision history for this message
Manfred Hampl (m-hampl) said : 		#5

Please provide the output of the following terminal commands:

ls -l /var/lib/dpkg/status*
file /var/lib/dpkg/status*
ls -l /var/backups/apt*
file /var/backups/apt*

Revision history for this message
Manfred Hampl (m-hampl) said : 		#6

... and please also

ls -l /var/backups/dpkg*
file /var/backups/dpkg*

Revision history for this message
Andrew Havens (z-email) said : 		#7

root@myserver:~# ls -l /var/lib/dpkg/status*
-rw-r--r-- 1 root root 444535 Apr 11 19:18 /var/lib/dpkg/status
-rw-r--r-- 1 root root 444535 Apr 11 18:55 /var/lib/dpkg/status-bad
-rw-r--r-- 1 root root 444535 Nov 15 06:33 /var/lib/dpkg/status-old
-rw-r--r-- 1 root root 444620 Nov 15 06:33 /var/lib/dpkg/status.bad
root@myserver:~# file /var/lib/dpkg/status*
/var/lib/dpkg/status: data
/var/lib/dpkg/status-bad: data
/var/lib/dpkg/status-old: data
/var/lib/dpkg/status.bad: data
root@myserver:~# ls -l /var/backups/apt*
-rw-r--r-- 1 root root 18265 Sep 20 2013 /var/backups/apt.extended_states.0
-rw-r--r-- 1 root root 1779 Sep 16 2013 /var/backups/apt.extended_states.1.gz
-rw-r--r-- 1 root root 1667 Sep 11 2013 /var/backups/apt.extended_states.2.gz
-rw-r--r-- 1 root root 1442 Sep 11 2013 /var/backups/apt.extended_states.3.gz
-rw-r--r-- 1 root root 1395 Apr 26 2012 /var/backups/apt.extended_states.4.gz
root@myserver:~# file /var/backups/apt*
/var/backups/apt.extended_states.0: ASCII text
/var/backups/apt.extended_states.1.gz: gzip compressed data, was "apt.extended_states.0", from Unix, last modified: Mon Sep 16 20:24:45 2013, max compression
/var/backups/apt.extended_states.2.gz: gzip compressed data, was "apt.extended_states.0", from Unix, last modified: Wed Sep 11 05:08:42 2013, max compression
/var/backups/apt.extended_states.3.gz: gzip compressed data, was "apt.extended_states.0", from Unix, last modified: Wed Sep 11 03:18:33 2013, max compression
/var/backups/apt.extended_states.4.gz: gzip compressed data, was "apt.extended_states.0", from Unix, last modified: Thu Apr 26 16:19:16 2012, max compression
root@myserver:~# ls -l /var/backups/dpkg*
-rw-r--r-- 1 root root 444620 Nov 15 06:33 /var/backups/dpkg.status.0
-rw-r--r-- 1 root root 128410 Nov 15 06:33 /var/backups/dpkg.status.1.gz
-rw-r--r-- 1 root root 128168 Sep 20 2013 /var/backups/dpkg.status.2.gz
-rw-r--r-- 1 root root 112545 Sep 16 2013 /var/backups/dpkg.status.3.gz
-rw-r--r-- 1 root root 107767 Sep 14 2013 /var/backups/dpkg.status.4.gz
-rw-r--r-- 1 root root 107766 Sep 11 2013 /var/backups/dpkg.status.5.gz
-rw-r--r-- 1 root root 94583 Sep 11 2013 /var/backups/dpkg.status.6.gz
root@myserver:~# file /var/backups/dpkg*
/var/backups/dpkg.status.0: data
/var/backups/dpkg.status.1.gz: gzip compressed data, was "dpkg.status.0", from Unix, last modified: Fri Nov 15 06:33:12 2013, max compression
/var/backups/dpkg.status.2.gz: gzip compressed data, was "dpkg.status.0", from Unix, last modified: Fri Sep 20 05:14:17 2013, max compression
/var/backups/dpkg.status.3.gz: gzip compressed data, was "dpkg.status.0", from Unix, last modified: Mon Sep 16 20:24:45 2013, max compression
/var/backups/dpkg.status.4.gz: gzip compressed data, was "dpkg.status.0", from Unix, last modified: Sat Sep 14 22:04:26 2013, max compression
/var/backups/dpkg.status.5.gz: gzip compressed data, was "dpkg.status.0", from Unix, last modified: Wed Sep 11 05:08:42 2013, max compression
/var/backups/dpkg.status.6.gz: gzip compressed data, was "dpkg.status.0", from Unix, last modified: Wed Sep 11 03:18:33 2013, max compression

Revision history for this message
Manfred Hampl (m-hampl) said : 		#8

Try

sudo cp /var/lib/dpkg/status.bad /var/lib/dpkg/status
sudo apt-get update
sudo apt-get upgrade

and copy/paste all output into this question document as done before.

Revision history for this message
Andrew Havens (z-email) said : 		#9

Same as before:

root@myserver:~# sudo cp /var/lib/dpkg/status.bad /var/lib/dpkg/status
root@myserver:~# sudo apt-get update
Hit http://archive.ubuntu.com precise Release.gpg
Get:1 http://archive.ubuntu.com precise-updates Release.gpg [198 B]
Hit http://archive.ubuntu.com precise Release
Get:2 http://archive.ubuntu.com precise-updates Release [49.6 kB]
Get:3 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get:4 http://security.ubuntu.com precise-security Release [49.6 kB]
Hit http://archive.ubuntu.com precise/main i386 Packages
Hit http://archive.ubuntu.com precise/universe i386 Packages
Hit http://archive.ubuntu.com precise/main TranslationIndex
Hit http://archive.ubuntu.com precise/universe TranslationIndex
Get:5 http://archive.ubuntu.com precise-updates/main i386 Packages [788 kB]
Get:6 http://security.ubuntu.com precise-security/main i386 Packages [402 kB]
Get:7 http://archive.ubuntu.com precise-updates/universe i386 Packages [244 kB]
Hit http://archive.ubuntu.com precise-updates/main TranslationIndex
Hit http://archive.ubuntu.com precise-updates/universe TranslationIndex
Hit http://archive.ubuntu.com precise/main Translation-en
Hit http://archive.ubuntu.com precise/universe Translation-en
Hit http://archive.ubuntu.com precise-updates/main Translation-en
Hit http://archive.ubuntu.com precise-updates/universe Translation-en
Hit http://security.ubuntu.com precise-security/main TranslationIndex
Hit http://security.ubuntu.com precise-security/main Translation-en
Fetched 1534 kB in 2s (557 kB/s)
Reading package lists... Error!
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/dpkg/status
E: The package lists or status file could not be parsed or opened.

Revision history for this message
Manfred Hampl (m-hampl) said : 		#10

what is the output of

head /var/lib/dpkg/status
tail /var/lib/dpkg/status

Revision history for this message
Andrew Havens (z-email) said : 		#11

It appears to be empty:

root@myserver:~# head /var/lib/dpkg/status
root@myserver:~# tail /var/lib/dpkg/status

Revision history for this message
Manfred Hampl (m-hampl) said : 		#12

That is contradictory, according to the output of the ls command, the file size should be 400KB.

Is there any output on

head /var/lib/dpkg/status*

(please copy/paste as done before)

Revision history for this message
Andrew Havens (z-email) said : 		#13

root@myserver:~# head /var/lib/dpkg/status*
==> /var/lib/dpkg/status <==

==> /var/lib/dpkg/status-bad <==

==> /var/lib/dpkg/status-old <==

==> /var/lib/dpkg/status.bad <==
root@myserver:~#

Revision history for this message
Manfred Hampl (m-hampl) said : 		#14

What is the output of

ls -l /var/lib/dpkg/status
hexdump -n 1000 /var/lib/dpkg/status

Revision history for this message
Andrew Havens (z-email) said : 		#15

root@myserver:~# ls -l /var/lib/dpkg/status
-rw-r--r-- 1 root root 444620 Apr 11 21:22 /var/lib/dpkg/status
root@myserver:~# hexdump -n 1000 /var/lib/dpkg/status
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
00003e8

Revision history for this message
Manfred Hampl (m-hampl) said : 		#16

Ok. I conclude, that the most recent versions of your dpkg status files (and their copies) seem all to be broken (binary zeroes where ascii text should be).
I think you have to try to restore older backup copies of dpkg.status.*.gz from /var/backups/.
Sorry that I cannot continue helping for now as I have to leave, and I will have a chance to look into your problem again only next week. Hopefully someone else can help.

Revision history for this message
Andrew Havens (z-email) said : 		#17

Ah, thank you! I got it to work by restoring from a backup (didn't know those existed). Here's what I did:

root@myserver:~# cd /var/backups/
root@myserver:/var/backups# ls -la
total 1488
drwxr-xr-x 3 root root 4096 Apr 11 22:01 .
drwxr-xr-x 11 root root 4096 May 3 2013 ..
-rw-r--r-- 1 root root 18265 Sep 20 2013 apt.extended_states.0
-rw-r--r-- 1 root root 1779 Sep 16 2013 apt.extended_states.1.gz
-rw-r--r-- 1 root root 1667 Sep 11 2013 apt.extended_states.2.gz
-rw-r--r-- 1 root root 1442 Sep 11 2013 apt.extended_states.3.gz
-rw-r--r-- 1 root root 1395 Apr 26 2012 apt.extended_states.4.gz
-rw-r--r-- 1 root root 444620 Nov 15 06:33 dpkg.status.0
-rw-r--r-- 1 root root 444620 Nov 15 06:33 dpkg.status.1
-rw-r--r-- 1 root root 128168 Sep 20 2013 dpkg.status.2.gz
-rw-r--r-- 1 root root 112545 Sep 16 2013 dpkg.status.3.gz
-rw-r--r-- 1 root root 107767 Sep 14 2013 dpkg.status.4.gz
-rw-r--r-- 1 root root 107766 Sep 11 2013 dpkg.status.5.gz
-rw-r--r-- 1 root root 94583 Sep 11 2013 dpkg.status.6.gz
-rw------- 1 root root 618 Nov 15 06:33 group.bak
-rw------- 1 root shadow 512 Nov 15 06:33 gshadow.bak
-rw------- 1 root root 1122 Nov 15 06:33 passwd.bak
drwxr-xr-x 2 postgres postgres 4096 Mar 27 04:37 postgres
-rw------- 1 root shadow 818 Nov 15 06:33 shadow.bak
root@myserver:~# head dpkg.status.0
root@myserver:/var/backups# gunzip dpkg.status.1.gz
root@myserver:/var/backups# head dpkg.status.1
Package: tcpd
Status: install ok installed
Multi-Arch: foreign
Priority: optional
Section: net
Installed-Size: 120
Maintainer: Ubuntu Developers <email address hidden>
Architecture: i386
Source: tcp-wrappers
Version: 7.6.q-21
root@myserver:/var# cp /var/backups/dpkg.status.1 /var/lib/dpkg/status
root@myserver:/var# head /var/lib/dpkg/status
Package: tcpd
Status: install ok installed
Multi-Arch: foreign
Priority: optional
Section: net
Installed-Size: 120
Maintainer: Ubuntu Developers <email address hidden>
Architecture: i386
Source: tcp-wrappers
Version: 7.6.q-21
root@myserver:/var# apt-get update
Get:1 http://security.ubuntu.com precise-security Release.gpg [198 B]
Get:2 http://security.ubuntu.com precise-security Release [49.6 kB]
Get:3 http://security.ubuntu.com precise-security/main i386 Packages [402 kB]
Hit http://archive.ubuntu.com precise Release.gpg
Get:4 http://archive.ubuntu.com precise-updates Release.gpg [198 B]
Hit http://archive.ubuntu.com precise Release
Get:5 http://archive.ubuntu.com precise-updates Release [49.6 kB]
Hit http://security.ubuntu.com precise-security/main TranslationIndex
Hit http://security.ubuntu.com precise-security/main Translation-en
Hit http://archive.ubuntu.com precise/main i386 Packages
Hit http://archive.ubuntu.com precise/universe i386 Packages
Hit http://archive.ubuntu.com precise/main TranslationIndex
Hit http://archive.ubuntu.com precise/universe TranslationIndex
Get:6 http://archive.ubuntu.com precise-updates/main i386 Packages [788 kB]
Get:7 http://archive.ubuntu.com precise-updates/universe i386 Packages [244 kB]
Hit http://archive.ubuntu.com precise-updates/main TranslationIndex
Hit http://archive.ubuntu.com precise-updates/universe TranslationIndex
Hit http://archive.ubuntu.com precise/main Translation-en
Hit http://archive.ubuntu.com precise/universe Translation-en
Hit http://archive.ubuntu.com precise-updates/main Translation-en
Hit http://archive.ubuntu.com precise-updates/universe Translation-en
Fetched 1534 kB in 4s (360 kB/s)
Reading package lists... Done

Revision history for this message
Swaroop S (sri999) said : 		#18

Hi, i'm working on Raspberry pi 3, i have an issues while update/upgrade any package. Please solve this and update me.
This is my error:
Reading package lists... Error!
E: Unable to parse package file /var/lib/dpkg/status (1)
E: The package lists or status file could not be parsed or opened.

Thank you...

Revision history for this message
Manfred Hampl (m-hampl) said : 		#19

@Swaroop S
Please do not piggyback on somebody else's question that has already been marked as solved, but create a question document of your own. There is no way that we can trace the status of your problem in a document of somebody else.