uncompressing failed: Unknown compression algorithm

Asked by Peter Schoo

Running regular backups, appreciating Deja-Dup quite a lot, one day it appears

GPGError: GPG Failed, see log below:
===== Begin GnuPG log =====
gpg: WARNUNG: "--no-use-agent" ist eine veraltete Option - sie hat keine Wirkung.
gpg: AES256 verschlüsselte Daten
gpg: Verschlüsselt mit einer Passphrase
gpg: uncompressing failed: Unbekanntes Komprimierungsverfahren
gpg: uncompressing failed: Unbekanntes Komprimierungsverfahren
gpg: WARNUNG: Botschaft wurde nicht integritätsgeschützt (integrity protected)
===== End GnuPG log =====

On Ubuntu Bionic, using deja-dup 37.0, haviing gpg (GnuPG) 2.2.4 in use that is capable to run uncompressed, ZIP, ZLIB, BZIP2.

Any idea what went wrong? Suggestions how to improve/correct will be highly appreciated.

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu deja-dup Edit question
Assignee:
No assignee Edit question
Solved by:
Michael Terry
Solved:
Last query:
Last reply:
Revision history for this message
Michael Terry (mterry) said :
#1

Hello! I’m sorry you’re having this problem.

I’m likely busy today, but this doesn’t seem super time sensitive?

One thing to try is blowing away ~/.cache/deja-dup and trying again.

If that doesn’t work, we should start examining the backup files themselves in the storage location. Are any of them a weirdly different size? (We’re looking for maybe a file that got malformed or something).

You could run “file” on all the files on the backend and maybe find something that doesn’t look right?

Revision history for this message
Peter Schoo (peat-pet) said :
#2

Thanks for your prompt reply. And yes, it's not super urgent.

It unfortunalety did not do the trick to `rm' the cache. So I inspected the backup files. To me all of them look quite fine. Though there is a considerable number of files, we're not running out of space on the 3T disk.

<pre>
$ ls | wc
  15689 15689 836321
$ ls duplicity-full.* | wc
  14706 14706 768304
$ du -h .
769G .
</pre>

All but one replied

<pre>
GPG symmetrically encrypted data (AES256 cipher)
</pre>

when you ask them their `file'. The one exceptional answer in the miid of the duplicity-full-series is

<pre>
DOS/MBR boot sector; partition 1 : ID=0x42, active 0x99, start-CHS (0x3d6,224,48), end-CHS (0x24e,220,24), startsector 883185466, 3010338292 sectors; partition 2 : ID=0xe2, active 0xeb, start-CHS (0x23d,72,4), end-CHS (0x275,159,1), startsector 3196491606, 323988048 sectors; partition 4 : ID=0xa8, active 0x9e, start-CHS (0x3da,146,19), end-CHS (0x2e,197,8), startsector 912481883, 451771139 sectors
</pre>

Thank you for your answer!

Revision history for this message
Michael Terry (mterry) said :
#3

That is a wild reply from “file”.

That is also a huge backup, yikes. :)

So that does seem like a malformed backup volume file in the middle... Same size as the others?

I don’t know why that would happen though. When restoring, duplicity would likely choke on that file. You would still be able to manually recover the rest of the backup if you needed, except for any data chunks in that one volume.

Thankfully, we caught this before you had to find out the hard way.

You were backing up to a local drive?

The weird thing is that it’s not like duplicity wigged out then gpg encrypted some garbage. It’s like the file writing itself messed up (or the gpg encoding).

Revision history for this message
Peter Schoo (peat-pet) said :
#4

Diving again into this

>> So that does seem like a malformed backup volume file in the middle... Same size as the others?

It's not really indicating anything odd

$ ls -l duplicity-full.20200601T102533Z.vol520[2-6].difftar.gpg
-rw------- 1 administrator administrator 52478169 Jun 1 18:34 duplicity-full.20200601T102533Z.vol5202.difftar.gpg
-rw------- 1 administrator administrator 52474507 Jun 1 18:34 duplicity-full.20200601T102533Z.vol5203.difftar.gpg
-rw------- 1 administrator administrator 52469494 Jun 1 18:34 duplicity-full.20200601T102533Z.vol5204.difftar.gpg
-rw------- 1 administrator administrator 52477985 Jun 1 18:34 duplicity-full.20200601T102533Z.vol5205.difftar.gpg
-rw------- 1 administrator administrator 52473444 Jun 1 18:34 duplicity-full.20200601T102533Z.vol5206.difftar.gpg

but there's this strange `file' answer

$ file duplicity-full.20200601T102533Z.vol520[2-6].difftar.gpg
duplicity-full.20200601T102533Z.vol5202.difftar.gpg: GPG symmetrically encrypted data (AES256 cipher)
duplicity-full.20200601T102533Z.vol5203.difftar.gpg: GPG symmetrically encrypted data (AES256 cipher)
duplicity-full.20200601T102533Z.vol5204.difftar.gpg: DOS/MBR boot sector; partition 1 : ID=0x42, active 0x99, start-CHS (0x3d6,224,48), end-CHS (0x24e,220,24), startsector 883185466, 3010338292 sectors; partition 2 : ID=0xe2, active 0xeb, start-CHS (0x23d,72,4), end-CHS (0x275,159,1), startsector 3196491606, 323988048 sectors; partition 4 : ID=0xa8, active 0x9e, start-CHS (0x3da,146,19), end-CHS (0x2e,197,8), startsector 912481883, 451771139 sectors
duplicity-full.20200601T102533Z.vol5205.difftar.gpg: GPG symmetrically encrypted data (AES256 cipher)
duplicity-full.20200601T102533Z.vol5206.difftar.gpg: GPG symmetrically encrypted data (AES256 cipher)

>> You were backing up to a local drive?

Yes, it's a second spacy local drive.
Strange enough, I do from time to time another full backup (to store away elsewhere) and that's entirely okay.

Maybe it falls under the category of 'very unique failure' and I may simply start all over with a full backup. So far there's no data loss that I actually experience.

If you have an idea why this happens, I'd be interested. The code runs perfect, the failure msg was provided, only the reasons for the failure is not clear to me.

Anyhow, Michael, many thanks for you swift responses and, pls, keep up the good work and deja-dup running.

Revision history for this message
Best Michael Terry (mterry) said :
#5

I’ll try to reproduce, like maybe just make a huge backup a few times and see if I can find a bogus volume. But I’ll close this question for now. Thanks for reaching out and sorry I don’t have any answers yet.

Revision history for this message
Peter Schoo (peat-pet) said :
#6

Thanks Michael Terry, that solved my question.