Duplicity

repair signature files

Asked by mcsniff

Hi,

I run a duplicity backup on a debian testing system to rackspace cloud files just with a gpg passphrase and otherwise default configuration.

It has been running fine (with incremental backups) but the other day I noticed ALL the signature files where deleted possibly because of this bug:

https://bugs.launchpad.net/deja-dup/+bug/409593

I believe it was version 0.6.04 (now updated to 0.6.05)

I have an old signature file from the original full backup (in an old ./cache/duplicity folder), I would be happy if I can use that to recreate the full backup and then rebuild the incrementals from there.

Can I copy this file to the server to repair it? What format does it need encrypted, gz, split? I have tried everything I can think of and searched google/forums etc

Thanks for your time

Langdon

Question information

Language:
English Edit question
Status:
Solved
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Solved by:
Kenneth Loafman
Solved:
Last query:
Last reply:
Revision history for this message
Kenneth Loafman (kenneth-loafman) said : 		#1

If you are running encrypted backups, then all you need to do is encrypt the local cache files and rename to .gpg from .gz.

If you are running unencrypted, then just copy the local cache files to the remote.

Revision history for this message
Best Kenneth Loafman (kenneth-loafman) said : 		#2

Kenneth Loafman wrote:
> Question #85268 on duplicity changed:
> https://answers.launchpad.net/duplicity/+question/85268
>
> Status: Open => Answered
>
> Kenneth Loafman proposed the following answer:
> If you are running encrypted backups, then all you need to do is encrypt
> the local cache files and rename to .gpg from .gz.
>
> If you are running unencrypted, then just copy the local cache files to
> the remote.

I misspoke, you first need to gunzip them then gpg them. Do this in a
copy of the cache directory to avoid mistakes.

Revision history for this message
mcsniff (langdon) said : 		#3

Thanks for your help!
> I misspoke, you first need to gunzip them then gpg them. Do this in a
> copy of the cache directory to avoid mistakes.
>
>
 I tried this:
gunzip duplicity-full-signatures.gz
gpg -c duplicity-full-signatures
upload duplicity-full-signatures.sigtar.gpg

Can now list files (previously it couldn't find a signature so couldn't
list), but it still insists on going back to full backup:
> Local and Remote metadata are synchronized, no sync needed.
> Warning, found signatures but no corresponding backup files
> Last full backup date: Tue Sep 22 16:27:04 2009
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Warning, found signatures but no corresponding backup files
> Last full backup date: Tue Sep 22 16:27:04 2009
> No signatures found, switching to full backup.
 cleanup says:

>
> Local and Remote metadata are synchronized, no sync needed.
> Warning, found signatures but no corresponding backup files
> Last full backup date: Tue Sep 22 16:27:04 2009
> Found the following files to delete:
> duplicity-full-signatures.20090922T062704Z.sigtar.gz
> duplicity-full-signatures.20090922T062704Z.sigtar.gpg
> Run duplicity again with the --force option to actually delete.
collection-status says:

> Local and Remote metadata are synchronized, no sync needed.
> Warning, found signatures but no corresponding backup files
> Last full backup date: Tue Sep 22 16:27:04 2009
> Collection Status
> -----------------
> Connecting with backend: CloudFilesBackend
> Archive dir: /root/.cache/duplicity/b6503acfd270d9afc23e3dd5a200f70c
>
> Found 1 backup chain without signatures.
> Signature-less chain 1 of 1:
> -------------------------
> Chain start time: Tue Sep 22 16:27:04 2009
> Chain end time: Mon Oct 5 23:00:24 2009
> Number of contained backup sets: 10
> Total number of contained volumes: 432
> Type of backup set: Time: Num volumes:
> Full Tue Sep 22 16:27:04 2009 422
> Incremental Fri Sep 25 23:17:01 2009 2
> Incremental Sat Sep 26 10:15:39 2009 1
> Incremental Mon Sep 28 23:00:21 2009 1
> Incremental Tue Sep 29 23:00:21 2009 1
> Incremental Wed Sep 30 23:00:23 2009 1
> Incremental Thu Oct 1 14:08:55 2009 1
> Incremental Thu Oct 1 23:00:21 2009 1
> Incremental Fri Oct 2 23:00:25 2009 1
> Incremental Mon Oct 5 23:00:24 2009 1
> -------------------------
>
> No backup chains with active signatures found
> No orphaned or incomplete backup sets found.
cache directory:
> duplicity-24Tz3b-tempdir
> duplicity-3h9gdz-tempdir
> duplicity-E-PQeo-tempdir
> duplicity-full.20090922T062704Z.manifest
> duplicity-full-signatures.20090922T062704Z.sigtar.gz
> duplicity-i2WFnn-tempdir
> duplicity-inc.20090922T062704Z.to.20090925T131701Z.manifest
> duplicity-inc.20090925T131701Z.to.20090926T001539Z.manifest
> duplicity-inc.20090926T001539Z.to.20090928T130021Z.manifest
> duplicity-inc.20090928T130021Z.to.20090929T130021Z.manifest
> duplicity-inc.20090929T130021Z.to.20090930T130023Z.manifest
> duplicity-inc.20090930T130023Z.to.20091001T040855Z.manifest
> duplicity-inc.20091001T040855Z.to.20091001T130021Z.manifest
> duplicity-inc.20091001T130021Z.to.20091002T130025Z.manifest
> duplicity-inc.20091002T130025Z.to.20091005T130024Z.manifest
> duplicity-JBQ8_k-tempdir
> duplicity-nFRaHj-tempdir
> duplicity-oORnds-tempdir
> duplicity-RZ7P0s-tempdir
> duplicity-ULOBqz-tempdir
> duplicity-XZEMsn-tempdir

I know your theory works as I have tried deleting and recovering the
signatures on a smaller backup (with no incremental sets though) maybe
this signature file isn't 100% valid (it was generated before any of the
incrementals)

I will keep trying, Thanks!

Langdon

Revision history for this message
Kenneth Loafman (kenneth-loafman) said : 		#4

mcsniff wrote:
> Question #85268 on duplicity changed:
> https://answers.launchpad.net/duplicity/+question/85268
>
> Status: Answered => Open
>
> mcsniff is still having a problem:
> Thanks for your help!
>> I misspoke, you first need to gunzip them then gpg them. Do this in a
>> copy of the cache directory to avoid mistakes.
>>
>>
> I tried this:
> gunzip duplicity-full-signatures.gz
> gpg -c duplicity-full-signatures
> upload duplicity-full-signatures.sigtar.gpg
>
> Can now list files (previously it couldn't find a signature so couldn't
> list), but it still insists on going back to full backup:
>> Local and Remote metadata are synchronized, no sync needed.
>> Warning, found signatures but no corresponding backup files
>> Last full backup date: Tue Sep 22 16:27:04 2009
>> No old backup sets found, nothing deleted.
>> Local and Remote metadata are synchronized, no sync needed.
>> Warning, found signatures but no corresponding backup files
>> Last full backup date: Tue Sep 22 16:27:04 2009
>> No signatures found, switching to full backup.
> cleanup says:
>
>> Local and Remote metadata are synchronized, no sync needed.
>> Warning, found signatures but no corresponding backup files
>> Last full backup date: Tue Sep 22 16:27:04 2009
>> Found the following files to delete:
>> duplicity-full-signatures.20090922T062704Z.sigtar.gz
>> duplicity-full-signatures.20090922T062704Z.sigtar.gpg
>> Run duplicity again with the --force option to actually delete.
> collection-status says:
>
>> Local and Remote metadata are synchronized, no sync needed.
>> Warning, found signatures but no corresponding backup files
>> Last full backup date: Tue Sep 22 16:27:04 2009
>> Collection Status
>> -----------------
>> Connecting with backend: CloudFilesBackend
>> Archive dir: /root/.cache/duplicity/b6503acfd270d9afc23e3dd5a200f70c
>>
>> Found 1 backup chain without signatures.
>> Signature-less chain 1 of 1:
>> -------------------------
>> Chain start time: Tue Sep 22 16:27:04 2009
>> Chain end time: Mon Oct 5 23:00:24 2009
>> Number of contained backup sets: 10
>> Total number of contained volumes: 432
>> Type of backup set: Time: Num volumes:
>> Full Tue Sep 22 16:27:04 2009 422
>> Incremental Fri Sep 25 23:17:01 2009 2
>> Incremental Sat Sep 26 10:15:39 2009 1
>> Incremental Mon Sep 28 23:00:21 2009 1
>> Incremental Tue Sep 29 23:00:21 2009 1
>> Incremental Wed Sep 30 23:00:23 2009 1
>> Incremental Thu Oct 1 14:08:55 2009 1
>> Incremental Thu Oct 1 23:00:21 2009 1
>> Incremental Fri Oct 2 23:00:25 2009 1
>> Incremental Mon Oct 5 23:00:24 2009 1
>> -------------------------
>>
>> No backup chains with active signatures found
>> No orphaned or incomplete backup sets found.
> cache directory:
>> duplicity-24Tz3b-tempdir
>> duplicity-3h9gdz-tempdir
>> duplicity-E-PQeo-tempdir
>> duplicity-full.20090922T062704Z.manifest
>> duplicity-full-signatures.20090922T062704Z.sigtar.gz
>> duplicity-i2WFnn-tempdir
>> duplicity-inc.20090922T062704Z.to.20090925T131701Z.manifest
>> duplicity-inc.20090925T131701Z.to.20090926T001539Z.manifest
>> duplicity-inc.20090926T001539Z.to.20090928T130021Z.manifest
>> duplicity-inc.20090928T130021Z.to.20090929T130021Z.manifest
>> duplicity-inc.20090929T130021Z.to.20090930T130023Z.manifest
>> duplicity-inc.20090930T130023Z.to.20091001T040855Z.manifest
>> duplicity-inc.20091001T040855Z.to.20091001T130021Z.manifest
>> duplicity-inc.20091001T130021Z.to.20091002T130025Z.manifest
>> duplicity-inc.20091002T130025Z.to.20091005T130024Z.manifest
>> duplicity-JBQ8_k-tempdir
>> duplicity-nFRaHj-tempdir
>> duplicity-oORnds-tempdir
>> duplicity-RZ7P0s-tempdir
>> duplicity-ULOBqz-tempdir
>> duplicity-XZEMsn-tempdir
>
> I know your theory works as I have tried deleting and recovering the
> signatures on a smaller backup (with no incremental sets though) maybe
> this signature file isn't 100% valid (it was generated before any of the
> incrementals)

Without the signature file for the last incremental it's not possible to
do another incremental. It would be possible to recover the latest, but
not to continue the incremental chain.

...Ken

Revision history for this message
mcsniff (langdon) said : 		#5

> Without the signature file for the last incremental it's not possible to
> do another incremental. It would be possible to recover the latest, but
> not to continue the incremental chain.
>
> ...Ken
>
Can I restore the full signature and start from there again (even if I
have to delete the old incrementals) Its just that it will take days to
upload the full backup again.

Langdon

Revision history for this message
Kenneth Loafman (kenneth-loafman) said : 		#6

mcsniff wrote:
> Question #85268 on duplicity changed:
> https://answers.launchpad.net/duplicity/+question/85268
>
> Status: Answered => Open
>
> mcsniff is still having a problem:
>
>> Without the signature file for the last incremental it's not possible to
>> do another incremental. It would be possible to recover the latest, but
>> not to continue the incremental chain.
>>
>> ...Ken
>>
> Can I restore the full signature and start from there again (even if I
> have to delete the old incrementals) Its just that it will take days to
> upload the full backup again.

Yes, from that point you should be able to do an incremental based on
the full backup.

...Ken

Revision history for this message
mcsniff (langdon) said : 		#7

Thanks Kenneth Loafman, that solved my question.

Revision history for this message
mcsniff (langdon) said : 		#8

>>
>>> Without the signature file for the last incremental it's not possible to
>>> do another incremental. It would be possible to recover the latest, but
>>> not to continue the incremental chain.
>>>
>>> ...Ken
>>>
>>>
>> Can I restore the full signature and start from there again (even if I
>> have to delete the old incrementals) Its just that it will take days to
>> upload the full backup again.
>>
>
> Yes, from that point you should be able to do an incremental based on
> the full backup.
>
> ...Ken
>
>
Finally the problem was that the incrementals were still on the server
which was interfering with what I wanted to do.

Basically it was as you said.

- gunzip the signature file
- gpg -c signature file with the password
- copy to server.

I needed to remove all traces of the incrementals from the server and
clear the cache , but it works now

Thanks your help! it saved me a lot of time and annoyance.

Langdon