Duplicity

verify --compare-data behaviour

Asked by Shuhao on 2016-06-02

I was reading the source code and I don't quite understand the --compare-data flag for duplicity.

As far as I can tell:

1. globals.compare_data is set by the command line flag in duplicity/commandline.py
2. verify is called, which calls compare_verbose(..., globals.compare_data) in bin/duplicity
3. compare_verbose is called with include_data = globals.compare_data. It checks if include_data is False, immediately returning True (verify is successful?) if it is the case.

My questions:

1. Does verify do anything without --compare-data? My tests with -v9 suggest it does, as it seems to pull the data down from remote (BackBlaze B2 in this case).
2. Is there any way to make duplicity do some sort of signature/meta data verification only without pulling down the whole archive? If i want to do that every day it will become very expensive with a provider like B2.

Question information

Language:
English Edit question
Status:
Solved
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Solved by:
edso
Solved:
2016-06-03
Last query:
2016-06-03
Last reply:
2016-06-03
Best edso (ed.so) said : #1

On 03.06.2016 01:52, Shuhao wrote:
> New question #294800 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/294800
>
> I was reading the source code and I don't quite understand the --compare-data flag for duplicity.
>
> As far as I can tell:
>
> 1. globals.compare_data is set by the command line flag in duplicity/commandline.py
> 2. verify is called, which calls compare_verbose(..., globals.compare_data) in bin/duplicity
> 3. compare_verbose is called with include_data = globals.compare_data. It checks if include_data is False, immediately returning True (verify is successful?) if it is the case.
>
> My questions:
>
> 1. Does verify do anything without --compare-data? My tests with -v9 suggest it does, as it seems to pull the data down from remote (BackBlaze B2 in this case).

yes, it restores file by file locally to a temp location essentially checking if the volumes can be restored flawlessly

> 2. Is there any way to make duplicity do some sort of signature/meta data verification only without pulling down the whole archive? If i want to do that every day it will become very expensive with a provider like B2.
>

nope. the concept of duplicity to have dumb backends. all logic happens locally and to verify the volumes have to be retrieved again for checkup ;)

..ede/duply.net

Aaron Whitehouse (aaron-whitehouse) said : #2

> 2. Is there any way to make duplicity do some sort of signature/meta data verification only without pulling
> down the whole archive? If i want to do that every day it will become very expensive with a provider like B2.

The way that I deal with this is to backup to a local hard drive and then, if (and only if) the backup and verify are successful, rsync the files to a remote location. That obviously requires local storage space, but makes things much faster as well as reducing transfers.

Shuhao (shuhao) said : #3

> The way that I deal with this is to backup to a local hard drive and then, if (and only if) the backup and verify are successful, rsync the files to a remote location. That obviously requires local storage space, but makes things much faster as well as reducing transfers.

Yup. This is the exact workaround I was planning. Thanks for all the help :)

Shuhao (shuhao) said : #4

Thanks edso, that solved my question.