No Signature found switching to full backup chronic issue.

Asked by Chris Stankaitis on 2011-12-20

Hi there I need some help. I am running incrementals on our one box or at least they are suposed to be incrementals but every day I get the following messages in my reports:

Synchronizing remote metadata to local cache...
Deleting local /var/duplicity/16066d0c9dd4b35c7c548107c36858ec/duplicity-full-signatures.20111218T233829Z.sigtar.gz (not authoritative at backend).
Deleting local /var/duplicity/16066d0c9dd4b35c7c548107c36858ec/duplicity-full.20111218T233829Z.manifest (not authoritative at backend).
Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1324334717.05 (Mon Dec 19 17:45:17 2011)
EndTime 1324349894.33 (Mon Dec 19 21:58:14 2011)
ElapsedTime 15177.28 (4 hours 12 minutes 57.28 seconds)
SourceFiles 1038
SourceFileSize 199852381384 (186 GB)
NewFiles 1038
NewFileSize 199852381384 (186 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 1038
RawDeltaSize 199852311752 (186 GB)
TotalDestinationSizeChange 39791702948 (37.1 GB)
Errors 0
-------------------------------------------------

I can't sustain this much disk space usage for very long I need these incremental to become incremental. Please help.

duplicity 0.6.14
python - 2.6.5
RHEL 6.0

Question information

Language:
English Edit question
Status:
Answered
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Last query:
2011-12-22
Last reply:
2011-12-22
Chris Stankaitis (cstankaitis) said : #1

Oh and my backup command is

duplicity --no-encryption --archive-dir "/var/duplicity" --tempdir "/var/tmp" PATH-TO-TARGET scp://backupuser@$backupServer:22//backup/backup2.tor.fmpub.net/PATH-TO-TARGET

edso (ed.so) said : #2

On 20.12.2011 17:45, Chris Stankaitis wrote:
> Question #182479 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/182479
>
> Chris Stankaitis gave more information on the question:
> Oh and my backup command is
>
> duplicity --no-encryption --archive-dir "/var/duplicity" --tempdir
> "/var/tmp" PATH-TO-TARGET
> scp://backupuser@$backupServer:22//backup/backup2.tor.fmpub.net/PATH-TO-
> TARGET
>

please send a private data obfuscated output of your command with maximum verbosity '-v9' to pastebin.

please also try latest release 0.6.19 .. ede/duply.net

Chris Stankaitis (cstankaitis) said : #3

Here is a pastebin lik to the verbose. This is a production server so I would like to avoid "upgrade" being the default solution to this issue, unless when looking at the verbose output you can say with certain that the upgrade will resolve this issue.

http://pastebin.com/9JAB10bM

Chris Stankaitis (cstankaitis) said : #4

I upgraded the client to 0.6.17 which is your current stable as far as the https://launchpad.net/duplicity lists, due to corporate policy I can't use dev releases in production. the behaviour presists in the current stable release.

I tried the work-around listed in

https://bugs.launchpad.net/duplicity/+bug/829198

and it works, would you please help me to understand the difference with the rsync (over ssh) backend vs the scp backend how are they different and why would changing the backend solve this problem? other then the obvious "there is a bug in the scp backend"

Is there any documentation I can reference to look at the differences between all the backend options to determine if I am using the optimal one?

edso (ed.so) said : #5

On 22.12.2011 20:00, Chris Stankaitis wrote:
> Question #182479 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/182479
>
> Chris Stankaitis gave more information on the question:
> I upgraded the client to 0.6.17 which is your current stable as far as
> the https://launchpad.net/duplicity lists, due to corporate policy I
> can't use dev releases in production. the behaviour presists in the
> current stable release.
>
> I tried the work-around listed in
>
> https://bugs.launchpad.net/duplicity/+bug/829198
>
> and it works, would you please help me to understand the difference with
> the rsync (over ssh) backend vs the scp backend how are they different

rsync over ssh actually uses the remote shell functionality of ssh. you can look that up in a search engine of your choice.
ssh backend actually uses the sftp binary (sftp client for shell) via pexpect to up/download/list files on the backend (sftp server).

> and why would changing the backend solve this problem? other then the
> obvious "there is a bug in the scp backend"

i couldn't reproduce it so i guess there is no bug per se but an incompatibility between your sftp client and the backends sftp service. could you state name and versions?
for the solution: if sftp does not output a proper listing or error duplicity of course assumes the backend is empty and starts a new full. rsync obviously does not seem to have that problem. btw. if it works i suggest you stick with it as i was told that rsync over ssh is faster than plain sftp.

>
> Is there any documentation I can reference to look at the differences
> between all the backend options to determine if I am using the optimal
> one?
>

if it is not in the manpage you probably have to search in the launchpad answer section or the mailing list...

ede/duply.net

Can you help with this problem?

Provide an answer of your own, or ask Chris Stankaitis for more information if necessary.

To post a message you must log in.