Restarting a new duplicity-backup on WebDav stops with a private key error

On 01.11.2012 10:31, Vera Schmidt wrote:
> New question #212966 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/212966
>
> I work with duply and Duplicity to make incremental data-backups on an Internet-Storage with WebDav.
> After updating my ubuntu 10.04 LTS to 12.04.1 LTS I wanted to start a new full backup by generating a new WebDav Folder and adapting the conf file. As the connection sometimes breaks I tried to restart the backup. At the beginning the script finds the secret gpg-key, tests the encrypt and decrypt OK, than starts the BKP command and stops because the passphrase ist false and the private key is not found. During my first tests duply/duplicity generated an own private and public key-file in the configuration folder. So I copied the gpg-key-files in the folder - didn't help with the problem, but now no more key-file ist generated by duply/duplicity.
> Generating and using a new gpg-key doesn't help.
> Making a new backup and a restore of a small data amount without connection break works OK.
> The used versions: duplicity 0.6.18, python 2.7.3, gpg 1.4.11.
>
> Would be happy about a hint.
>

probably same issue as this.
https://answers.launchpad.net/duplicity/+question/183711
try the patch in the end.. it's not in trunk.

btw. 0.6.20 is released with lot's of changes
http://duplicity.nongnu.org/CHANGELOG

@Mike any news on this?

..ede/duply.net

Downloaded the patch: Where to insert the patch code lines (Brock yet did'nt answer)?
Ubuntu LTS philosophy meens waiting for ubuntu-update to btw 0.6.20 - and for somebody filling the days working under Windows it is good to wait ;-)

On 11.11.2012 14:41, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Status: Answered => Open
>
> Vera Schmidt is still having a problem:
> Downloaded the patch: Where to insert the patch code lines (Brock yet did'nt answer)?
> Ubuntu LTS philosophy meens waiting for ubuntu-update to btw 0.6.20 - and for somebody filling the days working under Windows it is good to wait ;-)
>

best to apply it manually. look around line 323 in duplicity executable.

if you are not familiar with procedures like that i suggest you simply delete the last incomplete backup manually. go wherever your backup files end up and delete all files starting with the date of the incomplete backup. you can of course as well simply start a new full and the incomplete incremental will be ignored subsequently.

..ede/duply.net

Thanks edso, that solved my question.

After some smaller problems with python syntax I worked it out. Nevertheless the backup still isn't done as it stopped after 16 volumes with a broken pipe error and now always stops with this error just a the beginning. It seems I have to contact the provider of my online space or my DSL-provider.

to properly assess your specific issue i need more information.

can you run your backup with max verbosity '-v9' and post the complete output to pastebin and the link here? scan and obfuscate private string in it beforehand.

did you try to raise --num-retries already? .. ede/duply.net

OK. In the past verbosity was 4 - changing to 9 is no problem. What do you mean with posting the output to pastebin?
Can't find a way to raise retries in the configuration file conf.

On 02.12.2012 16:01, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Status: Solved => Open
>
> Vera Schmidt is still having a problem:
> OK. In the past verbosity was 4 - changing to 9 is no problem.

you can always add manually parameters to your duply run .. e.g. duply profile backup -v9
this way you can use parameters just one time without making them permanent.

> What do you mean with posting the output to pastebin?

http://pastebin.com/ is an easy way to share text snippets over the internet. you can unfortunately not attach the output as textfile to your question ticket on launchpad. that's only possible with bug tickets.

> Can't find a way to raise retries in the configuration file conf.
>

please read the duplicity manpage
http://duplicity.nongnu.org/duplicity.1.html

you can add '--num-retries number' manually at the end of the duply conf file.

..ede/duply.net

I posted the obfuscated content of the log to pastebin (http://pastebin.com/yc0MgBJ4).

On 08.12.2012 14:41, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Status: Answered => Open
>
> Vera Schmidt is still having a problem:
> I posted the obfuscated content of the log to pastebin
> (http://pastebin.com/yc0MgBJ4).
>

thanks. looks like the retry routine was overly impressed by the socket error.. i wrote a patch that should make retrying in webdav much more robust. please replace your versions of the following files with these

http://bazaar.launchpad.net/~ed.so/duplicity/webdav.fix-retry/view/head:/duplicity/backend.py
http://bazaar.launchpad.net/~ed.so/duplicity/webdav.fix-retry/view/head:/duplicity/backends/webdavbackend.py

and try if it comes up with proper retrying on disrupted socket connections.

..ede/duply.net

Sorry, seems stupid, but I can't find these 2 files, not even a duplicity folder. The duplicity file is in /usr/bin.

On 16.12.2012 15:01, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Status: Answered => Open
>
> Vera Schmidt is still having a problem:
> Sorry, seems stupid, but I can't find these 2 files, not even a
> duplicity folder. The duplicity file is in /usr/bin.
>

probably under
/usr/lib/python<version>/dist-packages/duplicity/

use the locate commandline tool as root if you can't find them
http://www.codecoffee.com/tipsforlinux/articles/20.html

..ede

Path is /usr/lib/python<version>/dist-packages/duplicity/ - the files are to find with locate command not with find command. Never needed root user since I changed to ubuntu in 2010, need to organize the root password before further actions, couldn't find it in the installation document :-(

On 25.12.2012 15:41, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Vera Schmidt posted a new comment:
> Path is /usr/lib/python<version>/dist-packages/duplicity/ - the files
> are to find with locate command not with find command. Never needed root
> user since I changed to ubuntu in 2010, need to organize the root
> password before further actions, couldn't find it in the installation
> document :-(
>

the standard ubuntu user should be able to use 'sudo' .. research how that command works and you should be set.

..ede

Well I do use sudo while working with those things.
Looking again with another account I found this:
webdavbackend.py in
/usr/lib/python2.7/dist-packages/duplicity/backends/webdavbackend.py with 65 Bytes (is a link, showing to the following file)
/usr/share/pyshared/duplicity/backends/webdavbackend.py with 11624 Bytes
The downloaded webdavbackend.py from link further above with 69140 Bytes

backend.py in
/usr/lib/python2.7/dist-packages/UbuntuSystemService/backend.py with 47 Bytes (is a link, showing to the following file)
/usr/share/pyshared/duplicity/backend.py with 21493 Bytes
The downloaded backend.py from link further above with 109746 Bytes

Comparing the files I found out that the contents are totally different. It seems that the download didn't work as I thought it would. So I took the content by copy and paste and will compare and try again.

Wishing you a Happy New Year
and Thanks alot for supporting

Vera

on either of those links you land on a weppage which has the correct download file link on the upper right side.

happy new year.. ede

Doing it right, it works at once ;-)
The download/browser seems to have made javasript files out of the files - however this may be - with copy and paste the contents came out correctly.
I tried the backup again with the changed files reached volume 16 - and ended there like always with a broken pipe. Trying again some minutes, hours or a day later makes no difference. Making the backup with another account doesn't work, too.
The last log is to find in http://pastebin.com/TkamUsgh.
The http-connection to my webdav-account works correctly, alot of space for backups is left.

Vera

On 01.01.2013 11:16, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Status: Answered => Open
>
> Vera Schmidt is still having a problem:
> Doing it right, it works at once ;-)
> The download/browser seems to have made javasript files out of the files - however this may be - with copy and paste the contents came out correctly.
> I tried the backup again with the changed files reached volume 16 - and ended there like always with a broken pipe. Trying again some minutes, hours or a day later makes no difference. Making the backup with another account doesn't work, too.
> The last log is to find in http://pastebin.com/TkamUsgh.
> The http-connection to my webdav-account works correctly, alot of space for backups is left.
>

the paste is private.. can't see it ;( .. ede/duply.net

Sorry, now it's public. Vera

It worked: After 1 and a half day and 2 nights the backup was finished. Log is 8752486 Bytes ;-) Thousand Thanks.

Thanks edso, that solved my question.

On 09.01.2013 20:15, Vera Schmidt wrote:
> Question #212966 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/212966
>
> Status: Answered => Solved
>
> Vera Schmidt confirmed that the question is solved:
> It worked: After 1 and a half day and 2 nights the backup was finished.
> Log is 8752486 Bytes ;-) Thousand Thanks.
>

finally.. glad to hear. thanks for complaining;) ..ede/duply.net