Duplicity

unable to restore or verify

Asked by Daniel Beck

Aloa,

i want to use duplicity to backup my server data on a secure webdav space. Backing up seems to work fine and i am able to get "collection-status", aswell as "list-current-files". The Webdav Device is mountet via /etc/fstab, so Backup runs like this:

"duplicity /home/Daten/Dokumente file:///home/smartdrive/Backup/Server/Dokumente >>/var/log/duplicity/Dokumente.log"

But if i try to "restore" or "verifiy", i receive the following errors for:
"duplicity restore file:///home/smartdrive/Backup/Server/Dokumente /home/becksta/test"

"Letzte vollständige Sicherung: Thu Feb 7 12:34:20 2013
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1411, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1404, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1338, in main
    restore(col_stats)
  File "/usr/bin/duplicity", line 629, in restore
    restore_get_patched_rop_iter(col_stats)):
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 524, in Write_ROPaths
    ITR( ropath.index, ropath )
  File "/usr/lib/python2.7/dist-packages/duplicity/lazy.py", line 335, in __call__
    last_branch.fast_process, args)
  File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error
    return function(*args)
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 577, in fast_process
    ropath.copy( self.base_path.new_index( index ) )
  File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 426, in copy
    other.writefileobj(self.open("rb"))
  File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 602, in writefileobj
    buf = fin.read(_copy_blocksize)
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 202, in read
    if not self.addtobuffer():
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 227, in addtobuffer
    self.tarinfo_list[0] = self.tar_iter.next()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 334, in next
    self.set_tarfile()
  File "/usr/lib/python2.7/dist-packages/duplicity/patchdir.py", line 323, in set_tarfile
    self.current_fp = self.fileobj_iter.next()
  File "/usr/bin/duplicity", line 665, in get_fileobj_iter
    backup_set.volume_name_dict[vol_num],
KeyError: 50"

Any ideas, what's the problem with that?

regards
Becksta

Question information

Language:
English Edit question
Status:
Answered
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
edso (ed.so) said : 		#1

what's the duplicity version you are using?

0.6.21 is the latest release and i advise you to use it as it has some crucial webdav fixes.

key errors in patchdir.py usually mean there is a volume missing. could you check on your webdav backend? we should write a proper error for that sometime ;9

anyway.. do a new full with 0.6.21 and everything should be peachy.

..ede/duply.net

Revision history for this message
Daniel Beck (daniel-beck) said : 		#2

Hi edso,

and thanks for quick answering :)

I'll try that out. but can a new version including webdav fixes solve my problem, even if my webdav destination is locally mounted and duplicity goes like "file:///home/Server/Backup/....etc"???

regards
Becksta

Revision history for this message
edso (ed.so) said : 		#3

On 08.02.2013 13:21, Daniel Beck wrote:
> Question #221369 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/221369
>
> Status: Answered => Open
>
> Daniel Beck is still having a problem:
> Hi edso,
>
> and thanks for quick answering :)

nP

>
> I'll try that out. but can a new version including webdav fixes solve my
> problem, even if my webdav destination is locally mounted and duplicity
> goes like "file:///home/Server/Backup/....etc"???

no, obviously you are _not_ using the webdav backend. why actually?

do you still have the log of the last full backup? do you see anything in there pointing out that the backup might not have been successful?

..ede/duply.net

Revision history for this message
Daniel Beck (daniel-beck) said : 		#4

Hey again,

actually i didn't get it working directly, because of the fact that my webdav username is an e-mail adress, containig "@".

i tried out like "webdavs://user%40domain.x@server/", but didn't get it working :(

right now, theres a full backup with "v-4" running. If there's still a problem, i'll post the log

regards
becksta

Revision history for this message
edso (ed.so) said : 		#5

On 08.02.2013 13:51, Daniel Beck wrote:
> actually i didn't get it working directly, because of the fact that my
> webdav username is an e-mail adress, containig "@".
>
> i tried out like "webdavs://user%40domain.x@server/", but didn't get it
> working :(

box.net does use email addresses as well for auth, but that works flawlessly on my side. %40 is the correct approach.

e.g.
duplicity collection-status ... 'webdavs://my.name%<email address hidden>/dav/foobar2'

..ede/duply.net

Revision history for this message
Daniel Beck (daniel-beck) said : 		#6

Hm,

it doesn't work for me... but that's not so bad, as long as it works the other way, which it still does not.... :(
i made another full backup, and now i receive input/output errors in case of verify or restore.

duplicity restore -v6 file:///home/smartdrive/Backup/Server/Dokumente/ /home/becksta/backup returns:
"[Errno 5] Input/output error: '/home/smartdrive/Backup/Server/Dokumente/duplicity-full.20130208T113442Z.vol10.difftar.gpg'"

duplicity verify file:///home/smartdrive/Backup/Server/Dokumente/ /home/Daten/Dokumente/ brings:
"[Errno 5] Input/output error: '/home/smartdrive/Backup/Server/Dokumente/duplicity-full.20130208T113442Z.vol72.difftar.gpg'"

the last log entry is like:
"Volumen 9 von 186 verarbeitet
Entfernen der temporären Datei /tmp/duplicity-8Mv2GT-tempdir/mktemp-vnGtvG-11 ist fehlgeschlagen" which means "unable to remove temp file.... "

Revision history for this message
Daniel Beck (daniel-beck) said : 		#7

Hey,

here's another log, trying to backup on webdav like this:

duplicity -v9 /home/Daten/Dokumente/ webdavs://pt47-399%40online.de:<hiddenpassword>@host/Backup/Server/Test --ssl-no-check-certificate

Listing directory /Backup/Server/Test/ on WebDAV server
WebDAV create connection on 'sd2dav.1und1.de' (retry 4)
WebDAV PROPFIND /Backup/Server/Test/ request with headers: {'Connection': 'keep-alive', 'Depth': '1', 'Authorization': 'Basic cHQ0NjQ3LTM2OUBvbmxpbmUuZGU6OGNJVlhiWjQ='}
WebDAV data length: 0
Attempt 4 failed. SSLError: The read operation timed out
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 342, in _retry_fatal
    return fn(self, *args)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/webdavbackend.py", line 300, in list
    raise e
 SSLError: The read operation timed out

Listing directory /Backup/Server/Test/ on WebDAV server
WebDAV create connection on 'sd2dav.1und1.de' (retry 5)
WebDAV PROPFIND /Backup/Server/Test/ request with headers: {'Connection': 'keep-alive', 'Depth': '1', 'Authorization': 'Basic cHQ0NjQ3LTM2OUBvbmxpbmUuZGU6OGNJVlhiWjQ='}
WebDAV data length: 0
Giving up after 5 attempts. SSLError: The read operation timed out
Entferne temporäre Datei /tmp/duplicity-JJw9M5-tempdir/mkstemp-cHTa2r-1

Can u guys help me with that??? i think it's better to try, getting it done strict via webdav instead of mounting it local and backup up then, the errors above (to me) seem to implicate upload errors, right?

regards
Becksta

Revision history for this message
edso (ed.so) said : 		#8

On 09.02.2013 09:55, Daniel Beck wrote:
> Question #221369 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/221369
>
> Daniel Beck gave more information on the question:
> Hey,
>
> here's another log, trying to backup on webdav like this:
>
 [SNIP]
>
> Can u guys help me with that??? i think it's better to try, getting it
> done strict via webdav instead of mounting it local and backup up then,

correct

> the errors above (to me) seem to implicate upload errors, right?
>

actually no, the first listing already fails because of a timeout. probably the webserver takes it's time to answer.

first try to raise --num-retries .. maybe 1und1 is simply very slow and you need to retry more often.

secondly.. next time please attach (zipped) the _full_ max verbose '-v9' output but obfuscate _all_ private data, especially the webdav auth data (which you just sent hashed over the list, you should change your password now, just to be sure).

.ede/duply.net

Revision history for this message
edso (ed.so) said : 		#9

On 08.02.2013 23:25, Daniel Beck wrote:
> Question #221369 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/221369
>
> Status: Answered => Open
>
> Daniel Beck is still having a problem:
> Hm,
>
> it doesn't work for me... but that's not so bad, as long as it works the other way, which it still does not.... :(
> i made another full backup, and now i receive input/output errors in case of verify or restore.
>
> duplicity restore -v6 file:///home/smartdrive/Backup/Server/Dokumente/ /home/becksta/backup returns:
> "[Errno 5] Input/output error: '/home/smartdrive/Backup/Server/Dokumente/duplicity-full.20130208T113442Z.vol10.difftar.gpg'"
>
> duplicity verify file:///home/smartdrive/Backup/Server/Dokumente/ /home/Daten/Dokumente/ brings:
> "[Errno 5] Input/output error: '/home/smartdrive/Backup/Server/Dokumente/duplicity-full.20130208T113442Z.vol72.difftar.gpg'"
>
> the last log entry is like:
> "Volumen 9 von 186 verarbeitet
> Entfernen der temporären Datei /tmp/duplicity-8Mv2GT-tempdir/mktemp-vnGtvG-11 ist fehlgeschlagen" which means "unable to remove temp file.... "
>

sorry, need your full debug output to interpretr what happened here... ede/duply.net

Can you help with this problem?

Provide an answer of your own, or ask Daniel Beck for more information if necessary.

To post a message you must log in.