"resume later" doesn't appear to work when creating a fresh backup

Asked by Ross Gayler

I have been using Deja Dup for about three months to back up a linux laptop over a LAN to a windows share on a NAS. The total amount to be backed up is about 75Gb and about half of that is one virtualbox .vdi file. The backup fails every now and then because of a connection timeout. This was most noticeable when I did the initial full backup. I just kept restarting the backup and it eventually finished.

Today when I started the backup Deja Dup said it was doing a "fresh backup" to avoid corruption. The backup has failed four or five times so far due to connection timeouts. Each time I restarted the backup DD deleted all the files it had uploaded on the previous failed run. So I looked on the support forum and found a question from a couple of years ago where someone was having the same problem of failure of long backups and the recommended solution was to use the "resume later" button to break the backup into shorter sessions.

I tried using the "resume later" button - the backup session stopped immediately and I could see the files that had been uploaded to the share, but when I started the next backup (expecting it to resume from the same point), DD deleted the files it had previously uploaded and started from the beginning again. Can anyone explain what is happening here and suggest a fix?

Thanks

Ross

Question information

Revision history for this message
Hans Harhoff Andersen (hansharhoff) said :
#1

I have seen similar issues. Deja-dup (DD) will run for a long time (backing up to Ubuntu One about 50 GB so it is taking a while). Since this is a laptop I have to put it to sleep and move around thereby loosing connection to Ubuntu One and interrupting DD.

After I turn on my laptop again I go to DD preferences which says that last backup was today(!), but I just pressed the resume later button so I do not trust this indication. Is there some way to determine whether DD is lying to me. What I would like is an indication of which complete/full backup are at the remote location and which incomplete backups are in progress. An incomplete backup which claims to be full is worse than no backup at all...
Are there logs (maybe inside of the GUi) that can inform me about what deja-dup has done "in the past".

I have tried just pressing the "backup now" button but I am unsure what specifically this button does. Does it start a new full backup, a new incremental backup or does it just "resume later" which is what I want.

Also. Does DD resume transfers of large files midway? I have some lecture movie clips that are ~ 4 GB that seem to stall the backup.

Revision history for this message
Michael Terry (mterry) said :
#2

Let me do some testing to confirm or shed light on the OP problem.

Hans, "Back Up Now" will resume a previous backup like you want. It will resume transfers of large files midway.

Revision history for this message
Vv (vivien-perez) said :
#3

Hello,

I've got the same problem.

DD starts and tells me it needs to do a full backup to prevent data corruption on the server. If I click on "Resume later", the next time DD starts, it will resume this full backup from scratch.

This is annoying because I rarely let DD run long enough (>12 hours) to do the full backup at once.

Is there a way to force DD to resume from where it stopped last time?

Thx,

Vivien

Revision history for this message
Launchpad Janitor (janitor) said :
#4

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Ross Gayler (r-gayler) said :
#5

The problem has recurred - this time on a standard back up, not a "fresh backup to avoid corruption". The first 5 attempts at backing up failed due to connection timeout. When I checked the backup directory I could see that files had been added. When I restarted the backup with the "Backup now" button all the previously added files were deleted and the backup started from the beginning again. Here is a rough trace of the status messages from Deja Dup for the last (successful) run:

(16,084 files in backup directory - starting state)
[failed backup]
(16,212 files in backup directory)
"Backup now"
"Preparing"
"Cleaning up"
(16,084 files in backup - the previously added files have been deleted and the backup directory taken back to the initial state)
"Preparing"
"Password"
"Preparing"
"Cleaning up"
"Preparing"
"Scanning"
"Backing up"
"Uploading"
"Backing up"
"Uploading"
etc.

It took me most of a day to do a 1 hour backup.

Revision history for this message
Hans Harhoff Andersen (hansharhoff) said :
#6

The same thing happned to me a couple of times. I am still unable to make a
single successful backup and deja-dup refuses to recognize the partial
backups when I try to press restore.

On Sun, Dec 11, 2011 at 11:11 PM, Ross Gayler <
<email address hidden>> wrote:

> Question #179453 on Déjà Dup changed:
> https://answers.launchpad.net/deja-dup/+question/179453
>
> Status: Expired => Open
>
> Ross Gayler is still having a problem:
> The problem has recurred - this time on a standard back up, not a "fresh
> backup to avoid corruption". The first 5 attempts at backing up failed
> due to connection timeout. When I checked the backup directory I could
> see that files had been added. When I restarted the backup with the
> "Backup now" button all the previously added files were deleted and the
> backup started from the beginning again. Here is a rough trace of the
> status messages from Deja Dup for the last (successful) run:
>
> (16,084 files in backup directory - starting state)
> [failed backup]
> (16,212 files in backup directory)
> "Backup now"
> "Preparing"
> "Cleaning up"
> (16,084 files in backup - the previously added files have been deleted and
> the backup directory taken back to the initial state)
> "Preparing"
> "Password"
> "Preparing"
> "Cleaning up"
> "Preparing"
> "Scanning"
> "Backing up"
> "Uploading"
> "Backing up"
> "Uploading"
> etc.
>
> It took me most of a day to do a 1 hour backup.
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
>

--
Hans Harhoff Andersen
PhD-student - Ion Trap Group
Department of Physics and Astronomy
University of Aarhus
Ny Munkegade 120, office 1525-224
8000 Aarhus - Denmark

T +45 8942 3532
M +45 28990982

Revision history for this message
Ross Gayler (r-gayler) said :
#7

This posting is just to set the status back to "unanswered". Hans' preceding post confirms the problem; it doesn't solve it.

Revision history for this message
Ross Gayler (r-gayler) said :
#8

I still have the problem:

* "backup now" over the LAN to a NAS
* DD says it is creating a fresh backup to avoid corruption
* 4 hours of backing up later DD fails with a connection timeout
* "backup now" (hoping that DD will resume from where it failed)
* DD goes through "preparing", "password", "preparing", "cleaning up", etc. (when it gets to "cleaning up" it deletes all the files it created in the first run and starts from the beginning again)

I played that game a few times and decided to take a different approach: I used DD to back up to a USB-attached external drive then copied the backup-files from the USB drive to the NAS

Revision history for this message
Stuart Bishop (stub) said :
#9

I've opened Bug #1210499 on this issue, as the only other related bug is FIX RELEASED yet I'm still seeing this behavior with Ubuntu 13.04

Revision history for this message
Launchpad Janitor (janitor) said :
#10

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Claudinho (boedosoy) said :
#11

I am having the same problem. It looks like nobody is maintaining this app. Any alternative for backing up on a cloud? (I was looking but this seems to be a pending assignment for Linux in general).

Revision history for this message
Ross Gayler (r-gayler) said :
#12

OP here. Sorry - I can't help. I haven't needed to resume for years, so I haven't been checking whether the problem still exists.

FWIW I use deja-dup weekly to backup my machines over a gigabit LAN to NAS. I don't get connection time-outs any more. I also have a continuous sync of my working directories to cloud storage running via Tresorit (because I want end-to-end encryption). I'm not overly impressed with the Tresorit linux client. I will probably try out Spider Oak some time.

Revision history for this message
Claudinho (boedosoy) said :
#13

I may have to write a script myself. I normally use rsync but
that was not possible on the cloud, that is why I tried Deja-dup.

BTW, I have recently looked into the cloud services and
decided to go with pCloud over Spider Oak, check it out if
you have not yet.

Regards,
Claudinho

On Wednesday, October 12, 2016 2:08:59 AM PDT you wrote:
> Question #179453 on Déjà Dup changed:
> https://answers.launchpad.net/deja-dup/+question/179453
>
> Ross Gayler posted a new comment:
> OP here. Sorry - I can't help. I haven't needed to resume for years, so
> I haven't been checking whether the problem still exists.
>
> FWIW I use deja-dup weekly to backup my machines over a gigabit LAN to
> NAS. I don't get connection time-outs any more. I also have a continuous
> sync of my working directories to cloud storage running via Tresorit
> (because I want end-to-end encryption). I'm not overly impressed with
> the Tresorit linux client. I will probably try out Spider Oak some time.
>
>

Revision history for this message
Martin (martin3000) said :
#14

You can do your backups with "duplicity".

1. duplicity collection-status <destination>
    example: duplicity collection-status s3+http://myname/dejadup

2. duplicity full --include=/home/jms/.cache/deja-dup/metadata --exclude=/home/jms/temp --exclude=/home/jms/.gvfs --exclude=/home/jms/.dbus --exclude=/home/jms/.gradle --exclude=/home/jms/.PlayOnLinux --exclude=/home/jms/Music --exclude=/home/jms/Videos --exclude=/home/jms/.android --exclude=/home/jms/.cache --exclude=/home/jms/Downloads --exclude=/home/jms/.local/share/Trash --exclude=/home/jms/.cache/deja-dup/tmp --exclude=/home/jms/.xsession-errors --exclude=/home/jms/.thumbnails --exclude=/home/jms/.gvfs --exclude=/home/jms/.cache/deja-dup --exclude=/home/jms/.cache --include=/home/jms --exclude=/sys --exclude=/run --exclude=/proc --exclude=/var/tmp --exclude=/tmp --exclude=** --s3-use-new-style --volsize=50 --no-encryption --verbosity=9 --gpg-options=--no-use-agent --archive-dir=/home/jms/.cache/deja-dup --tempdir=/home/jms/.cache/deja-dup/tmp / <destination>

Revision history for this message
Ari Torhamo (ari-torhamo) said :
#15

I'm having this problem too. I'm trying to do my first backup, and when I try to resume the backup process after pressing "Resume later", Déjà Dup always deletes the already existing files.