Warn user if files weren't able to be read during backup

Bug #907846 reported by William H. Allen
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Déjà Dup
Fix Released
Medium
Unassigned

Bug Description

I have just replaced a crashed HD. When I loaded my backup, one user lost most of his files.

Detailed process is as follows:
1. Format & Part HD, Install Ubuntu 11.10 w/ online update, install software.
2. Create users. Confirm new UID/GID matches pre-crash UID/GID scheme.
3. Connect USB HD, start Deja Dup
4. Select location, select most recent backup
5. Select Restore
5b. eat lunch
6. Check data
7. One user was missing most of his data.

On further review (using 6 month old manual copy backup), it appears that most files for this user were set with permission (-rwx------). This can not be confirmed because the old manual backup was ugly. During the manual backup process, most of the files from that user were converted to the administrator's UID/GID. I know on the old HD (pre-crash), these files were set with that user's UID/GID.

Some files which were intentionally changed to (-rwxrw----) pre-backup did successfully recover. Other files with the (-rwxrw----) permissions, but which were buried further in the tree, did not recover.

I also noticed that data files and folders created after updating to Ubuntu 11.10 were included in the recovery.

I have several timestamps in my main backup, and an earlier independent backup. I tried the independent backup as well as several of the more recent timestamps. None of the Deja Dup backups were able to recover this data.

I do not know if this is a recovery problem, or an initial backup problem. I do not remember any errors during any of the backups. No errors were reported on recovery.

Description: Ubuntu 11.10
deja-dup: 20.1-0ubuntu0.2
duplicity: 0.6.15-0ubuntu2

Related branches

Revision history for this message
William H. Allen (william-allen) wrote :
Revision history for this message
Michael Terry (mterry) wrote :

What user ran the deja-dup backup? i.e. was this something you ran for other users, selecting their home folders as "folders to back up", or was it something each user ran him/her-self?

Changed in deja-dup:
status: New → Incomplete
Revision history for this message
William H. Allen (william-allen) wrote : Re: [Bug 907846] Re: Data loss on attempted recovery

Michael,

I say it was another user in the visible report, but it was actually
myself. I have one admin account, one personal account, and one business
account.

The backup was for the home folder, which included all three user
accounts. It was run from my admin account. My business & admin file
branches appear to have restored without problem; however, my personal
branch only restored 'Music' and 3 out of 15 folders in 'Documents'.
This means approximately 1400 files & folders were not properly
restored, including pictures, videos, and files of almost every type.

As a side note, I also had no problems recovering a separate backup
which was run from my admin account but which exclusively saved critical
business files (business UID/GID)

Let me know if you need any other information.

W. H. Allen

On 12/22/2011 01:05 PM, Michael Terry wrote:
> What user ran the deja-dup backup? i.e. was this something you ran for
> other users, selecting their home folders as "folders to back up", or
> was it something each user ran him/her-self?
>
> ** Changed in: deja-dup
> Status: New => Incomplete
>

Revision history for this message
Michael Terry (mterry) wrote : Re: Data loss on attempted recovery

Is it possible that the admin user did not have permission to read the missing files in the personal account? I believe Deja Dup does not currently warn the user about its inability to read files during a backup, giving a false sense of security in such a case.

That may be happening here? (I'm working on a patch as I type this to fix that.)

Revision history for this message
William H. Allen (william-allen) wrote : Re: [Bug 907846] Re: Data loss on attempted recovery

That is what I suspected. The admin was part of the user's group;
however, many of the files/branches which did not backup had local user
only permissions (drwx------).

I thought Deja Dup backup operated on a root level. If so, doesn't it
have the ability to copy all files, regardless of permissions?

If not, I am really scared I may have lost some other important files.
Last year, I transferred a large amount of legacy data from another
backup. That is the same time I converted from Win to Ubuntu. I know
many of these files had user only permission after I copied them into
Ubuntu, although I am not sure why.

The sooner you can patch this - the better.

Thanks for the quick response.
W. H. Allen

On 12/23/2011 12:44 AM, Michael Terry wrote:
> Is it possible that the admin user did not have permission to read the
> missing files in the personal account? I believe Deja Dup does not
> currently warn the user about its inability to read files during a
> backup, giving a false sense of security in such a case.
>
> That may be happening here? (I'm working on a patch as I type this to
> fix that.)
>

Revision history for this message
Michael Terry (mterry) wrote : Re: Data loss on attempted recovery

Ah, no. Deja Dup operates as the user running it. So that sounds like the issue here. I'm really sorry. :-/

My patch will make it so that the user is notified after a backup finishes which files did not get backed up due to lack-of-read-permissions.

summary: - Data loss on attempted recovery
+ Warn user if files weren't able to be read during backup
Changed in deja-dup:
importance: Undecided → Medium
status: Incomplete → Triaged
Revision history for this message
William H. Allen (william-allen) wrote : Re: [Bug 907846] Re: Data loss on attempted recovery

Rats.

I think your patch will at least help my setup. I did not intend to make
so many user only files, it was just a result of the legacy data dump.

The warning will at least let me know something is not quite right.

One suggestion - make sure there is a way to handle large numbers or
switch to a summary/count if the details are over a few lines. I had
about 1400 files which were not backed up. I would hate to watch that
many files scroll by in a single screen display.

I hope I was able to help fix this.

Have a good weekend.

On 12/23/2011 01:25 PM, Michael Terry wrote:
> Ah, no. Deja Dup operates as the user running it. So that sounds like
> the issue here. I'm really sorry. :-/
>
> My patch will make it so that the user is notified after a backup
> finishes which files did not get backed up due to lack-of-read-
> permissions.
>
> ** Summary changed:
>
> - Data loss on attempted recovery
> + Warn user if files weren't able to be read during backup
>
> ** Changed in: deja-dup
> Importance: Undecided => Medium
>
> ** Changed in: deja-dup
> Status: Incomplete => Triaged
>

Michael Terry (mterry)
Changed in deja-dup:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.