Ubuntu 20.04.2 LTS - Deja-Dup Backup Restore - "End of File Error"

Asked by Anand Manthena

I am a relatively new user of Ubuntu. I am in the process of getting familiarized to its functioning and the ways and means of troubleshooting commonly faced issues. I faced a unique problem as described below:

Narration of the problem: The main /Home folder and its contents of my Ubuntu 20.04.2 device disappeared from its original location due to sudden power failure.

My present problem:

I am unable to restore the /Home folder and its contents with Deja-Dup backup files, fortunately available in an external HDD partition. The backed up content has an intact manifest file; and, with 192 volumes - all "full" tar.gz backups {no incremental backups, I presume, because I found no file with 'inc' figuring in the filenames} spread over the period from 2021/07/15 to 2021/07/20.

I ran the Duplicity Backup tool to restore the backups of a particular date (2021-07-15) from the Storage location (external HDD partition) to the non-root location of restoration: /tmp/restore (where 'restore' is the name of the tmp directory created with the terminal command: $ make dir -p /tmp/restore) in the main Ubuntu 20.04.2 device. After the Backup tool worked for a while, I encountered the following "End-of-File Error, and the restore process stopped with the following alert:

"Failed to read /tmp/duplicity-m8zm8_uz-tempdir/mktemp-pmxky8w_-1: (<class 'EOFError'>, EOFEerror('Compressed file ended before the end-of-stream marker was reached'), <traceback object at 0X7f10c2d57200>)

I tried to repeat the same exercise with a Folder with a different name created in the main Ubuntu (blank) /Home location. The same error (with a different traceback object description) as above repeated again.

How do I proceed further to retrieve my lost /Home folder and its contents?

Question information

Language:
English Edit question
Status:
Open
For:
Déjà Dup Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Anand Manthena (manan49) said :
#1

Please read typo error "make dir" wrongly entered. Actually executed mkdir only.

Revision history for this message
Anand Manthena (manan49) said :
#2

The terminal output of the command $ sudo duplicity collection-status file://media/Ubuntu_Backup reads as follows:

Connecting with backend: BackendWrapper

Archive dir: /root/.cache/duplicity/f317f7b99b723af52c1b43300afc3a05
Found 0 secondary backup chains
No backup chains with active signatures found
No orphaned or incomplete backup sets found

Revision history for this message
Michael Terry (mterry) said :
#3

A small caution: you wrote that you ran "sudo duplicity collection-status file://media/Ubuntu_Backup"

But you are missing a slash there. It should be "sudo duplicity collection-status file:///media/Ubuntu_Backup"

The command you ran was looking at the relative directory ./media/Ubuntu_Backup - which duplicity created and then decided there was nothing in it (correctly).

What happens when you run the corrected version with three slashes?

Revision history for this message
Anand Manthena (manan49) said :
#4

Thank you for your prompt response, Mr. Michael Terry.

The terminal has given the following output with $ *sudo duplicity
collection-status file///media/Ubuntu_Backup*:

Bad URL ' file///media/Ubuntu_Backup'. Examples of URL strings are "scp://
<email address hidden>:1234/path" and "file:///usr/local". See the man page for more
information.

Waiting for your support for troubleshooting the "Restore with Deja-Dup
failure problem".

Regards,

Anand

On Sat, 24 Jul, 2021, 14:02 Michael Terry, <
<email address hidden>> wrote:

> Your question #698099 on Déjà Dup changed:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> Status: Open => Needs information
>
> Michael Terry requested more information:
> A small caution: you wrote that you ran "sudo duplicity collection-
> status file://media/Ubuntu_Backup"
>
> But you are missing a slash there. It should be "sudo duplicity
> collection-status file:///media/Ubuntu_Backup"
>
> The command you ran was looking at the relative directory
> ./media/Ubuntu_Backup - which duplicity created and then decided there
> was nothing in it (correctly).
>
> What happens when you run the corrected version with three slashes?
>
> --
> To answer this request for more information, you can either reply to
> this email or enter your reply at the following page:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> You received this question notification because you asked the question.
>

Revision history for this message
Manfred Hampl (m-hampl) said :
#5

Now the colon is missing in the file name

Revision history for this message
Anand Manthena (manan49) said :
#6

Further;
While Deja-Dup backup path to the 3rd partition (labelled "Ubuntu_Backup")
of an external, usb-mounted HDD has taken place normally during the period
from 15/07/21 to 20/07/21 (without encryption password), *thereafter, it is
asking for encryption password*.
The terminal output of the command for reviewing permissions in the media
folder reads as follows:

anand@ubuntu-focal:~/media$ ls -alF

total 12

drwxr-xr-x 3 root root 4096 Jul 21 18:17 ./

drwxrwxr-x 20 anand anand 4096 Jul 24 07:59 ../

*drwxr-xr-x 2 root root 4096 Jul 21 18:17 Ubuntu_Backup/*

in which *"Ubuntu_Backup"* folder (same name as that of the 3rd partition
of the HDD on usb) is the location for restoration of the backups in the
'media' folder (last line in bold in the above terminal output).

Looks like my troubleshooting efforts as of 21/07/21 have landed up with
*/media/Ubuntu_Backup* with root permissions, which is not warranted, I
suppose; hence, the failure to "Restore" backups to their original
locations from the external HDD; is my assessment in order?

Waiting for your support to resolve the issue,

Regards,

Anand

---------- Forwarded message ---------
From: Anand Manthena <email address hidden>
Date: Sun, Jul 25, 2021 at 4:30 PM
Subject: Re: [Question #698099]: Ubuntu 20.04.2 LTS - Deja-Dup Backup
Restore - "End of File Error"
To: <email address hidden>

Thank you for your prompt response, Mr. Michael Terry.

The terminal has given the following output with $ *sudo duplicity
collection-status file:///media/Ubuntu_Backup*:

QUOTE:

*anand@ubuntu-focal:~/media$ sudo duplicity collection-status
file:///media/Ubuntu_BackupFace detection timeout reached[sudo] password
for anand: Last full backup date: Thu Jul 15 13:39:28 2021Collection
Status-----------------Connecting with backend: BackendWrapperArchive dir:
/root/.cache/duplicity/3b114cf101f5065ce0efb18fabe233e9Found 0 secondary
backup chains.Found primary backup chain with matching signature
chain:-------------------------Chain start time: Thu Jul 15 13:39:28
2021Chain end time: Tue Jul 20 12:32:02 2021Number of contained backup
sets: 6Total number of contained volumes: 579 Type of backup set:
                 Time: Num volumes: Full Thu
Jul 15 13:39:28 2021 574 Incremental Fri Jul
16 13:54:30 2021 1 Incremental Sat Jul 17
06:46:49 2021 1 Incremental Sun Jul 18
11:59:29 2021 1 Incremental Mon Jul 19
17:52:53 2021 1 Incremental Tue Jul 20
12:32:02 2021 1-------------------------No orphaned or
incomplete backup sets found.*
UNQUOTE
Waiting for your support for troubleshooting the "Restore with Deja-Dup
failure problem".

Regards,

Anand

On Sat, 24 Jul, 2021, 14:02 Michael Terry, <
<email address hidden>> wrote:

> Your question #698099 on Déjà Dup changed:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> Status: Open => Needs information
>
> Michael Terry requested more information:
> A small caution: you wrote that you ran "sudo duplicity collection-
> status file://media/Ubuntu_Backup"
>
> But you are missing a slash there. It should be "sudo duplicity
> collection-status file:///media/Ubuntu_Backup"
>
> The command you ran was looking at the relative directory
> ./media/Ubuntu_Backup - which duplicity created and then decided there
> was nothing in it (correctly).
>
> What happens when you run the corrected version with three slashes?
>
> --
> To answer this request for more information, you can either reply to
> this email or enter your reply at the following page:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> You received this question notification because you asked the question.
>

Revision history for this message
Anand Manthena (manan49) said :
#7

Now Deja-Dup is neither backing up (after 21/07/21 - asking for encryption
password) nor restoring the /home folder, which disappeared due to sudden
power failure.

Anand

On Sun, 25 Jul, 2021, 18:31 Anand Manthena, <
<email address hidden>> wrote:

> Your question #698099 on Déjà Dup changed:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> Status: Answered => Open
>
> You are still having a problem:
> Further;
> While Deja-Dup backup path to the 3rd partition (labelled "Ubuntu_Backup")
> of an external, usb-mounted HDD has taken place normally during the period
> from 15/07/21 to 20/07/21 (without encryption password), *thereafter, it is
> asking for encryption password*.
> The terminal output of the command for reviewing permissions in the media
> folder reads as follows:
>
> anand@ubuntu-focal:~/media$ ls -alF
>
> total 12
>
> drwxr-xr-x 3 root root 4096 Jul 21 18:17 ./
>
> drwxrwxr-x 20 anand anand 4096 Jul 24 07:59 ../
>
> *drwxr-xr-x 2 root root 4096 Jul 21 18:17 Ubuntu_Backup/*
>
> in which *"Ubuntu_Backup"* folder (same name as that of the 3rd partition
> of the HDD on usb) is the location for restoration of the backups in the
> 'media' folder (last line in bold in the above terminal output).
>
> Looks like my troubleshooting efforts as of 21/07/21 have landed up with
> */media/Ubuntu_Backup* with root permissions, which is not warranted, I
> suppose; hence, the failure to "Restore" backups to their original
> locations from the external HDD; is my assessment in order?
>
> Waiting for your support to resolve the issue,
>
> Regards,
>
> Anand
>
> ---------- Forwarded message ---------
> From: Anand Manthena <email address hidden>
> Date: Sun, Jul 25, 2021 at 4:30 PM
> Subject: Re: [Question #698099]: Ubuntu 20.04.2 LTS - Deja-Dup Backup
> Restore - "End of File Error"
> To: <email address hidden>
>
>
> Thank you for your prompt response, Mr. Michael Terry.
>
> The terminal has given the following output with $ *sudo duplicity
> collection-status file:///media/Ubuntu_Backup*:
>
> QUOTE:
>
>
>
>
>
>
>
>
>
>
>
>
>
> *anand@ubuntu-focal:~/media$ sudo duplicity collection-status
> file:///media/Ubuntu_BackupFace detection timeout reached[sudo] password
> for anand: Last full backup date: Thu Jul 15 13:39:28 2021Collection
> Status-----------------Connecting with backend: BackendWrapperArchive dir:
> /root/.cache/duplicity/3b114cf101f5065ce0efb18fabe233e9Found 0 secondary
> backup chains.Found primary backup chain with matching signature
> chain:-------------------------Chain start time: Thu Jul 15 13:39:28
> 2021Chain end time: Tue Jul 20 12:32:02 2021Number of contained backup
> sets: 6Total number of contained volumes: 579 Type of backup set:
> Time: Num volumes: Full Thu
> Jul 15 13:39:28 2021 574 Incremental Fri Jul
> 16 13:54:30 2021 1 Incremental Sat Jul 17
> 06:46:49 2021 1 Incremental Sun Jul 18
> 11:59:29 2021 1 Incremental Mon Jul 19
> 17:52:53 2021 1 Incremental Tue Jul 20
> 12:32:02 2021 1-------------------------No orphaned or
> incomplete backup sets found.*
> UNQUOTE
> Waiting for your support for troubleshooting the "Restore with Deja-Dup
> failure problem".
>
> Regards,
>
> Anand
>
>
> On Sat, 24 Jul, 2021, 14:02 Michael Terry, <
> <email address hidden>> wrote:
>
> > Your question #698099 on Déjà Dup changed:
> > https://answers.launchpad.net/deja-dup/+question/698099
> >
> > Status: Open => Needs information
> >
> > Michael Terry requested more information:
> > A small caution: you wrote that you ran "sudo duplicity collection-
> > status file://media/Ubuntu_Backup"
> >
> > But you are missing a slash there. It should be "sudo duplicity
> > collection-status file:///media/Ubuntu_Backup"
> >
> > The command you ran was looking at the relative directory
> > ./media/Ubuntu_Backup - which duplicity created and then decided there
> > was nothing in it (correctly).
> >
> > What happens when you run the corrected version with three slashes?
> >
> > --
> > To answer this request for more information, you can either reply to
> > this email or enter your reply at the following page:
> > https://answers.launchpad.net/deja-dup/+question/698099
> >
> > You received this question notification because you asked the question.
> >
>
> --
> You received this question notification because you asked the question.
>

Revision history for this message
Michael Terry (mterry) said :
#8

OK, so the collection-status run showed that there's a backup there, that's good.

I'm a little confused about whether the backup is encrypted or not though. Looks like the backups from July 16->20 are encrypted (the ones returned from collection-status). But your initial message indicated that your backup files were all tar.gz files, which would not be encrypted.

Maybe somehow you have a mix of encrypted and non-encrypted, and that is confusing duplicity?

If you run the following command, do you get any useful files out of it, in the /tmp/restored directory?

duplicity restore file:///media/Ubuntu_Backup /tmp/restored

Revision history for this message
Anand Manthena (manan49) said :
#9

The output of the suggested command $ duplicity restore
file:///media/Ubuntu_Backup /tmp/restored
reads as follows:
*Local and Remote metadata are synchronized, no sync needed.*
*Last full backup date: none*
*GnuPG passphrase for decryption:*
I didn't get any useful files out of the above command (though
collection-status output displayed "full" 574 volumes which are *difftar.gz*
files (which I suppose are encrypted files), and 5 "inc"/incremental
volumes, totalling to 579 volumes).

On Mon, Jul 26, 2021 at 2:11 AM Michael Terry <
<email address hidden>> wrote:

> Your question #698099 on Déjà Dup changed:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> Status: Open => Needs information
>
> Michael Terry requested more information:
> OK, so the collection-status run showed that there's a backup there,
> that's good.
>
> I'm a little confused about whether the backup is encrypted or not
> though. Looks like the backups from July 16->20 are encrypted (the ones
> returned from collection-status). But your initial message indicated
> that your backup files were all tar.gz files, which would not be
> encrypted.
>
> Maybe somehow you have a mix of encrypted and non-encrypted, and that is
> confusing duplicity?
>
> If you run the following command, do you get any useful files out of it,
> in the /tmp/restored directory?
>
> duplicity restore file:///media/Ubuntu_Backup /tmp/restored
>
> --
> To answer this request for more information, you can either reply to
> this email or enter your reply at the following page:
> https://answers.launchpad.net/deja-dup/+question/698099
>
> You received this question notification because you asked the question.
>

Can you help with this problem?

Provide an answer of your own, or ask Anand Manthena for more information if necessary.

To post a message you must log in.