Duplicity

Errors don't go to logfile

Asked by Rodrigo Alvarez

I'm running duplicity from a script (triggered by a cronjob) and piping the output to a log file:

duplicity my_options source target >> logfile

BTW I'm using -v5 as logging verbosity.

When duplicity finds and error and crashes, my logfile has no information about the crash, however if I manually trigger the exact same script from the terminal, upon crashing duplicity does return a lot of stuff. See the example below.

Not logging these errors to file makes the whole point of logging pointless for debugging.

Is there anything I can do to route these error messages to my log file? Is this a bug?

-- begin example ---
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1236, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1229, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1118, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.6/dist-packages/duplicity/commandline.py", line 865, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/lib/python2.6/dist-packages/duplicity/commandline.py", line 760, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/lib/python2.6/dist-packages/duplicity/backend.py", line 153, in get_backend
    return _backends[pu.scheme](pu)
  File "/usr/lib/python2.6/dist-packages/duplicity/backends/localbackend.py", line 42, in __init__
    self.remote_pathdir = path.Path(parsed_url.path[2:])
  File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 470, in __init__
    self.setdata()
  File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 475, in setdata
    self.stat = os.lstat(self.name)
OSError: [Errno 112] Host is down: '/media/Backup_server/raid_bkup/Home'

--- end example ---

Question information

Language:
English Edit question
Status:
Solved
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Solved by:
edso
Solved:
Last query:
Last reply:
Revision history for this message
edso (ed.so) said : 		#1

duplicity is logging errors to STDERR .. hence using

#>duplicity my_options source target >> logfile 2>&1

will do what you want.
learn more about redirecting for example here
http://www.linuxtopia.org/online_books/advanced_bash_scripting_guide/io-redirection.html

regards ede

Revision history for this message
Kenneth Loafman (kenneth-loafman) said : 		#2

This is normal Unix behavior since you only redirected stdout, not
stderr. To redirect both, use:
  duplicity options src tgt 2>&1 >> logfile
which will redirect both stdout and stderr to the logfile

...Ken

Rodrigo Alvarez wrote:
> Question #109951 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/109951
>
> Description changed to:
> I'm running duplicity from a script (triggered by a cronjob) and piping
> the output to a log file:
>
> duplicity my_options source target >> logfile
>
> BTW I'm using -v5 as logging verbosity.
>
> When duplicity finds and error and crashes, my logfile has no
> information about the crash, however if I manually trigger the exact
> same script from the terminal, upon crashing duplicity does return a lot
> of stuff. See the example below.
>
> Not logging these errors to file makes the whole point of logging
> pointless for debugging.
>
> Is there anything I can do to route these error messages to my log file?
> Is this a bug?
>
>
> -- begin example ---
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1236, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1229, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1118, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
> File "/usr/lib/python2.6/dist-packages/duplicity/commandline.py", line 865, in ProcessCommandLine
> backup, local_pathname = set_backend(args[0], args[1])
> File "/usr/lib/python2.6/dist-packages/duplicity/commandline.py", line 760, in set_backend
> globals.backend = backend.get_backend(bend)
> File "/usr/lib/python2.6/dist-packages/duplicity/backend.py", line 153, in get_backend
> return _backends[pu.scheme](pu)
> File "/usr/lib/python2.6/dist-packages/duplicity/backends/localbackend.py", line 42, in __init__
> self.remote_pathdir = path.Path(parsed_url.path[2:])
> File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 470, in __init__
> self.setdata()
> File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 475, in setdata
> self.stat = os.lstat(self.name)
> OSError: [Errno 112] Host is down: '/media/Backup_server/raid_bkup/Home'
>
> --- end example ---
>
> You received this question notification because you are a member of
> duplicity-team, which is an answer contact for Duplicity.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~duplicity-team
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~duplicity-team
> More help : https://help.launchpad.net/ListHelp
>

Revision history for this message
Best edso (ed.so) said : 		#3

I am pretty sure that order matters here .. first redirect STDOUT, then
redirect STDERR to STDOUT ..meaning
duplicity options src tgt >> logfile 2>&1

which is also what I proposed as answer .. ede
--

On 06.05.2010 22:06, Kenneth Loafman wrote:
> Question #109951 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/109951
>
> Kenneth Loafman proposed the following answer:
> This is normal Unix behavior since you only redirected stdout, not
> stderr. To redirect both, use:
> duplicity options src tgt 2>&1 >> logfile
> which will redirect both stdout and stderr to the logfile
>
> ...Ken
>
> Rodrigo Alvarez wrote:
>> Question #109951 on Duplicity changed:
>> https://answers.launchpad.net/duplicity/+question/109951
>>
>> Description changed to:
>> I'm running duplicity from a script (triggered by a cronjob) and piping
>> the output to a log file:
>>
>> duplicity my_options source target >> logfile
>>
>> BTW I'm using -v5 as logging verbosity.
>>
>> When duplicity finds and error and crashes, my logfile has no
>> information about the crash, however if I manually trigger the exact
>> same script from the terminal, upon crashing duplicity does return a lot
>> of stuff. See the example below.
>>
>> Not logging these errors to file makes the whole point of logging
>> pointless for debugging.
>>
>> Is there anything I can do to route these error messages to my log file?
>> Is this a bug?
>>
>>
>> -- begin example ---
>> Traceback (most recent call last):
>> File "/usr/bin/duplicity", line 1236, in <module>
>> with_tempdir(main)
>> File "/usr/bin/duplicity", line 1229, in with_tempdir
>> fn()
>> File "/usr/bin/duplicity", line 1118, in main
>> action = commandline.ProcessCommandLine(sys.argv[1:])
>> File "/usr/lib/python2.6/dist-packages/duplicity/commandline.py", line 865, in ProcessCommandLine
>> backup, local_pathname = set_backend(args[0], args[1])
>> File "/usr/lib/python2.6/dist-packages/duplicity/commandline.py", line 760, in set_backend
>> globals.backend = backend.get_backend(bend)
>> File "/usr/lib/python2.6/dist-packages/duplicity/backend.py", line 153, in get_backend
>> return _backends[pu.scheme](pu)
>> File "/usr/lib/python2.6/dist-packages/duplicity/backends/localbackend.py", line 42, in __init__
>> self.remote_pathdir = path.Path(parsed_url.path[2:])
>> File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 470, in __init__
>> self.setdata()
>> File "/usr/lib/python2.6/dist-packages/duplicity/path.py", line 475, in setdata
>> self.stat = os.lstat(self.name)
>> OSError: [Errno 112] Host is down: '/media/Backup_server/raid_bkup/Home'
>>
>> --- end example ---
>>
>> You received this question notification because you are a member of
>> duplicity-team, which is an answer contact for Duplicity.
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~duplicity-team
>> Post to : <email address hidden>
>> Unsubscribe : https://launchpad.net/~duplicity-team
>> More help : https://help.launchpad.net/ListHelp
>>
>
> You received this question notification because you are a member of
> duplicity-team, which is an answer contact for Duplicity.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~duplicity-team
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~duplicity-team
> More help : https://help.launchpad.net/ListHelp

Revision history for this message
Rodrigo Alvarez (rodrigo-alvarez-i) said : 		#4

Thank you both!

I only tried

duplicity options src tgt >> logfile 2>&1

And it seems to work like a charm : )

Revision history for this message
Rodrigo Alvarez (rodrigo-alvarez-i) said : 		#5

Thanks edso, that solved my question.

Revision history for this message
mogliii (mogliii) said : 		#6

May I suggest one more

If you use

duplicity src tgt 2>&1 | tee logfile

you will get both errors and normal stdout in the logfile PLUS you see everything on the screen. This is nice when running interactively. You dont have to check the logfile constantly.

http://www.linuxquestions.org/questions/linux-software-2/bash-how-to-redirect-output-to-file-and-still-have-it-on-screen-412611/

Revision history for this message
mogliii (mogliii) said : 		#7

One more correction: use

duplicity src tgt 2>&1 | tee -a logfile

to not overwrite the previous logfile