Task 'BKP' failed with exit code '30'

Asked by juniorfga on 2016-10-10

Hello

I have an issue after update to Centos 7 x64, this is the error on the ssh session where you run the backup command:

root@s1 [~]# /usr/bin/duply cpanel_s3 backup 2>&1 >> /var/log/duply_ispconfig.log
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1546, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1540, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1375, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib64/python2.7/site-packages/duplicity/commandline.py", line 1117, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/lib64/python2.7/site-packages/duplicity/commandline.py", line 1006, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/lib64/python2.7/site-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib64/python2.7/site-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib64/python2.7/site-packages/duplicity/backends/_boto_single.py", line 166, in __init__
    self.resetConnection()
  File "/usr/lib64/python2.7/site-packages/duplicity/backends/_boto_single.py", line 188, in resetConnection
    self.conn = get_connection(self.scheme, self.parsed_url, self.storage_uri)
  File "/usr/lib64/python2.7/site-packages/duplicity/backends/_boto_single.py", line 99, in get_connection
    is_secure=(not globals.s3_unencrypted_connection))
  File "/usr/lib/python2.7/site-packages/boto/storage_uri.py", line 117, in connect
    **connection_args)
  File "/usr/lib/python2.7/site-packages/boto/s3/connection.py", line 191, in __init__
    validate_certs=validate_certs, profile_name=profile_name)
  File "/usr/lib/python2.7/site-packages/boto/connection.py", line 569, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/usr/lib/python2.7/site-packages/boto/auth.py", line 991, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))
NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV1Handler'] Check your credentials

09:36:59.434 Task 'BKP' failed with exit code '30'.

this is the result on the log file:

Start duply v1.11.3, time is 2016-10-10 09:39:19.
Using profile '/etc/duply/cpanel_s3'.
Using installed duplicity version 0.7.10, python 2.7.5, gpg 2.0.22 (Home: ~/.gnupg), awk 'GNU Awk 4.0.2', grep 'grep (GNU grep) 2.20', bash '4.2.46(1)-release (x86_64-redhat-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry 'XXXXXXXX' for signing.
Checking TEMP_DIR '/tmp' is a folder and writable (OK)
Test - Encrypt to 'XXXXXXXX','XXXXXXXX','XXXXXXXX' & Sign with 'XXXXXXXX' (OK)
Test - Decrypt (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.21923.1476106760_*'(OK)

--- Start running command PRE at 09:39:20.309 ---
Running '/etc/duply/cpanel_s3/pre' - OK
Output: Comenzando el respaldo de las bases de datos
Puedes revisar el log en /root/SK-BackupLog.txt y el errorlog en /tmp/skdump_errorlog
Se respaldaron 69 Bases de datos
--- Finished state OK at 09:39:42.424 - Runtime 00:00:22.114 ---

--- Start running command BKP at 09:39:42.439 ---
Using archive dir: /root/.cache/duplicity/duply_cpanel_s3
Using backup name: duply_cpanel_s3
Import of duplicity.backends.acdclibackend Succeeded
Import of duplicity.backends.azurebackend Succeeded
Import of duplicity.backends.b2backend Succeeded
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.copycombackend Succeeded
Import of duplicity.backends.dpbxbackend Failed: No module named dropbox
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.giobackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.hubicbackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.lftpbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.mediafirebackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.multibackend Succeeded
Import of duplicity.backends.ncftpbackend Succeeded
Import of duplicity.backends.onedrivebackend Succeeded
Import of duplicity.backends.par2backend Succeeded
Import of duplicity.backends.pydrivebackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.ssh_paramiko_backend Succeeded
Import of duplicity.backends.ssh_pexpect_backend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.sxbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Using temporary directory /tmp/duplicity-A6tpL8-tempdir
--- Finished state FAILED 'code 30' at 09:39:43.748 - Runtime 00:00:01.309 ---

--- Start running command POST at 09:39:43.773 ---
Running '/etc/duply/cpanel_s3/post' - OK
--- Finished state OK at 09:39:43.791 - Runtime 00:00:00.018 ---

The amazon S3 bucket is clean, no previous backup on it, could you help me please?

regards

Question information

Language:
English Edit question
Status:
Solved
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Solved by:
juniorfga
Solved:
2016-11-01
Last query:
2016-11-01
Last reply:
2016-10-11

Could you verify which version of boto you are using?

Also, what is the full command line you are using?

On Mon, Oct 10, 2016 at 9:08 AM, juniorfga <
<email address hidden>> wrote:

> New question #402905 on Duplicity:
> https://answers.launchpad.net/duplicity/+question/402905
>
> Hello
>
> I have an issue after update to Centos 7 x64, this is the error on the ssh
> session where you run the backup command:
>
> root@s1 [~]# /usr/bin/duply cpanel_s3 backup 2>&1 >>
> /var/log/duply_ispconfig.log
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1546, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1540, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1375, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
> File "/usr/lib64/python2.7/site-packages/duplicity/commandline.py",
> line 1117, in ProcessCommandLine
> backup, local_pathname = set_backend(args[0], args[1])
> File "/usr/lib64/python2.7/site-packages/duplicity/commandline.py",
> line 1006, in set_backend
> globals.backend = backend.get_backend(bend)
> File "/usr/lib64/python2.7/site-packages/duplicity/backend.py", line
> 223, in get_backend
> obj = get_backend_object(url_string)
> File "/usr/lib64/python2.7/site-packages/duplicity/backend.py", line
> 209, in get_backend_object
> return factory(pu)
> File "/usr/lib64/python2.7/site-packages/duplicity/backends/_boto_single.py",
> line 166, in __init__
> self.resetConnection()
> File "/usr/lib64/python2.7/site-packages/duplicity/backends/_boto_single.py",
> line 188, in resetConnection
> self.conn = get_connection(self.scheme, self.parsed_url,
> self.storage_uri)
> File "/usr/lib64/python2.7/site-packages/duplicity/backends/_boto_single.py",
> line 99, in get_connection
> is_secure=(not globals.s3_unencrypted_connection))
> File "/usr/lib/python2.7/site-packages/boto/storage_uri.py", line 117,
> in connect
> **connection_args)
> File "/usr/lib/python2.7/site-packages/boto/s3/connection.py", line
> 191, in __init__
> validate_certs=validate_certs, profile_name=profile_name)
> File "/usr/lib/python2.7/site-packages/boto/connection.py", line 569,
> in __init__
> host, config, self.provider, self._required_auth_capability())
> File "/usr/lib/python2.7/site-packages/boto/auth.py", line 991, in
> get_auth_handler
> 'Check your credentials' % (len(names), str(names)))
> NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were
> checked. ['HmacAuthV1Handler'] Check your credentials
>
> 09:36:59.434 Task 'BKP' failed with exit code '30'.
>
> this is the result on the log file:
>
> Start duply v1.11.3, time is 2016-10-10 09:39:19.
> Using profile '/etc/duply/cpanel_s3'.
> Using installed duplicity version 0.7.10, python 2.7.5, gpg 2.0.22 (Home:
> ~/.gnupg), awk 'GNU Awk 4.0.2', grep 'grep (GNU grep) 2.20', bash
> '4.2.46(1)-release (x86_64-redhat-linux-gnu)'.
> Autoset found secret key of first GPG_KEY entry 'XXXXXXXX' for signing.
> Checking TEMP_DIR '/tmp' is a folder and writable (OK)
> Test - Encrypt to 'XXXXXXXX','XXXXXXXX','XXXXXXXX' & Sign with 'XXXXXXXX'
> (OK)
> Test - Decrypt (OK)
> Test - Compare (OK)
> Cleanup - Delete '/tmp/duply.21923.1476106760_*'(OK)
>
> --- Start running command PRE at 09:39:20.309 ---
> Running '/etc/duply/cpanel_s3/pre' - OK
> Output: Comenzando el respaldo de las bases de datos
> Puedes revisar el log en /root/SK-BackupLog.txt y el errorlog en
> /tmp/skdump_errorlog
> Se respaldaron 69 Bases de datos
> --- Finished state OK at 09:39:42.424 - Runtime 00:00:22.114 ---
>
> --- Start running command BKP at 09:39:42.439 ---
> Using archive dir: /root/.cache/duplicity/duply_cpanel_s3
> Using backup name: duply_cpanel_s3
> Import of duplicity.backends.acdclibackend Succeeded
> Import of duplicity.backends.azurebackend Succeeded
> Import of duplicity.backends.b2backend Succeeded
> Import of duplicity.backends.botobackend Succeeded
> Import of duplicity.backends.cfbackend Succeeded
> Import of duplicity.backends.copycombackend Succeeded
> Import of duplicity.backends.dpbxbackend Failed: No module named dropbox
> Import of duplicity.backends.gdocsbackend Succeeded
> Import of duplicity.backends.giobackend Succeeded
> Import of duplicity.backends.hsibackend Succeeded
> Import of duplicity.backends.hubicbackend Succeeded
> Import of duplicity.backends.imapbackend Succeeded
> Import of duplicity.backends.lftpbackend Succeeded
> Import of duplicity.backends.localbackend Succeeded
> Import of duplicity.backends.mediafirebackend Succeeded
> Import of duplicity.backends.megabackend Succeeded
> Import of duplicity.backends.multibackend Succeeded
> Import of duplicity.backends.ncftpbackend Succeeded
> Import of duplicity.backends.onedrivebackend Succeeded
> Import of duplicity.backends.par2backend Succeeded
> Import of duplicity.backends.pydrivebackend Succeeded
> Import of duplicity.backends.rsyncbackend Succeeded
> Import of duplicity.backends.ssh_paramiko_backend Succeeded
> Import of duplicity.backends.ssh_pexpect_backend Succeeded
> Import of duplicity.backends.swiftbackend Succeeded
> Import of duplicity.backends.sxbackend Succeeded
> Import of duplicity.backends.tahoebackend Succeeded
> Import of duplicity.backends.webdavbackend Succeeded
> Using temporary directory /tmp/duplicity-A6tpL8-tempdir
> --- Finished state FAILED 'code 30' at 09:39:43.748 - Runtime 00:00:01.309
> ---
>
> --- Start running command POST at 09:39:43.773 ---
> Running '/etc/duply/cpanel_s3/post' - OK
> --- Finished state OK at 09:39:43.791 - Runtime 00:00:00.018 ---
>
> The amazon S3 bucket is clean, no previous backup on it, could you help me
> please?
>
> regards
>
> --
> You received this question notification because your team duplicity-team
> is an answer contact for Duplicity.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~duplicity-team
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~duplicity-team
> More help : https://help.launchpad.net/ListHelp
>

juniorfga (juniorfga) said : #2

This are the versions of the packages

root@s1 [~]# yum install duply duplicity ncftp python-boto -y;
Loaded plugins: fastestmirror
MariaDB100 | 2.9 kB 00:00:00
base | 3.6 kB 00:00:00
epel/x86_64/metalink | 12 kB 00:00:00
extras | 3.4 kB 00:00:00
updates | 3.4 kB 00:00:00
vz-base | 951 B 00:00:00
vz-updates | 951 B 00:00:00
Loading mirror speeds from cached hostfile
 * base: mirror.tzulo.com
 * epel: mirror.oss.ou.edu
 * extras: centos.chicago.waneq.com
 * updates: pubmirrors.dal.corespace.com
Package duply-1.11.3-1.el7.noarch already installed and latest version
Package duplicity-0.7.10-1.el7.x86_64 already installed and latest version
Package 2:ncftp-3.2.5-7.el7.x86_64 already installed and latest version
Package python-boto-2.25.0-2.el7.centos.noarch is obsoleted by python2-boto-2.41.0-1.el7.noarch which is already installed
Nothing to do

I'm using duply to run duplicity, do you need the config?
regards

juniorfga (juniorfga) said : #3

Hello Keneth
the command I execute is this /usr/bin/duply cpanel_s3 backup 2>&1 >> /var/log/duply_ispconfig.log
do you need my duply config?
regards

edso (ed.so) said : #4

hey Junior,

Kenneth probably needs the full duplicity command line. you can see it via
  /usr/bin/duply cpanel_s3 backup --preview
. take care to obfuscate infos you deem private before you send it!

..ede/duply.net

On 11.10.2016 20:52, juniorfga wrote:
> Question #402905 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/402905
>
> juniorfga posted a new comment:
> Hello Keneth
> the command I execute is this /usr/bin/duply cpanel_s3 backup 2>&1 >> /var/log/duply_ispconfig.log
> do you need my duply config?
> regards
>

juniorfga (juniorfga) said : #5

Thanks Ede

This is the output of the command

root@s1 [~]# /usr/bin/duply cpanel_s3 backup --preview
Start duply v1.11.3, time is 2016-10-11 16:58:40.
Using profile '/etc/duply/cpanel_s3'.
Using installed duplicity version 0.7.10, python 2.7.5, gpg 2.0.22 (Home: ~/.gnupg), awk 'GNU Awk 4.0.2', grep 'grep (GNU grep) 2.20', bash '4.2.46(1)-release (x86_64-redhat-linux-gnu)'.
Autoset found secret key of first GPG_KEY entry 'XXXXXX' for signing.
-- Run cmd -- Checking TEMP_DIR '/tmp' is a folder and writable --
test -d '/tmp' && test -w '/tmp' 2>&1
-- Run cmd -- Test - Encrypt to 'XXXXXX','XXXXXX','XXXXXX' & Sign with 'XXXXXX' --
echo YYYYYYYYYYY | gpg --sign --default-key XXXXXX --passphrase-fd 0 --batch -r XXXXXX -r XXXXXX -r XXXXXX --status-fd 1 -o '/tmp/duply.22888.1476219521_ENC' -e '/usr/bin/duply' 2>&1
-- Run cmd -- Test - Decrypt --
echo YYYYYYYYYYY | gpg --passphrase-fd 0 --batch -o '/tmp/duply.22888.1476219521_DEC' -d '/tmp/duply.22888.1476219521_ENC' 2>&1
-- Run cmd -- Test - Compare --
test "$(cat '/usr/bin/duply')" = "$(cat '/tmp/duply.22888.1476219521_DEC')" 2>&1
Cleanup - Delete '/tmp/duply.22888.1476219521_*'(FAILED)

--- Start running command PRE at 16:58:41.169 ---
. /etc/duply/cpanel_s3/pre
--- Finished state OK at 16:58:41.183 - Runtime 00:00:00.013 ---

--- Start running command BKP at 16:58:41.193 ---
TMPDIR='/tmp' PASSPHRASE=YYYYYYYYYYY FTP_PASSWORD='ZZZZZZZZZZ' duplicity --name duply_cpanel_s3 --encrypt-key XXXXXX --encrypt-key XXXXXX --encrypt-key XXXXXX --sign-key XXXXXX --verbosity '9' --exclude-if-present '.duplicity-ignore' --full-if-older-than 1M --volsize 250 --exclude-filelist '/etc/duply/cpanel_s3/exclude' '/' 's3://<email address hidden>/kkkkkkkkkk/server.com'
--- Finished state OK at 16:58:41.235 - Runtime 00:00:00.041 ---

--- Start running command POST at 16:58:41.246 ---
. /etc/duply/cpanel_s3/post
--- Finished state OK at 16:58:41.259 - Runtime 00:00:00.013 ---

If you need anything else please don't hesitate in telling me

regards

juniorfga (juniorfga) said : #6

Hello ede, keneth

could you help me please? some ideas?

regards

Take a look here: https://stackoverflow.com/questions/5396932/why-are-no-amazon-s3-authentication-handlers-ready

I suspect you are missing a config file for S3, or the environment vars are not set properly.

juniorfga (juniorfga) said : #8

I've found the solution on this page

http://boto.cloudhackers.com/en/latest/getting_started.html

under the "Configuring Boto Credentials" topic, it seems that Duply/Duplicity is not updating the environment variables (e.g. AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SECURITY_TOKEN and AWS_PROFILE). so I've created a ~/.boto file with these contents:

[Credentials]
aws_access_key_id = YOURACCESSKEY
aws_secret_access_key = YOURSECRETKEY

and it start working

edso (ed.so) said : #9

On 01.11.2016 13:38, juniorfga wrote:
> Question #402905 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/402905
>
> Status: Open => Solved
>
> juniorfga confirmed that the question is solved:
> I've found the solution on this page
>
> http://boto.cloudhackers.com/en/latest/getting_started.html
>
> under the "Configuring Boto Credentials" topic, it seems that
> Duply/Duplicity is not updating the environment variables (e.g.
> AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SECURITY_TOKEN and
> AWS_PROFILE). so I've created a ~/.boto file with these contents:
>
> [Credentials]
> aws_access_key_id = YOURACCESSKEY
> aws_secret_access_key = YOURSECRETKEY
>
> and it start working
>

Junior,

why would you expect duply/duplicity to generate the file for you?
the boto backend sources say

    """
    Backend for Amazon's Simple Storage System, (aka Amazon S3), though
    the use of the boto module, (http://code.google.com/p/boto/).

    To make use of this backend you must set aws_access_key_id
    and aws_secret_access_key in your ~/.boto or /etc/boto.cfg
    with your Amazon Web Services key id and secret respectively.
    Alternatively you can export the environment variables
    AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
    """

did you export the AWS_* env vars beforehand at all?

note to self: the manpage is missing the infos above about amazon s3 access auth completely. have to update manpage accordingly.

..ede/duply.net