Duplicity

BackendException: hostname 'x.y.z.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'

Asked by Yoav Aner

I'm facing a strange problem when trying to retrieve the collection-status. It seems to work ok with python 2.7.3, but not with 2.7.10

$ duplicity --encrypt-key=******* collection-status --s3-use-new-style s3+http://backups.testuff.com/x.y.z/weekly/
BackendException: hostname 'x.y.z.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'

If I remove '--s3-new-new-style' it also seems to work, but it used to work fine even with this setting on python 2.7.3

Using latest 0.6.26 version of duplicity and tested against both versions of python.

more detailed trace using -v9

$ duplicity -v9 --encrypt-key=******* collection-status --s3-use-new-style s3+http://backups.testuff.com/x.y.z/weekly/

Duplicity 0.6 series is being deprecated:
See http://www.nongnu.org/duplicity/

Using archive dir: /root/.cache/duplicity/f1f371ac61f2275b203deb75c28cb854
Using backup name: f1f371ac61f2275b203deb75c28cb854
Import of duplicity.backends.botobackend Succeeded
Import of duplicity.backends.cfbackend Succeeded
Import of duplicity.backends.dpbxbackend Succeeded
Import of duplicity.backends.ftpbackend Succeeded
Import of duplicity.backends.ftpsbackend Succeeded
Import of duplicity.backends.gdocsbackend Succeeded
Import of duplicity.backends.hsibackend Succeeded
Import of duplicity.backends.imapbackend Succeeded
Import of duplicity.backends.localbackend Succeeded
Import of duplicity.backends.megabackend Succeeded
Import of duplicity.backends.rsyncbackend Succeeded
Import of duplicity.backends.sshbackend Succeeded
Import of duplicity.backends.swiftbackend Succeeded
Import of duplicity.backends.tahoebackend Succeeded
Import of duplicity.backends.webdavbackend Succeeded
Import of duplicity.backends.~par2wrapperbackend Succeeded
Using temporary directory /tmp/duplicity-jOtAmf-tempdir
Backend error detail: Traceback (most recent call last):
  File "/usr/bin/testuff/virtualenv/server2/bin/duplicity", line 1509, in <module>
    with_tempdir(main)
  File "/usr/bin/testuff/virtualenv/server2/bin/duplicity", line 1503, in with_tempdir
    fn()
  File "/usr/bin/testuff/virtualenv/server2/bin/duplicity", line 1336, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/bin/testuff/virtualenv/server2/lib/python2.7/site-packages/duplicity/commandline.py", line 1054, in ProcessCommandLine
    globals.backend = backend.get_backend(args[0])
  File "/usr/bin/testuff/virtualenv/server2/lib/python2.7/site-packages/duplicity/backend.py", line 163, in get_backend
    return _backends[pu.scheme](pu)
  File "/usr/bin/testuff/virtualenv/server2/lib/python2.7/site-packages/duplicity/backends/_boto_single.py", line 163, in __init__
    self.resetConnection()
  File "/usr/bin/testuff/virtualenv/server2/lib/python2.7/site-packages/duplicity/backends/_boto_single.py", line 189, in resetConnection
    raise BackendException(err.message)
BackendException: hostname 'x.y.z.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'

Question information

Language:
English Edit question
Status:
Answered
For:
Duplicity Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Launchpad Janitor (janitor) said : 		#1

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Yoav Aner (yoav-gingerlime) said : 		#2

I believe it's a valid question/bug

On Tuesday, 10 November 2015, Launchpad Janitor <
<email address hidden>> wrote:

> Your question #273031 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/273031
>
> Status: Open => Expired
>
> Launchpad Janitor expired the question:
> This question was expired because it remained in the 'Open' state
> without activity for the last 15 days.
>
> --
> If you're still having this problem, you can reopen your question either
> by replying to this email or by going to the following page and
> entering more information about your problem:
> https://answers.launchpad.net/duplicity/+question/273031
>
> You received this question notification because you asked the question.
>

Revision history for this message
edso (ed.so) said : 		#3

On 10.11.2015 09:43, Yoav Aner wrote:
> Question #273031 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/273031
>
> Status: Expired => Open
>
> Yoav Aner is still having a problem:
> I believe it's a valid question/bug
>

you wrote yourself

"seems to work ok with python 2.7.3, but not with 2.7.10"
and
"If I remove '--s3-new-new-style' it also seems to work"

also: the error is legitimate. the hostname really doesn't match.

as the error occurs in boto, which is used to access s3 i suggest to update that to the latest greatest. if the error persists file a bug with them ;)

good luck.. ede/duply.net

Revision history for this message
Yoav Aner (yoav-gingerlime) said : 		#4

ok, thanks for pointing boto out. I'll see if upgrading to the latest version of boto resolves this, or will try to report this to them.

Otherwise, I don't think those 'workarounds' -- downgrade python to 2.7.3 or avoid using this switch, that should be backwards compatible with old-style buckets, or at least it was -- are really a solution.

> also: the error is legitimate. the hostname really doesn't match.

I don't know the definition of `*` matches in this context, but in some cases `*` should match both `x` and `x.y.z.`, and it's odd that the same error doesn't happen with earlier version of python.

Revision history for this message
Yoav Aner (yoav-gingerlime) said : 		#5

updating boto to latest (2.38.0) didn't make a difference unfortunately

Revision history for this message
edso (ed.so) said : 		#6

On 10.11.2015 11:02, Yoav Aner wrote:
> Question #273031 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/273031
>
> Yoav Aner posted a new comment:
> ok, thanks for pointing boto out. I'll see if upgrading to the latest
> version of boto resolves this, or will try to report this to them.
>
> Otherwise, I don't think those 'workarounds' -- downgrade python to
> 2.7.3 or avoid using this switch, that should be backwards compatible
> with old-style buckets, or at least it was -- are really a solution.

never suggested that. just pointed out that obviously a change there but not in duplicity caused the hickup.

>> also: the error is legitimate. the hostname really doesn't match.
>
> I don't know the definition of `*` matches in this context, but in some
> cases `*` should match both `x` and `x.y.z.`,

right, depends on the context. afaik there is a limitation of one level subdomains for wildcard certs.

> and it's odd that the same
> error doesn't happen with earlier version of python.
>

my guess would be they (python) updated their code to enforce the limitation above and boto/amazon stumbled over it. but that's what it is, just a guess ;)

..ede/duply.net

Revision history for this message
Yoav Aner (yoav-gingerlime) said : 		#7

Thanks edso, I reported it to boto here: https://github.com/boto/boto/issues/2836#issuecomment-155377133

Revision history for this message
edso (ed.so) said : 		#8

On 10.11.2015 11:08, Yoav Aner wrote:
> Question #273031 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/273031
>
> Yoav Aner posted a new comment:
> updating boto to latest (2.38.0) didn't make a difference unfortunately
>

a simple web search
 https://www.google.com/search?q=doesn't+match+either+of+'*.s3.amazonaws.com'
just came up with this
 https://github.com/boto/boto/issues/2836

smells like the issue of yours.. ede/duply.net

Revision history for this message
Yoav Aner (yoav-gingerlime) said : 		#9

yep, that's exactly where I reported this :)

Can you help with this problem?

Provide an answer of your own, or ask Yoav Aner for more information if necessary.

To post a message you must log in.