Problem running S3 middleware ...

Hi All,

Anyone has any idea?

Thanks.

I tried jets3t Java API using 'test:tester' as id and 'testing' as key, and got exactly the same error as when I ran s3curl which is [401 Unauthorized]. Can anyone help? Thanks.

Jun 28, 2011 7:59:52 PM org.jets3t.service.impl.rest.httpclient.RestStorageService performRequest
WARNING: Error Response: GET '/auth/v1.0/' -- ResponseCode: 401, ResponseStatus: Unauthorized, Request Headers: [Content-Type: , Date: Wed, 29 Jun 2011 02:59:50 GMT, Authorization: AWS test:tester:/fk0olUEoelY4uXJXb6xbW1igRQ=, User-Agent: JetS3t/0.8.1 (Linux/2.6.32-28-generic; amd64; en; JVM 1.6.0_20), Host: 127.0.0.1:8080], Response Headers: [Content-Length: 253, Content-Type: text/plain; charset=UTF-8, Date: Wed, 29 Jun 2011 02:59:52 GMT]
Exception in thread "main" org.jets3t.service.S3ServiceException: Request Error [401 Unauthorized

This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required.

   ]. GET '/auth/v1.0/' on Host '127.0.0.1:8080' @ 'Wed, 29 Jun 2011 02:59:52 GMT' -- ResponseCode: 401, ResponseStatus: Unauthorized
        at org.jets3t.service.S3Service.listAllBuckets(S3Service.java:1375)
        at org.jets3t.samples.CodeSamples.main(CodeSamples.java:89)
Caused by: org.jets3t.service.impl.rest.HttpException
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRequest(RestStorageService.java:525)
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.performRestGet(RestStorageService.java:911)
        at org.jets3t.service.impl.rest.httpclient.RestStorageService.listAllBucketsImpl(RestStorageService.java:1302)
        at org.jets3t.service.StorageService.listAllBuckets(StorageService.java:565)
        at org.jets3t.service.S3Service.listAllBuckets(S3Service.java:1372)
        ... 1 more

When using S3 style requests, you don't talk to auth to get a token. You have to make the request directly to your storage url, so on my system a s3curl requests looks something like:

./s3curl.pl --id system:root --key testpass --get -- -s -v -k
http://localhost:8080/v1.0/ACCOUNTHASH/CONTAINER/OBJECT

Hi Chuck,

Thanks for your reply. So what would the storage url be if I just followed the SAIO 1.4.0 instructions (loopback) to setup SWIFT? Would it be http://127.0.0.1:8080/v1.0 (without auth)? I tried it but still didn't work (see below).

ghuang@ubuntu:~$ s3curl.pl --id test:tester --key testing --get -- -s -v -k http://127.0.0.1:8080/v1.0
Unknown option: get
* About to connect() to 127.0.0.1 port 8080 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /v1.0 HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:8080
> Accept: */*
> Date: Thu, 21 Jul 2011 21:16:47 +0000
> Authorization: AWS test:tester:qAfC7Rvn17/xaqrVOAOeC8Bapgg=
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Content-Length: 872
< Date: Thu, 21 Jul 2011 21:16:53 GMT
< Connection: close
<

Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/home/ghuang/swift/trunk/swift/common/middleware/healthcheck.py", line 38, in __call__
    return self.app(env, start_response)
  File "/home/ghuang/swift/trunk/swift/common/middleware/memcache.py", line 32, in __call__
    return self.app(env, start_response)
  File "/home/ghuang/swift/trunk/swift/common/middleware/swauth.py", line 133, in __call__
    groups = self.get_groups(env, token)
  File "/home/ghuang/swift/trunk/swift/common/middleware/swauth.py", line 225, in get_groups
    msg = base64.urlsafe_b64decode(unquote(token))
  File "/usr/lib/python2.6/urllib.py", line 1170, in unquote
    res = s.split('%')
AttributeError: 'NoneType' object has no attribute 'split'
* Closing connection #0
ghuang@ubuntu:~$

Thanks.

I'm not sure if this is the problem that you are running into, but you also have to modify the script and add your storage hostname to @endpoints.

For example, I made this work on my dev instance by adding 'localhost' to the list.

And also I just noticed that you are not including the account hash in your url.

http://localhost:8080/v1.0/ACCOUNTHASH

Hi Chuck,

Thanks for the prompt response. I did add '127.0.0.1' to @endpoint in s3curl.pl. I tried what you suggested with account hash, but still got the same error:

ghuang@ubuntu:~$ s3curl.pl --id test:tester --key testing --get -- -s -v -k http://127.0.0.1:8080/v1.0/ACCOUNTHASH
Unknown option: get
* About to connect() to 127.0.0.1 port 8080 (#0)
* Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /v1.0/ACCOUNTHASH HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: 127.0.0.1:8080
> Accept: */*
> Date: Thu, 21 Jul 2011 21:47:09 +0000
> Authorization: AWS test:tester:V/M/6Qqjpk193D2wIxPULMsztKQ=
>
< HTTP/1.1 500 Internal Server Error
< Content-Type: text/plain
< Content-Length: 872
< Date: Thu, 21 Jul 2011 21:47:09 GMT
< Connection: close
<
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/home/ghuang/swift/trunk/swift/common/middleware/healthcheck.py", line 38, in __call__
    return self.app(env, start_response)
  File "/home/ghuang/swift/trunk/swift/common/middleware/memcache.py", line 32, in __call__
    return self.app(env, start_response)
  File "/home/ghuang/swift/trunk/swift/common/middleware/swauth.py", line 133, in __call__
    groups = self.get_groups(env, token)
  File "/home/ghuang/swift/trunk/swift/common/middleware/swauth.py", line 225, in get_groups
    msg = base64.urlsafe_b64decode(unquote(token))
  File "/usr/lib/python2.6/urllib.py", line 1170, in unquote
    res = s.split('%')
AttributeError: 'NoneType' object has no attribute 'split'
* Closing connection #0
ghuang@ubuntu:~$

I will continue to investigate. Please let me know if you have any other idea that I might do something wrong.

Thanks.

oh...

Your account hash should look something like AUTH_1234567678762

The hash will be in the storage url after making an auth request. The following example was pulled from the SAIO doc:

curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0