Is OpenStack swift compatible with the Rackspace Cloud Files API?

Asked by GalacticJello

From the documentation, it looks like OpenStack swift == Rackspace Cloud Files, so can we assume that we can use the existing Cloud Files API bindings for java and .net to access swift? Thanks...

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
GalacticJello
Solved:
Last query:
Last reply:
Revision history for this message
gholt (gholt) said :
#1

Yes, they should work fine. I'm not specifically certain with java and .net, but here's a quick session I used on my development box with python-cloudfiles (Note: I'm already running OpenStack Swift on the box)

$ git clone http://github.com/rackspace/python-cloudfiles.git
$ cd python-cloudfiles
$ python -c "import cloudfiles; print cloudfiles.get_connection(authurl='http://127.0.0.1:11000/v1.0', username='test:tester', api_key='testing').get_info()"
(4, 13641451)

Revision history for this message
csxbwang (xbwang-zh) said :
#2

Hi, gholt,

I have tried Python bindings as your suggestion. After setting up virtual machine, I installed Python-cloudfiles.

Following steps are ok:
>>> conn = cloudfiles.get_connection(authurl='http://127.0.0.1:11000/v1.0',username='test:tester', api_key='testing')
>>> containers = conn.get_all_containers()
>>> type(containers)
<class 'cloudfiles.container.ContainerResults'>
>>> len(containers)
2
>>> for container in containers: print container.name
...
ttt
ttt2
>>> fruit_container = containers[0]
>>> myfile = fruit_container.create_object('file1')

However, when uploading files, I got error messages.
>>> myfile.write('COPYING,my first file.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "cloudfiles/utils.py", line 43, in decorator
    return f(*args, **kwargs)
  File "cloudfiles/storage_object.py", line 334, in write
    raise ResponseError(response.status, response.reason)
cloudfiles.errors.ResponseError: 503: Internal Server Error

>>> myfile.load_from_filename('COPYING')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "cloudfiles/storage_object.py", line 451, in load_from_filename
    self.write(fobj, verify=verify, callback=callback)
  File "cloudfiles/utils.py", line 43, in decorator
    return f(*args, **kwargs)
  File "cloudfiles/storage_object.py", line 334, in write
    raise ResponseError(response.status, response.reason)
cloudfiles.errors.ResponseError: 503: Internal Server Error

The syslog information (for >>>myfile.load_from_filename('COPYING')):
Aug 21 15:19:07 csxbwang-ubuntu account 127.0.0.1 - - [21/Aug/2010:07:19:07 +0000] "HEAD /sdb2/231428/85c07fae-5caf-416a-b68e-8758ab30232b" 204 - "tx37d8b49d-b25d-479d-b515-26b5f8c37d76" "-" "-" 0.0016 ""

Aug 21 15:19:07 csxbwang-ubuntu container 127.0.0.1 - - [21/Aug/2010:07:19:07 +0000] "HEAD /sdb4/61660/85c07fae-5caf-416a-b68e-8758ab30232b/ttt" 204 - "tx37d8b49d-b25d-479d-b515-26b5f8c37d76" "-" "-" 0.0015

Aug 21 15:19:07 csxbwang-ubuntu object ERROR __call__ error with PUT /sdb3/217384/85c07fae-5caf-416a-b68e-8758ab30232b/ttt/file1 transaction tx37d8b49d-b25d-479d-b515-26b5f8c37d76: #012Traceback (most recent call last):#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 570, in __call__#012 res = getattr(self, req.method)(req)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/usr/lib/python2.6/contextlib.py", line 34, in __exit__#012 self.gen.throw(type, value, traceback)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 188, in mkstemp#012 yield fd, tmppath#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 215, in put#012 setxattr(fd, '%s%s' % (METADATA_KEY, key or ''), metastr[:254])#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 188, in setxattr#012 return xattr(f).set(attr, value, options=options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012 self._set(name, value, 0, options | self.options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 16, in _func#012 return func(first, *args)#012IOError: [Errno 95] Operation not supported

Aug 21 15:19:07 csxbwang-ubuntu object 127.0.0.1 - - [21/Aug/2010:07:19:07 +0000] "PUT /sdb1/217384/85c07fae-5caf-416a-b68e-8758ab30232b/ttt/file1" 500 1133 "-" "tx37d8b49d-b25d-479d-b515-26b5f8c37d76" "python-cloudfiles/1.7.2" 0.0230

Aug 21 15:19:07 csxbwang-ubuntu proxy ERROR 500 Traceback (most recent call last):#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 570, in __call__#012 res = getattr(self, req.method)(req)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/usr/lib/python2.6/contextlib.py", line 34, in __exit__#012 self.gen.throw(type, value, traceback)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 188, in mkstemp#012 yield fd, tmppath#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 215, in put#012 setxattr(fd, '%s%s' % (METADATA_KEY, key or ''), metastr[:254])#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 188, in setxattr#012 return xattr(f).set(attr, value, options=options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012 self._set(name, value, 0, options | self.options)#012 File "/usr/lib/pymodules/python2.6/xat From Object Server re: /v1/85c07fae-5caf-416a-b68e-8758ab30232b/ttt/file1 127.0.0.1:6030

Aug 21 15:19:07 csxbwang-ubuntu proxy ERROR 500 Traceback (most recent call last):#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 570, in __call__#012 res = getattr(self, req.method)(req)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/usr/lib/python2.6/contextlib.py", line 34, in __exit__#012 self.gen.throw(type, value, traceback)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 188, in mkstemp#012 yield fd, tmppath#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 215, in put#012 setxattr(fd, '%s%s' % (METADATA_KEY, key or ''), metastr[:254])#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 188, in setxattr#012 return xattr(f).set(attr, value, options=options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012 self._set(name, value, 0, options | self.options)#012 File "/usr/lib/pymodules/python2.6/xat From Object Server re: /v1/85c07fae-5caf-416a-b68e-8758ab30232b/ttt/file1 127.0.0.1:6040

Aug 21 15:19:07 csxbwang-ubuntu proxy ERROR 500 Traceback (most recent call last):#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 570, in __call__#012 res = getattr(self, req.method)(req)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/usr/lib/python2.6/contextlib.py", line 34, in __exit__#012 self.gen.throw(type, value, traceback)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 188, in mkstemp#012 yield fd, tmppath#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 408, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/home/csxbwang/swift/trunk/swift/obj/server.py", line 215, in put#012 setxattr(fd, '%s%s' % (METADATA_KEY, key or ''), metastr[:254])#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 188, in setxattr#012 return xattr(f).set(attr, value, options=options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012 self._set(name, value, 0, options | self.options)#012 File "/usr/lib/pymodules/python2.6/xat From Object Server re: /v1/85c07fae-5caf-416a-b68e-8758ab30232b/ttt/file1 127.0.0.1:6010

Aug 21 15:19:07 csxbwang-ubuntu proxy Object PUT returning 503 for [500, 500, 500], transaction tx37d8b49d-b25d-479d-b515-26b5f8c37d76

Aug 21 15:19:07 csxbwang-ubuntu proxy - 127.0.0.1 21/Aug/2010/07/19/07 PUT /v1/85c07fae-5caf-416a-b68e-8758ab30232b/ttt/file1 HTTP/1.0 503 - python-cloudfiles/1.7.2 tkfd7861e7-acda-4c6c-9aec-82a390163bae 1420 - - tx37d8b49d-b25d-479d-b515-26b5f8c37d76 - 0.0901

What is the problem?

Revision history for this message
gholt (gholt) said :
#3

This error:
...xattr/__init__.py", line 16, in _func#012 return func(first, *args)#012IOError: [Errno 95] Operation not supported...

This is the code trying to write xattrs to a file system that does not support xattrs. Swift data drives, the ones pointed to by the 'devices' configuration variable, require xattr support. It has been tested with xfs, but other file systems also support xattrs (see http://en.wikipedia.org/wiki/Extended_file_attributes) but often have to have it explicitly turned on with 'user_xattr' in /etc/fstab.

Also, see http://swift.openstack.org/deployment_guide.html#filesystem-considerations

Revision history for this message
GalacticJello (myspambasket) said :
#4

After testing out the .NET bindings a bit, the answer is "not without a lot of refactoring". As mentioned in another question, the encoding of the username (like test:tester) causes problems, and most of the container access parts of the API is assuming the existence of the CloudFiles' "X-CDN-Management-URL" header in order to function.