Swift Disk Over SSH

Asked by Jerome Bell

I want to build a private storage cloud and I have limited resources. I have 2 machines to work with, one runs Ubuntu Server 12.04 (holding 3 hard disks) and the other runs Debian 6 (holding 1 hard disk) thus having a total of 4 disks where each disk corresponds to a zone. The Ubuntu node runs Horizon, proxy server and 3 storage servers where the 3 hard disks map to zones 1-3. I tried setting up swift storage services on the Debian node for zone 4 but I simply could not get it to work and I could not find any complete documentation. (The Debian node belongs to my roommate and he's letting me host a hard disk since my Ubuntu machine is maxed out on hard disk slots.) I then used sshfs to remote mount the disk on the Debian node to the Ubuntu node so I can run a fourth storage server mapped to the disk on the Debian node. Files were copied over just fine but then I got the following errors:

Aug 18 20:41:23 samson object-auditor ERROR Object /srv/4/node/sdb1/objects/85255/331/5341c94981388cdafe3fde905a6e6331/1340842112.58513.data failed audit and will be quarantined: Error when reading metadata:
Aug 18 20:41:23 samson object-auditor ERROR Object /srv/4/node/sdb1/objects/85255/331/5341c94981388cdafe3fde905a6e6331/1340842112.58513.data failed audit and will be quarantined: Error when reading metadata:
Aug 18 20:41:23 samson object-auditor ERROR auditing: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 209, in run_forever#012 self.run_once(**kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 220, in run_once#012 worker.audit_all_objects(mode=mode)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 79, in audit_all_objects#012 self.object_audit(path, device, partition)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 174, in object_audit#012 os.path.join(self.devices, device), path)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/replicator.py", line 63, in quarantine_renamer#012 renamer(from_dir, to_dir)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 197, in renamer#012 os.rename(old, new)#012OSError: [Errno 1] Operation not permitted
Aug 18 20:41:23 samson object-auditor ERROR auditing: #012Traceback (most recent call last):#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 209, in run_forever#012 self.run_once(**kwargs)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 220, in run_once#012 worker.audit_all_objects(mode=mode)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 79, in audit_all_objects#012 self.object_audit(path, device, partition)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/auditor.py", line 174, in object_audit#012 os.path.join(self.devices, device), path)#012 File "/usr/lib/python2.7/dist-packages/swift/obj/replicator.py", line 63, in quarantine_renamer#012 renamer(from_dir, to_dir)#012 File "/usr/lib/python2.7/dist-packages/swift/common/utils.py", line 197, in renamer#012 os.rename(old, new)#012OSError: [Errno 1] Operation not permitted

The /etc/fstab mount entry for the remote directory is

sshfs#jerome@192.168.0.3:/srv/4/node/sdb1 /srv/4/node/sdb1 fuse defaults,idmap=user,uid=109,gid=117,umask=0,allow_other,workaround=rename,defau
lt_permissions

Where the UID and GID corresponds to the swift user and group.

Is there something in swift's implementation that prevents it from reading metadata on a remotely mounted directory? Any help would be greatly appreciated.

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Constantine Peresypkin (constantine-q) said :
#1

Swift requires xattr support for object nodes filesystem.
SSHFS does not support xattr

Can you help with this problem?

Provide an answer of your own, or ask Jerome Bell for more information if necessary.

To post a message you must log in.