How to enable xattr for xfs on ubuntu?

xattrs are supported automatically by xfs as far as I know. My /etc/fstab line reads:

/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0

It could be the problem is that your object server is not configured to write on the xfs drive but somewhere else. Can you post your /etc/swift/object-server.conf? (Or /etc/swift/object-server/1.conf depending on how you have things set up).

Here's my /etc/swift/object-server/1.conf:

[DEFAULT]
devices = /srv/1/node
mount_check = false
bind_port = 6010
user = ghuang
log_facility = LOG_LOCAL2

[pipeline:main]
pipeline = object-server

[app:object-server]
use = egg:swift#object

[object-replicator]
vm_test_mode = yes

[object-updater]

[object-auditor]

Okay, so you're mounting at

/srv/swift-disk

And your object server is set to write to:

/srv/1/node

Do those end up the same device? Via symlinks or something?

I was following the instructions on (http://swift.openstack.org/development_saio.html) using a loopback device for storage. According to the instructions, /srv/swift-disk was created by

mkfs.xfs -i size=1024 /srv/swift-disk

and mounted as the following entry in /etc/fstab:

/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0

ghuang@ubuntu:~$ ls -la /srv
total 5388
drwxr-xr-x 6 root root 4096 2011-05-16 08:23 .
drwxr-xr-x 22 root root 4096 2011-05-16 08:41 ..
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 1
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 2
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 3
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 4
-rw-r--r-- 1 root root 1024000000 2011-05-17 11:17 swift-disk

ghuang@ubuntu:/srv/1/node/sdb1$ ls -la
total 20
drwxr-xr-x 5 ghuang ghuang 4096 2011-05-16 10:06 .
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 ..
drwxrwxrwx 33 ghuang ghuang 4096 2011-05-17 11:22 accounts
drwxrwxrwx 22 ghuang ghuang 4096 2011-05-16 11:30 containers
drwxrwxrwx 2 ghuang ghuang 4096 2011-05-17 11:22 tmp

I am not sure if this list is expected. I see accounts and containers, but not objects. Is it supposed to have objects directory listed, too? Is there any other information I can provide to help resolve this issue?

Thanks.

Ah okay, you're using the loopback method. If the accounts and containers are working, that's odd that the objects aren't.

Can you run the following:

touch /srv/1/node/testing; xattr /srv/1/node/testing user.test test ; xattr /srv/1/node/testing user.test

You should get the output of just the word test.

Saw this instruction

for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done

So /srv/1 was a symlink to /mnt/sdb1/1 where /mnt/sdb1 was mounted to /srv/swift-disk in /etc/fstab. Object server should be mounted to the same device.

Yeah, it definitely seems to be set up correctly, which is why it's weird it's not working. Try that 'touch /srv/1/node/testing; xattr /srv/1/node/testing user.test test ; xattr /srv/1/node/testing user.test' and see what you get.

Also, you might try just doing a 'swift-init object-server restart'. Lame, but maybe somehow a bad config was loaded, corrected, but not reloaded.

Here's the output from xattr testing after 'swift-init object-server restart' - not as expected :-(

ghuang@ubuntu:~$ swift-init object-server restart
Signal object-server pid: 21093 signal: 15
Signal object-server pid: 21094 signal: 15
Signal object-server pid: 21095 signal: 15
Signal object-server pid: 21096 signal: 15
object-server (21096) appears to have stopped
object-server (21093) appears to have stopped
object-server (21094) appears to have stopped
object-server (21095) appears to have stopped
WARNING: Unable to increase file descriptor limit. Running as non-root?
Starting object-server...(/etc/swift/object-server/1.conf)
Starting object-server...(/etc/swift/object-server/2.conf)
Starting object-server...(/etc/swift/object-server/3.conf)
Starting object-server...(/etc/swift/object-server/4.conf)

ghuang@ubuntu:~$ startmain
WARNING: Unable to increase file descriptor limit. Running as non-root?
proxy-server running (1304 - /etc/swift/proxy-server.conf)
proxy-server already started...
container-server running (1305 - /etc/swift/container-server/1.conf)
container-server running (1317 - /etc/swift/container-server/4.conf)
container-server running (1310 - /etc/swift/container-server/2.conf)
container-server running (1311 - /etc/swift/container-server/3.conf)
container-server already started...
account-server running (1321 - /etc/swift/account-server/1.conf)
account-server running (1330 - /etc/swift/account-server/3.conf)
account-server running (1332 - /etc/swift/account-server/4.conf)
account-server running (1325 - /etc/swift/account-server/2.conf)
account-server already started...
object-server running (21138 - /etc/swift/object-server/1.conf)
object-server running (21139 - /etc/swift/object-server/2.conf)
object-server running (21140 - /etc/swift/object-server/3.conf)
object-server running (21141 - /etc/swift/object-server/4.conf)
object-server already started...

ghuang@ubuntu:~$ touch /srv/1/node/testing; xattr /srv/1/node/testing user.test test; xattr /srv/1/node/testing user.test
Traceback (most recent call last):
  File "/usr/bin/xattr", line 8, in <module>
    load_entry_point('xattr==0.4', 'console_scripts', 'xattr')()
  File "/usr/lib/pymodules/python2.6/xattr/tool.py", line 70, in main
    attrs[attr_name] = attr_value
  File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 115, in __setitem__
    self.set(item, value)
  File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set
    self._set(name, value, 0, options | self.options)
  File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 16, in _func
    return func(first, *args)
IOError: [Errno 95] Operation not supported: '/srv/1/node/testing'
No such attribute.

Hmm. Very odd. Here's a reference to xattr on xfs being the default (it should *just* work):

http://wiki.kaspersandberg.com/doku.php?id=howtos:xattr
"If you are using xfs, xattr is automatically enabled..."

So let's test the mounted file system directly...

How about:

touch /mnt/sdb1/testing; xattr /mnt/sdb1/testing user.test test; xattr

If that doesn't work, well, I'm just not sure why your xfs wouldn't have xattr support. I didn't think that was even possible to not have it with xfs.

Also, I guess just run 'mount' and 'ls -la /mnt/sdb1' and 'ls -la /srv/node' and paste those results.

Eh, that last message didn't get pasted quite right, here's the commands to run and paste so we can hopefully figure out what's going on:

touch /mnt/sdb1/testing; xattr /mnt/sdb1/testing user.test test; xattr /mnt/sdb1/testing user.test

mount

ls -la /mnt/sdb1/

ls -la /srv/node/

Gah!, that last line should be

ls -la /srv/

Thanks for the information and suggestion. Here's the xattr testing - simple xattr test for root worked, but still failed when creating account:

ghuang@ubuntu:~$ touch /mnt/sdb1/testing
touch: cannot touch `/mnt/sdb1/testing': Permission denied
ghuang@ubuntu:~$ su
Password:
root@ubuntu:/home/ghuang# touch /mnt/sdb1/testing; xattr /mnt/sdb1/testing user.test test; xattr /mnt/sdb1/testing user.test
test
root@ubuntu:/home/ghuang# cd bin
root@ubuntu:/home/ghuang/bin# ls
recreateaccounts remakerings resetswift startmain startrest
root@ubuntu:/home/ghuang/bin# ./recreateaccounts
Account creation failed: 500 Server Error
User creation failed: 500 Server Error
Account creation failed: 500 Server Error
User creation failed: 500 Server Error
Account creation failed: 500 Server Error
User creation failed: 500 Server Error
Account creation failed: 500 Server Error
User creation failed: 500 Server Error
root@ubuntu:/home/ghuang/bin# swauth-prep -K password
root@ubuntu:/home/ghuang/bin# swauth-add-account -K password test
Account creation failed: 500 Server Error
root@ubuntu:/home/ghuang/bin# ls -la /mnt/sdb1/
total 4
drwxr-xr-x 6 root root 52 2011-05-18 09:35 .
drwxr-xr-x 4 root root 4096 2011-05-16 08:20 ..
drwxr-xr-x 2 ghuang ghuang 6 2011-05-16 08:20 1
drwxr-xr-x 2 ghuang ghuang 6 2011-05-16 08:20 2
drwxr-xr-x 2 ghuang ghuang 6 2011-05-16 08:20 3
drwxr-xr-x 2 ghuang ghuang 6 2011-05-16 08:20 4
-rw-r--r-- 1 root root 0 2011-05-18 09:35 testing
root@ubuntu:/home/ghuang/bin# ls -la /srv/
total 5388
drwxr-xr-x 6 root root 4096 2011-05-16 08:23 .
drwxr-xr-x 22 root root 4096 2011-05-16 08:41 ..
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 1
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 2
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 3
drwxr-xr-x 3 ghuang ghuang 4096 2011-05-16 08:23 4
-rw-r--r-- 1 root root 1024000000 2011-05-18 09:35 swift-disk
root@ubuntu:/home/ghuang/bin#

Looking at /var/log/swift/all.log, still got this 'Operation not supported' error:

May 18 09:36:51 ubuntu object-server ERROR __call__ error with PUT /sdb3/30251/AUTH_.auth/.account_id/AUTH_6449380b-f31b-4eb6-931e-71867d3b9b64 : #012Traceback (most recent call last):#012 File "/home/ghuang/swift/trunk/swift/obj/server.py", line 728, in __call__#012 res = getattr(self, req.method)(req)#012 File "/home/ghuang/swift/trunk/swift/obj/server.py", line 545, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/usr/lib/python2.6/contextlib.py", line 34, in __exit__#012 self.gen.throw(type, value, traceback)#012 File "/home/ghuang/swift/trunk/swift/obj/server.py", line 277, in mkstemp#012 yield fd, tmppath#012 File "/home/ghuang/swift/trunk/swift/obj/server.py", line 545, in PUT#012 file.put(fd, tmppath, metadata)#012 File "/home/ghuang/swift/trunk/swift/obj/server.py", line 301, in put#012 write_metadata(fd, metadata)#012 File "/home/ghuang/swift/trunk/swift/obj/server.py", line 89, in write_metadata#012 setxattr(fd, '%s%s' % (METADATA_KEY, key or ''), metastr[:254])#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 188, in setxattr#012 return xattr(f).set(attr, value, options=options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 81, in set#012 self._set(name, value, 0, options | self.options)#012 File "/usr/lib/pymodules/python2.6/xattr/__init__.py", line 16, in _func#012 return func(first, *args)#012IOError: [Errno 95] Operation not supported

That's it. I was wondering about why they are not symbolic links. I must have screwed up when executing the 'ln -s' command. I removed /srv/1, /srv/2, /srv/3, /srv/4 and re-created the symbolic links. And it worked. Thank you so much for your time and help.

Thanks gholt, that solved my question.

I ran into this issue (thanks for the resolution, it worked great!) and I found out why I had it that way. The initial setup instructions of SAIO create symbolic links, but the resetswift script has mkdir instead of ln:

mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4

This is the section I'm referring to in the SAIO instructions: http://swift.openstack.org/development_saio.html#setting-up-scripts-for-running-swift