cinder-volume service fails to start

Asked by steve A

Cinder-volume fails to start
package version - 2012.2-0ubuntu2~cloud0
OS - Ubuntu 12.04 - 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

problem statement -
Unable to start Cinder-volume service, it fails on startup when configured to use ceph RBD backend.
The Cinder-api and Cinder scheduler service start Ok 100% of the time

Symptoms - unable to start cinder-volume with the following error appearing in cinder-volume.log:

2013-01-14 14:00:04 9327 AUDIT cinder.service [-] Starting cinder-volume node (version 2012.2-LOCALBRANCH:LOCALREVISION)
2013-01-14 14:00:04 DEBUG cinder.utils [req-2c7afcd5-12ac-414a-b175-0b449a211a4b None None] Running cmd (subprocess): rados lspools execute /usr/lib/python2.7/dist-packages/cinder/utils.py:156
2013-01-14 14:00:04 DEBUG cinder.utils [req-2c7afcd5-12ac-414a-b175-0b449a211a4b None None] Result was 254 execute /usr/lib/python2.7/dist-packages/cinder/utils.py:172
2013-01-14 14:00:04 9327 CRITICAL cinder [-] Unexpected error while running command.
Command: rados lspools
Exit code: 254
Stdout: ''
Stderr: "2013-01-14 14:00:04.291794 7fc8bffee780 -1 asok(0x14d9a70) AdminSocketConfigObs::init: failed: AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/client.admin.so
ck': (13) Permission denied\n2013-01-14 14:00:04.292146 7fc8bffee780 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication\n2013-01-14 14:00:04.292152 7fc8bffee780 0 librados:
 client.admin initialization error (2) No such file or directory\ncouldn't connect to cluster! error -2\n"

if you start the service manually as per this , it does start and run:

exec /bin/sh -c "exec cinder-volume --config-file=/etc/cinder/cinder.conf --log-file=/var/log/cinder/cinder-volume.log" cinder

the service stays up and connects to RBD volumes successfully

I suspect i need to do something to Cinders rootwrap configuration, but cant find a lot of guidance as to how to get this setup

Any suggestions or guidance would be really appreciated

My Cinder.conf is as follows:

[DEFAULT]
debug=true
sql_connection = mysql://cinder:(pw>@<cloud controller IP>:3306/cinder
rabbit_password = <rabbitpw>
rootwrap_config = /etc/cinder/rootwrap.conf
api_paste_confg = /etc/cinder/api-paste.ini
volume_driver = cinder.volume.driver.RBDDriver
rbd_pool=nova
rbd_user=admin
rbd_secret_uuid=<secretUIUIDHere)
volume_tmp_dir=/var/lib/nova/voltemp
#### (StrOpt) where to store temporary image files if the volume driver
#### does not write them directly to the volume
# iscsi_helper = tgtadm
volume_name_template = volume-%s
volume_group = volumes
# verbose = True
auth_strategy = keystone
state_path = /var/lib/cinder
# volumes_dir = /var/lib/cinder/volumes

Also, if there is a detailed installation guide showing all configuration steps needed to integrate Cinder into Openstack and Ceph, I'd appreciate a link to any documentation available

Question information

Language:
English Edit question
Status:
Solved
For:
Cinder Edit question
Assignee:
No assignee Edit question
Solved by:
steve A
Solved:
Last query:
Last reply:
Revision history for this message
steve A (dafridgie) said :
#1

Ok, solved it
It was a specific permissions problem on the ceph keyring file that prevented the keyring being read by the cinder user
Thanks to Inktank for their guidance on this!
Once the keyring could be read by the cinder user, the volume service fired up as it should