OpenStack Compute (nova)

nova.service IOError: [Errno 13] Permission denied: '/var/lock/nova/nova-iptables'

Asked by Rashmi

I am using the Folsom release and when I try to run a VM instance and it fails in the dashboard. The nova-api.log gives the below error:

2012-10-17 14:28:07 TRACE nova.service
2012-10-17 14:28:07 INFO nova.wsgi [-] Stopping WSGI server.
2012-10-17 14:28:07 INFO nova.service [-] Child 117963 exited with status 2
2012-10-17 14:28:07 INFO nova.service [-] Started child 117964
2012-10-17 14:28:07 DEBUG nova.utils [-] Got semaphore "iptables" for method "_apply"... from (pid=117964) inner /usr/lib/python2.7/dist-packages/nova/utils.py:713
2012-10-17 14:28:07 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "_apply"... from (pid=117964) inner /usr/lib/python2.7/dist-packages/nova/utils.py:717
2012-10-17 14:28:07 ERROR nova.service [-] Unhandled exception
2012-10-17 14:28:07 TRACE nova.service Traceback (most recent call last):
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 290, in _start_child
2012-10-17 14:28:07 TRACE nova.service self._child_process(wrap.server)
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 267, in _child_process
2012-10-17 14:28:07 TRACE nova.service launcher.run_server(server)
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 132, in run_server
2012-10-17 14:28:07 TRACE nova.service server.start()
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/service.py", line 619, in start
2012-10-17 14:28:07 TRACE nova.service self.manager.init_host()
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/api/manager.py", line 42, in init_host
2012-10-17 14:28:07 TRACE nova.service self.network_driver.metadata_accept()
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 532, in metadata_accept
2012-10-17 14:28:07 TRACE nova.service iptables_manager.apply()
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/network/linux_net.py", line 345, in apply
2012-10-17 14:28:07 TRACE nova.service self._apply()
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 740, in inner
2012-10-17 14:28:07 TRACE nova.service with lock:
2012-10-17 14:28:07 TRACE nova.service File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 605, in __enter__
2012-10-17 14:28:07 TRACE nova.service self.lockfile = open(self.fname, 'w')
2012-10-17 14:28:07 TRACE nova.service IOError: [Errno 13] Permission denied: '/var/lock/nova/nova-iptables'
2012-10-17 14:28:07 TRACE nova.service
2012-10-17 14:28:07 INFO nova.wsgi [-] Stopping WSGI server.
2012-10-17 14:28:07 INFO nova.service [-] Child 117964 exited with status 2

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Rashmi
Solved:
Last query:
Last reply:
Revision history for this message
Tiantian Gao (gtt116) said : 		#1

change the owner and owner group of /var/lock/nova to the user who run nova-api, nova-compute, nova-network etc.
Because all these service may read-write the lock file in /var/lock/nova.
For example, if you use nova to run these service, you can do 'chown nova:nova -R /var/lock/nova'

Hope it is helpful to you.

Revision history for this message
Rashmi (rushmep3) said : 		#2

Thanks for your reply. I am using the dashboard to run the VM instance.
Despite having changed the owner and owner group of /var/lock/nova as you said, I still see the same error in nova-api.log and eventually the VM nstance shows an Error status on the dashboard.
Is there something else that I am missing? Would any other logs help?

Revision history for this message
Tiantian Gao (gtt116) said : 		#3

If you are sure these is no service is using the lock, you can just delete it. And try to boot a new VM.

Revision history for this message
Rashmi (rushmep3) said : 		#4

After restarting nova services lock files get recreated! However, if i do a chmod 777 on the files everything goes smooth..

Revision history for this message
Rashmi (rushmep3) said : 		#5

Also I would like to know how to figure out which services are using the lock files?

Revision history for this message
Tiantian Gao (gtt116) said : 		#6

nova-iptables.lock?

nova-compute
nova-network
If you use quantum, it maybe use the lock too.

Revision history for this message
wangqiang.sheng (wangqiangsheng) said : 		#7

packages/nova/extend/network.py", line 160, in add_bridge_vlan
    fp = open(bond_script , 'w')
IOError: [Errno 13] Permission denied: u'/etc/sysconfig/network-scripts//ifcfg-bond0.210'

******************************************************************************************
        utils.execute("chmod","777",CONF.network_script_path, run_as_root=True)
        fp = open(bond_script , 'w')
        fp.write("DEVICE=%s \n" % network_info['bond_dev'])
        fp.write("ONBOOT=yes \n")
        fp.write("NM_CONTROLLED=no \n")
        fp.write("BOOTPROTO=static \n")
        fp.write("VLAN=yes \n")
        fp.close()

        fp = open(vlan_script , 'w')
        fp.write("DEVICE=%s \n" % network_info['vlan_name'])
        fp.write("BOOTPROTO=static \n")
        fp.write("ONBOOT=yes \n")
        fp.write("TYPE=Bridge \n")
        fp.close()
        utils.execute("chmod","755",CONF.network_script_path, run_as_root=True)
******************************************************************************************