neutron

Neutron - DHCP Agent is unable to enable dhcp

Asked by Kausum Kumar

I am not able to login to the VM that I am have created. I am using a CentOS 6.4 + Havana on 3 node installation. On the network node, I see message in the dhcp-agent.log file - unable to enable dhcp. Any idea why?

2014-02-04 17:40:00.172 10487 ERROR neutron.agent.dhcp_agent [-] Unable to enable dhcp.
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Traceback (most recent call last):
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/dhcp_agent.py", line 126, in call_driver
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent getattr(driver, action)(**action_kwargs)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/dhcp.py", line 167, in enable
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent reuse_existing=True)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/dhcp.py", line 705, in setup
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent namespace=network.namespace)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/interface.py", line 174, in plug
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent namespace2=namespace)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 126, in add_veth
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent self.ensure_namespace(namespace2)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 136, in ensure_namespace
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent ip = self.netns.add(name)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 447, in add
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent self._as_root('add', name, use_root_namespace=True)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 217, in _as_root
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent kwargs.get('use_root_namespace', False))
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 70, in _as_root
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent namespace)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 81, in _execute
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent root_helper=root_helper)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 62, in execute
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent raise RuntimeError(m)
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent RuntimeError:
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'add', 'qdhcp-14b6e9ee-50d4-44fd-916e-91260ff62f32']
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Exit code: 255
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Stdout: ''
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent Stderr: 'Bind /proc/self/ns/net -> /var/run/netns/qdhcp-14b6e9ee-50d4-44fd-916e-91260ff62f32 failed: No such file or directory\n'
2014-02-04 17:40:00.172 10487 TRACE neutron.agent.dhcp_agent
2014-02-04 17:40:00.280 10487 DEBUG neutron.openstack.common.rpc.amqp [-] Making asynchronous cast on q-plugin... cast /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:559
2014-02-04 17:40:00.280 10487 DEBUG neutron.openstack.common.rpc.amqp [-] UNIQUE_ID is c03bacfe49a94ee6a8e038f696be74c5. _add_unique_id /usr/lib/python2.6/site-packages/neutron/openstack/common/rpc/amqp.py:339
2014-02-04 17:40:00.283 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: ExchangeQuery(name='neutron', id=serial(0), sync=True) write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685
2014-02-04 17:40:00.285 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: '\x0f\x01\x00\x1a\x00\x01\x00\x00\x00\x00\x00\x00\x07\x03\x01\x01\x01\x00\x07neutron\x0f\x01\x00\x1a\x00\x01\x00\x00\x00\x00\x00\x00\x08\x04\x01\x01\x01\x00\x07neutron' writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480
2014-02-04 17:40:00.285 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: '\x0f\x01\x00,\x00\x01\x00\x00\x00\x00\x00\x00\x03\x02\x01\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x12\x07\x01\t\x00\x05topic\x00\x00\x00\x04\x00\x00\x00\x00\x0f\x00\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x00\x02\n\x01\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x00\x0f\x01\x00\x1e\x00\x01\x00\x00\x00\x00\x00\x00\x03\x02\x01\x00\x03\x00\x00\x00\x00\x01\x00\x00\x00\x04\x08\x01\x00\x00\x0f\x00\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x00\x02\n\x01\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x01' readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416
2014-02-04 17:40:00.287 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: ExecutionResult(command_id=serial(0), value=ExchangeQueryResult(type=u'topic', arguments={}), id=serial(0)) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.287 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCompleted(commands=[0-0]) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.288 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: ExecutionResult(command_id=serial(1), value=QueueQueryResult(), id=serial(1)) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.289 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCompleted(commands=[0-1]) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.291 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: MessageTransfer(destination='neutron', id=serial(2), sync=True, headers=(DeliveryProperties(routing_key='q-plugin'), MessageProperties(content_type='amqp/map', application_headers={'qpid.subject': 'q-plugin'})), payload='\x00\x00\x02\xde\x00\x00\x00\x02\x0coslo.message\x95\x02\xb7{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}\x0coslo.version\x95\x00\x032.0') write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685
2014-02-04 17:40:00.292 10487 DEBUG qpid.messaging [-] SENT[25ec2d8]: Message(properties={'qpid.subject': 'q-plugin'}, content={'oslo.message': '{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}', 'oslo.version': '2.0'}) send /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:1282
2014-02-04 17:40:00.293 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: '\x0b\x01\x00\x1a\x00\x01\x00\x00\x00\x00\x00\x00\x04\x01\x01\x01\x01\x00\x07neutron\x03\x02\x00N\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\r\x04\x01\x00\x02\x08q-plugin\x00\x00\x00-\x04\x03\x10\x01\x08amqp/map\x00\x00\x00\x1c\x00\x00\x00\x01\x0cqpid.subject\x95\x00\x08q-plugin\x07\x03\x02\xee\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x02\xde\x00\x00\x00\x02\x0coslo.message\x95\x02\xb7{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}\x0coslo.version\x95\x00\x032.0' writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480
2014-02-04 17:40:00.294 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: '\x0f\x00\x00\x1a\x00\x00\x00\x00\x00\x00\x00\x00\x02\n\x01\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x02' readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416
2014-02-04 17:40:00.294 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCompleted(commands=[0-2]) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.294 10487 DEBUG qpid.messaging [-] RACK[25ec2d8]: Message(properties={'qpid.subject': 'q-plugin'}, content={'oslo.message': '{"_context_roles": ["admin"], "_context_read_deleted": "no", "args": {"agent_state": {"agent_state": {"binary": "neutron-dhcp-agent", "topic": "dhcp_agent", "host": "Openstack-Network", "agent_type": "DHCP agent", "configurations": {"subnets": 0, "use_namespaces": true, "dhcp_lease_duration": 86400, "dhcp_driver": "neutron.agent.linux.dhcp.Dnsmasq", "networks": 0, "ports": 0}}}, "time": "2014-02-05T01:40:00.279910"}, "namespace": null, "_unique_id": "c03bacfe49a94ee6a8e038f696be74c5", "_context_timestamp": "2014-02-05 01:40:00.279761", "_context_is_admin": true, "version": "1.0", "_context_project_id": null, "_context_tenant_id": null, "_context_user_id": null, "method": "report_state"}', 'oslo.version': '2.0'}) msg_acked /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:1271
2014-02-04 17:40:00.295 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: SessionDetach(name='ab612efb-6727-41c1-a542-edc60a14633d:24') write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685
2014-02-04 17:40:00.295 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: "\x0f\x00\x009\x00\x00\x00\x00\x00\x00\x00\x00\x02\x03\x01\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:24" writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480
2014-02-04 17:40:00.296 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: "\x0f\x00\x00:\x00\x00\x00\x00\x00\x00\x00\x00\x02\x04\x03\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:24\x00" readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416
2014-02-04 17:40:00.296 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionDetached(name='ab612efb-6727-41c1-a542-edc60a14633d:24', code=0) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.297 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: SessionAttach(name='ab612efb-6727-41c1-a542-edc60a14633d:25') write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685
2014-02-04 17:40:00.297 10487 DEBUG qpid.messaging.io.ops [-] SENT[25d9cf8]: SessionCommandPoint(command_id=serial(0), command_offset=0) write_op /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:685
2014-02-04 17:40:00.297 10487 DEBUG qpid.messaging.io.raw [-] SENT[25d9cf8]: "\x0f\x00\x009\x00\x00\x00\x00\x00\x00\x00\x00\x02\x01\x01\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:25\x0f\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x00\x02\x07\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" writeable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:480
2014-02-04 17:40:00.298 10487 DEBUG qpid.messaging.io.raw [-] READ[25d9cf8]: "\x0f\x00\x009\x00\x00\x00\x00\x00\x00\x00\x00\x02\x02\x01\x00\x00'ab612efb-6727-41c1-a542-edc60a14633d:25\x0f\x00\x00\x1c\x00\x00\x00\x00\x00\x00\x00\x00\x02\x07\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" readable /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:416
2014-02-04 17:40:00.298 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionAttached(name='ab612efb-6727-41c1-a542-edc60a14633d:25') write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653
2014-02-04 17:40:00.298 10487 DEBUG qpid.messaging.io.ops [-] RCVD[25d9cf8]: SessionCommandPoint(command_id=serial(0), command_offset=0) write /usr/lib/python2.6/site-packages/qpid/messaging/driver.py:653

Question information

Language:
English Edit question
Status:
Solved
For:
neutron Edit question
Assignee:
No assignee Edit question
Solved by:
Kausum Kumar
Solved:
Last query:
Last reply:
Revision history for this message
yong sheng gong (gongysh) said : 		#1

I think you have to use namespace enabled kernel and iproute package from RDO,
otherwise you have to disable namespace.

Revision history for this message
Alfred Shen (alfredcs) said : 		#2

Besides the suggested namespace, you might also want to double check your neutron conf to make sure agents can communicate with qpid on controller.

Revision history for this message
Kausum Kumar (kausum-kumar) said : 		#3

Thanks. I had the iproute package that supports namespace, but my kernel was not updated. I realized that the order of repositories was incorrect. This did not allow the namespace supported kernel.