I'm deploying openstack icehouse version with three-node architecture (controller node, network node, and compute node1) on Debian wheezy.
When I try to create new instance, I got the following error in my /var/log/nova/nova-compute.log(computeNode1):
2014-07-02 10:35:51.958 17976 WARNING nova.virt.disk.vfs.guestfs [req-ea863486-b2e2-405e-92b5-256313ad6796 535734c287154185a7ab705a0c446553 d8a4955dbd9d450287afe06a2a0a7d9c] Failed to close augeas aug_close: do_aug_close: you must call 'aug-init' first to initialize Augeas
2014-07-02 10:35:52.334 17976 ERROR nova.compute.manager [req-ea863486-b2e2-405e-92b5-256313ad6796 535734c287154185a7ab705a0c446553 d8a4955dbd9d450287afe06a2a0a7d9c] [instance: 43aa4cfc-4f48-4b69-b004-1c94aacb73f4] Instance failed to spawn
2014-07-02 10:35:52.334 17976 TRACE nova.compute.manager [instance: 43aa4cfc-4f48-4b69-b004-1c94aacb73f4]
2014-07-02 10:35:52.444 17976 ERROR nova.virt.libvirt.driver [-] [instance: 43aa4cfc-4f48-4b69-b004-1c94aacb73f4] During wait destroy, instance disappeared.
2014-07-02 10:35:53.014 17976 ERROR nova.compute.manager [req-ea863486-b2e2-405e-92b5-256313ad6796 535734c287154185a7ab705a0c446553 d8a4955dbd9d450287afe06a2a0a7d9c] [instance: 43aa4cfc-4f48-4b69-b004-1c94aacb73f4] Error: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf brctl addbr qbr9a10ed23-a0
Exit code: 97
Stdout: ''
2014-07-02 10:36:01.012 17976 ERROR nova.openstack.common.periodic_task [-] Error during ComputeManager._run_image_cache_manager_pass: Unexpected error while running command.
Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chown 106 /var/lib/nova/instances/_base/9d0e2db99fa349c8b32613d554acbb6c9ce68079
Exit code: 97
2014-07-02 10:36:42.702 17976 TRACE oslo.messaging.rpc.dispatcher raise exception.InstanceNotFound(instance_id=instance_name)
2014-07-02 10:36:42.702 17976 TRACE oslo.messaging.rpc.dispatcher InstanceNotFound: Instance instance-0000000f could not be found.
2014-07-02 10:36:42.702 17976 TRACE oslo.messaging.rpc.dispatcher
2014-07-02 10:36:42.705 17976 ERROR oslo.messaging._drivers.common [-] Returning exception Instance instance-0000000f could not be found. to caller
2014-07-02 10:36:42.706 17976 ERROR oslo.messaging._drivers.common [-] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139, in inner\n return func(*args, **kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped\n payload)\n', ' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped\n return f(self, context, *args, **kw)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 290, in decorated_function\n return function(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3927, in get_console_output\n output = self.driver.get_console_output(context, instance)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2294, in get_console_output\n virt_dom = self._lookup_by_name(instance.name)\n', ' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3488, in _lookup_by_name\n raise exception.InstanceNotFound(instance_id=instance_name)\n', 'InstanceNotFound: Instance instance-0000000f could not be found.\n']
2014-07-02 10:36:55.155 17976 WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor.
And got the following errors in /var/log/nova/nova-scheduler.log (ControllerNode):
2014-07-02 10:35:53.045 15167 ERROR nova.scheduler.filter_scheduler [req-ea863486-b2e2-405e-92b5-256313ad6796 535734c287154185a7ab705a0c446553 d8a4955dbd9d450287afe06a2a0a7d9c] [instance: 43aa4cfc-4f48-4b69-b004-1c94aacb73f4] Error from last host: hadoop03 (node hadoop03): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1305, in _build_instance\n set_access_ip=set_access_ip)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 393, in decorated_function\n return function(self, context, *args, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1717, in _spawn\n LOG.exception(_(\'Instance failed to spawn\'), instance=instance)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', u' File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1714, in _spawn\n block_device_info)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2263, in spawn\n block_device_info)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3648, in _create_domain_and_network\n self.plug_vifs(instance, network_info)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 859, in plug_vifs\n self.vif_driver.plug(instance, vif)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 601, in plug\n self.plug_ovs(instance, vif)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 466, in plug_ovs\n self.plug_ovs_hybrid(instance, vif)\n', u' File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 446, in plug_ovs_hybrid\n utils.execute(\'brctl\', \'addbr\', br_name, run_as_root=True)\n', u' File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 164, in execute\n return processutils.execute(*cmd, **kwargs)\n', u' File "/usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py", line 193, in execute\n cmd=\' \'.join(cmd))\n', u"ProcessExecutionError: Unexpected error while running command.\nCommand: sudo nova-rootwrap /etc/nova/rootwrap.conf brctl addbr qbr9a10ed23-a0\nExit code: 97\nStdout: ''\nStderr: '/usr/bin/nova-rootwrap: Incorrect configuration file: /etc/nova/rootwrap.conf\\n'\n"]
2014-07-02 10:35:53.052 15167 WARNING nova.scheduler.driver [req-ea863486-b2e2-405e-92b5-256313ad6796 535734c287154185a7ab705a0c446553 d8a4955dbd9d450287afe06a2a0a7d9c] [instance: 43aa4cfc-4f48-4b69-b004-1c94aacb73f4] Setting instance to ERROR state.
The log of neutron seems OK. see following:
2014-07-02 10:35:40.442 18954 INFO neutron.agent.securitygroups_rpc [req-f9da6c2f-bcb4-48c3-8de6-236f80848007 None] Security group member updated [u'1dcfbb08-d717-4f7f-98ff-02e2c5ca6f15']
2014-07-02 10:35:40.691 18954 INFO neutron.agent.securitygroups_rpc [req-76beceb9-755a-42e9-b122-5e4118d758fe None] Provider rule updated
2014-07-02 10:35:41.343 18954 INFO neutron.agent.securitygroups_rpc [-] Refresh firewall rules
2014-07-02 10:35:41.343 18954 INFO neutron.agent.securitygroups_rpc [-] No ports here to refresh firewall
2014-07-02 10:35:52.827 18954 INFO neutron.agent.securitygroups_rpc [req-fbed2687-c21a-4a01-8716-10a3fc6fe291 None] Security group member updated [u'1dcfbb08-d717-4f7f-98ff-02e2c5ca6f15']
I've searched almost one day, links https://ask.openstack.org/en/question/28734/instance-failed-to-spawn-you-must-call-aug-init-first-to-initialize-augeas/ and http://osdir.com/ml/general/2014-05/msg39859.html are similar with mine, but the suggestions given all don't work. Do you have any idea about this?
ps: following are compute services (basically ok):
nova-conductor hadoop01 internal enabled up 0 mins
nova-cert hadoop01 internal enabled up 0 mins
nova-consoleauth hadoop01 internal enabled up 0 mins
nova-scheduler hadoop01 internal enabled up 0 mins
nova-compute hadoop03 nova enabled up 0 mins
