Hi stackers,
I have a question regarding the nova DB. I have currently a cluster of 3 Openstack Essex servers (2012.1-0ubuntu2.1) on Precise. Earlier today, one of the nova node crashed (the service nova-compute from the host too), without noticed it, i tried to terminated an instance on that host. It tried to terminated it.. usually it takes 5 seconds to delete an instances but not this time...because the host was down. I connected the node, restart nova-api, nova-network and finally nova-compute. But nova-compute doesn't start because the instance is not on kvm anymore:
//////////
root@nova-compute:~# virsh list --all
Id Name State
----------------------------------
root@nova-compute:~#
//////////
When i am trying to launch nova-compute here is what I get from the logs:
root@nova-compute:~# service nova-compute start
LOG file:
May 10 14:07:20 nova-compute 2012-05-10 14:07:20 WARNING nova.compute.manager [req-738f927d-ab0a-4da1-b795-6723df600067 None None] trying to reboot a non-running instance: 5ed0222e-3220-4be8-849a-d92916728ef9 (state: 8 expected: 1)
May 10 14:07:20 nova-compute 2012-05-10 14:07:20 CRITICAL nova [-] Instance instance-00000004 could not be found.#0122012-05-10 14:07:20 TRACE nova Traceback (most recent call last):#0122012-05-10 14:07:20 TRACE nova File "/usr/bin/nova-compute", line 49, in <module>#0122012-05-10 14:07:20 TRACE nova service.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 413, in wait#0122012-05-10 14:07:20 TRACE nova _launcher.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 131, in wait#0122012-05-10 14:07:20 TRACE nova service.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 166, in wait#0122012-05-10 14:07:20 TRACE nova return self._exit_event.wait()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait#0122012-05-10 14:07:20 TRACE nova return hubs.get_hub().switch()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 177, in switch#0122012-05-10 14:07:20 TRACE nova return self.greenlet.switch()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 192, in main#0122012-05-10 14:07:20 TRACE nova result = function(*args, **kwargs)#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 101, in run_server#0122012-05-10 14:07:20 TRACE nova server.start()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 162, in start#0122012-05-10 14:07:20 TRACE nova self.manager.init_host()#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 247, in init_host#0122012-05-10 14:07:20 TRACE nova self.reboot_instance(context, instance['uuid'])#0122012-05-10 14:07:20 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrappe
May 10 14:07:20 nova-compute kernel: [ 2745.651930] init: nova-compute main process (4788) terminated with status 1
So, here I am, stuck in this limbo state.. I tried to remove the row from the nova DB but there is a foreign key that block me. And I don't want to destroy my DB (even though I dump nova DB).
I was wondering if you could let me know how can I "clean" nova DB so I can restart nova-compute on that host.
Thanks a lot.
Warik