euca-run-instance instance in pending state

Which scheduler are you using? In other words, do you have --scheduler_driver set in your flagfile?

Vish

On Jan 30, 2012, at 8:15 PM, Larry Wang wrote:

> New question #186362 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/186362
>
> Have openstack cactus clouds based on Redhat6.x, Scientific Linux 6.1 both running fine for few months, both single zone, recently applied OS upgrade and security patches, both experienced images upload and run instance problems. image upload issues were resolved by downgrade python-boto from 2.0 to 1.9, but run instance by using euca-run-instance still having problems, the instance will stay in the "pending" mode, the only error message was from nova-scheduler.log:
>
> 2012-01-30 20:01:36,526 nova.scheduler.manager: WARNING [3a0dc601-835f-4aa3-a61c-5d171feb33f9 lwang test] Driver Method schedule_run_instance missing: 'ZoneScheduler' object has no attribute 'schedule_run_instance'.Reverting to schedule()
> 2012-01-30 20:01:36,530 nova.rpc: ERROR [3a0dc601-835f-4aa3-a61c-5d171feb33f9 lwang test] Exception during message handling
> (nova.rpc): TRACE: Traceback (most recent call last):
> (nova.rpc): TRACE: File "/usr/lib/python2.6/site-packages/nova/rpc/impl_kombu.py", line 620, in _process_data
> (nova.rpc): TRACE: rval = node_func(context=ctxt, **node_args)
> (nova.rpc): TRACE: File "/usr/lib/python2.6/site-packages/nova/scheduler/manager.py", line 103, in _schedule
> (nova.rpc): TRACE: host = real_meth(*args, **kwargs)
> (nova.rpc): TRACE: File "/usr/lib/python2.6/site-packages/nova/scheduler/zone.py", line 55, in schedule
> (nova.rpc): TRACE: raise driver.NoValidHost(_("Scheduler was unable to locate a host"
> (nova.rpc): TRACE: NoValidHost: Scheduler was unable to locate a host for this request. Is the appropriate service running?
> (nova.rpc): TRACE:
>
> Can anyone help?
>
> Thanks in advance
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Yes, here is the scheduler in nova.conf file:
--scheduler_driver=nova.scheduler.zone.ZoneScheduler

Thank You
Larry

can you try?
--scheduler_driver=nova.scheduler.distributed_scheduler.DistributedScheduler
or
--scheduler_driver=nova.scheduler.simple.SimpleScheduler

Thank you again for helping.
I've tried both, but still same situation.
I tried the SimpleScheduler, do db sync, restarted all service, got the following scheduler errors when did the run instance:
2012-01-31 10:56:01,953 nova: Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/rpc.py", line 188, in _receive
(nova): TRACE: rval = node_func(context=ctxt, **node_args)
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/scheduler/manager.py", line 83, in _schedule
(nova): TRACE: **kwargs)
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/scheduler/simple.py", line 65, in schedule_run_instance
(nova): TRACE: raise driver.NoValidHost(_("All hosts have too many cores"))
(nova): TRACE: NoValidHost: All hosts have too many cores
(nova): TRACE:

When I tried the DistributedScheduler, do db sync, restarted all service, got the following NotFound error messages, I think this probably because I'm on cactus release, the DistributedScheduler is not in the RPMs when I configured these clouds:
2012-01-31 10:56:48,601 nova: Class DistributedScheduler cannot be found
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/bin/nova-scheduler", line 48, in <module>
(nova): TRACE: service.serve()
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/service.py", line 270, in serve
(nova): TRACE: services = [Service.create()]
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/service.py", line 173, in create
(nova): TRACE: report_interval, periodic_interval)
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/service.py", line 73, in __init__
(nova): TRACE: self.manager = manager_class(host=self.host, *args, **kwargs)
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/scheduler/manager.py", line 47, in __init__
(nova): TRACE: self.driver = utils.import_object(scheduler_driver)
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/utils.py", line 75, in import_object
(nova): TRACE: cls = import_class(import_str)
(nova): TRACE: File "/usr/lib/python2.6/site-packages/nova/utils.py", line 66, in import_class
(nova): TRACE: raise exception.NotFound(_('Class %s cannot be found') % class_str)
(nova): TRACE: NotFound: Class DistributedScheduler cannot be found
(nova): TRACE:

I just have another new discovery, nova-compute is not come up properly, may be the main reason for all the above errors, but above were the only messages that can be obtained, no other log messages showed up any place:
nova-manage service list:
server.xxxxx.xxx nova-compute enabled XXX 2012-01-30 20:37:39
server.xxxxx.xxx nova-network enabled :-) 2012-01-31 19:02:46
server.xxxxx.xxx nova-scheduler enabled XXX 2012-01-31 18:56:44
server.xxxxx.xxx nova-vncproxy enabled :-) 2012-01-31 19:02:46
server.xxxxx.xxx nova-volume enabled :-) 2012-01-31 19:02:43
(sorry, has to cross out org name, due to corp requirements)

Can anyone help looking on this issue? I am currently unable to deploy any new instances, can't access any instances that are show running state. Just have no clue on how to trouble shoot nova-compute part, the nova-scheduler is working now. I've tried Vish's suggestions, but they either yield errors or can't find that driver. Else the original warnings, can't find any errors meaningful.
Thanks in advance
Larry

Ok, this issue was resolved, the major issue was the libvirt version was upgraded to 0.9.x after the os upgrade and patches and libvirt-0.9.x turned out not very compatible with openstack cactus, at least in my case, after down grade libvirt to 0.8.7, I was able to startup nova-compute, nova-network, and things seemed fine now.
Thanks to you for providing help
Larry