Large number of VMs (~1000) and libvirt exception

Asked by Pawel Koniszewski

Hi,

I have a problem with uncaught exception in nova. I was not able to schedule one thousand of VMs, because the closer I was to schedule all needed VMs, the more VMs were throwing exceptions. This is what 'nova show' shows:

 {u'message': u'AttributeError', u'code': 500, u'details': u'\'NoneType\' object has no attribute \'lookupByName\'
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 224, in decorated_function
  return function(self, context, *args, **kwargs)
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1224, in run_instance
  do_run_instance()
File "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line 242, in inner
  retval = f(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1223, in do_run_instance
  admin_password, is_first_time, node, instance)
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 879, in _run_instance
  self._set_instance_error_state(context, instance[\'uuid\'])
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
  self.gen.next()
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 790, in _run_instance
  self._check_instance_exists(context, instance)
File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 987, in _check_instance_exists
  if self.driver.instance_exists(instance[\'name\']):
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 647, in instance_exists
  self._lookup_by_name(instance_name)
File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2354, in _lookup_by_name
  return self._conn.lookupByName(instance_name)
', u'created': u'2013-09-03T11:57:38Z'}

Any suggestions on that?

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Tiantian Gao (gtt116) said :
#1

Hi, Pawel

You create 1,000 instances at one request? That was crazy!! wah

It seems like libvirt is crash or not connected.

2013/9/5 Paweł Koniszewski <email address hidden>

> New question #235190 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/235190
>
> Hi,
>
> I have a problem with uncaught exception in nova. I was not able to
> schedule one thousand of VMs, because the closer I was to schedule all
> needed VMs, the more VMs were throwing exceptions. This is what 'nova show'
> shows:
>
> {u'message': u'AttributeError', u'code': 500, u'details': u'\'NoneType\'
> object has no attribute \'lookupByName\'
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 224,
> in decorated_function
> return function(self, context, *args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 1224, in run_instance
> do_run_instance()
> File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line
> 242, in inner
> retval = f(*args, **kwargs)
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 1223, in do_run_instance
> admin_password, is_first_time, node, instance)
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 879,
> in _run_instance
> self._set_instance_error_state(context, instance[\'uuid\'])
> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> self.gen.next()
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 790,
> in _run_instance
> self._check_instance_exists(context, instance)
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 987,
> in _check_instance_exists
> if self.driver.instance_exists(instance[\'name\']):
> File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
> 647, in instance_exists
> self._lookup_by_name(instance_name)
> File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
> 2354, in _lookup_by_name
> return self._conn.lookupByName(instance_name)
> ', u'created': u'2013-09-03T11:57:38Z'}
>
> Any suggestions on that?
>
> --
> You received this question notification because you are an answer
> contact for OpenStack Compute (nova).
>

--
Best regards,
gtt

Can you help with this problem?

Provide an answer of your own, or ask Pawel Koniszewski for more information if necessary.

To post a message you must log in.