OpenStack Compute (nova)

Error connecting to XenServer

Asked by Laurent Borensky

Hello,

I get an error with XenServer. I do not understand. Do you have an idea in relation to this error?

ERROR nova.compute.manager [6ORAHQW0FV8ZJUL5MBTF client1 appli1] Instance '1' failed to spawn. Is virtualization enabled in the BIOS?

Regards,

===============
Details (context, log, ...)
===============
# The error is relative to the X server stopped
root@6Cloud01:/var/log/nova# cat nova-compute.log <CR>
2011-04-22 14:25:31,400 ERROR nova [-] in Service.create()
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/home/openstack/nova/nova/service.py", line 270, in serve
(nova): TRACE: services = [Service.create()]
(nova): TRACE: File "/home/openstack/nova/nova/service.py", line 173, in create
(nova): TRACE: report_interval, periodic_interval)
(nova): TRACE: File "/home/openstack/nova/nova/service.py", line 73, in __init__
(nova): TRACE: self.manager = manager_class(host=self.host, *args, **kwargs)
(nova): TRACE: File "/home/openstack/nova/nova/compute/manager.py", line 127, in __init__
(nova): TRACE: utils.import_object(compute_driver),
(nova): TRACE: File "/home/openstack/nova/nova/utils.py", line 76, in import_object
(nova): TRACE: return cls()
(nova): TRACE: File "/home/openstack/nova/nova/virt/connection.py", line 69, in get_connection
(nova): TRACE: conn = xenapi_conn.get_connection(read_only)
(nova): TRACE: File "/home/openstack/nova/nova/virt/xenapi_conn.py", line 160, in get_connection
(nova): TRACE: return XenAPIConnection(url, username, password)
(nova): TRACE: File "/home/openstack/nova/nova/virt/xenapi_conn.py", line 168, in __init__
(nova): TRACE: session = XenAPISession(url, user, pw)
(nova): TRACE: File "/home/openstack/nova/nova/virt/xenapi_conn.py", line 328, in __init__
(nova): TRACE: self._session.login_with_password(user, pw)
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/XenAPI.py", line 182, in <lambda>
(nova): TRACE: return lambda *params: self._login(name, params)
(nova): TRACE: File "/usr/local/lib/python2.6/dist-packages/XenAPI.py", line 148, in _login
(nova): TRACE: result = _parse_result(getattr(self, 'session.%s' % method)(*params))
(nova): TRACE: File "/usr/lib/python2.6/xmlrpclib.py", line 1199, in __call__
(nova): TRACE: return self.__send(self.__name, args)
(nova): TRACE: File "/usr/lib/python2.6/xmlrpclib.py", line 1489, in __request
(nova): TRACE: verbose=self.__verbose
(nova): TRACE: File "/usr/lib/python2.6/xmlrpclib.py", line 1235, in request
(nova): TRACE: self.send_content(h, request_body)
(nova): TRACE: File "/usr/lib/python2.6/xmlrpclib.py", line 1349, in send_content
(nova): TRACE: connection.endheaders()
(nova): TRACE: File "/usr/lib/python2.6/httplib.py", line 904, in endheaders
(nova): TRACE: self._send_output()
(nova): TRACE: File "/usr/lib/python2.6/httplib.py", line 776, in _send_output
(nova): TRACE: self.send(msg)
(nova): TRACE: File "/usr/lib/python2.6/httplib.py", line 735, in send
(nova): TRACE: self.connect()
(nova): TRACE: File "/usr/lib/python2.6/httplib.py", line 1108, in connect
(nova): TRACE: sock = socket.create_connection((self.host, self.port), self.timeout)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/eventlet/green/socket.py", line 59, in create_connection
(nova): TRACE: raise error, msg
(nova): TRACE: error: [Errno 113] EHOSTUNREACH
(nova): TRACE:

root@6Cloud01:/home/openstack# nova image-list <CR>
+----+---------------------+--------+
| ID | Name | Status |
+----+---------------------+--------+
| 1 | Lunix-tty-ramdisk | ACTIVE |
| 2 | Lunix-tty-kernel | ACTIVE |
| 3 | Lunix-tty-image | ACTIVE |
| 4 | Lunix-lucid-ramdisk | ACTIVE |
| 5 | Lunix-lucid-kernel | ACTIVE |
| 6 | Lunix-lucid-image | ACTIVE |
+----+---------------------+--------+
root@6Cloud01:/home/openstack# euca-describe-images <CR>
IMAGE ari-00000001 None (Lunix-tty-ramdisk) available public ramdisk
IMAGE aki-00000002 None (Lunix-tty-kernel) available public kernel
IMAGE ami-00000003 None (Lunix-tty-image) available public machine
IMAGE ari-00000004 None (Lunix-lucid-ramdisk) available public ramdisk
IMAGE aki-00000005 None (Lunix-lucid-kernel) available public kernel
IMAGE ami-00000006 None (Lunix-lucid-image) available public machine
root@6Cloud01:/home/openstack#

root@6Cloud01:/home/openstack# nova boot VM01 --flavor 2 --image 5 <CR>
+-----------+---------------------------------+
| Property | Value |
+-----------+---------------------------------+
| addresses | {u'public': [], u'private': []} |
| adminPass | kvMWBc9bCuWZUgMg |
| flavorId | 2 |
| hostId | |
| id | 1 |
| imageId | 5 |
| metadata | {} |
| name | VM01 |
| status | BUILD |
+-----------+---------------------------------+
root@6Cloud01:/home/openstack# nova list <CR>
+----+------+--------+-----------+------------+
| ID | Name | Status | Public IP | Private IP |
+----+------+--------+-----------+------------+
| 1 | VM01 | BUILD | | 10.0.0.2 |
+----+------+--------+-----------+------------+
root@6Cloud01:/home/openstack# nova list <CR>
+----+------+--------+-----------+------------+
| ID | Name | Status | Public IP | Private IP |
+----+------+--------+-----------+------------+
| 1 | VM01 | ERROR | | 10.0.0.2 |
+----+------+--------+-----------+------------+
root@6Cloud01:/home/openstack# extract /var/log/nova/nova-compute.log <CR>
2011-04-22 14:50:06,213 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenserver vm state -> |Running|
2011-04-22 14:50:06,213 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenapi power_state -> |1|
2011-04-22 14:51:09,136 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenserver vm state -> |Running|
2011-04-22 14:51:09,136 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenapi power_state -> |1|
2011-04-22 14:52:12,017 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenserver vm state -> |Running|
2011-04-22 14:52:12,018 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenapi power_state -> |1|
2011-04-22 14:53:14,908 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenserver vm state -> |Running|
2011-04-22 14:53:14,909 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenapi power_state -> |1|
2011-04-22 14:53:43,765 DEBUG nova.rpc [-] received {u'_context_request_id': u'6ORAHQW0FV8ZJUL5MBTF', u'_context_read_deleted': False, u'args': {u'instance_id': 1, u'injected_files': [], u'availability_zone': None}, u'_context_is_admin': True, u'_context_timestamp': u'2011-04-22T12:53:43Z', u'_context_user': u'client1', u'method': u'run_instance', u'_context_project': u'appli1', u'_context_remote_address': None} from (pid=3736) _receive /home/openstack/nova/nova/rpc.py:167
2011-04-22 14:53:43,765 DEBUG nova.rpc [-] unpacked context: {'timestamp': u'2011-04-22T12:53:43Z', 'remote_address': None, 'project': u'appli1', 'is_admin': True, 'user': u'client1', 'request_id': u'6ORAHQW0FV8ZJUL5MBTF', 'read_deleted': False} from (pid=3736) _unpack_context /home/openstack/nova/nova/rpc.py:331
2011-04-22 14:53:46,557 AUDIT nova.compute.manager [6ORAHQW0FV8ZJUL5MBTF client1 appli1] instance 1: starting...
2011-04-22 14:53:46,659 DEBUG nova.rpc [-] Making asynchronous call on network ... from (pid=3736) call /home/openstack/nova/nova/rpc.py:350
2011-04-22 14:53:46,660 DEBUG nova.rpc [-] MSG_ID is 6816f34ded4546caba0c7e6c76c179a6 from (pid=3736) call /home/openstack/nova/nova/rpc.py:353
2011-04-22 14:53:46,936 DEBUG nova.rpc [-] Making asynchronous call on network.6Cloud01 ... from (pid=3736) call /home/openstack/nova/nova/rpc.py:350
2011-04-22 14:53:46,936 DEBUG nova.rpc [-] MSG_ID is e8ce9c2ece8c4d5cabb74701fd6bd5e1 from (pid=3736) call /home/openstack/nova/nova/rpc.py:353
2011-04-22 14:53:47,400 DEBUG nova.virt.xenapi.vm_utils [-] Detected KERNEL_RAMDISK format for image 5, instance 1 from (pid=3736) log_disk_format /home/openstack/nova/nova/virt/xenapi/vm_utils.py:494
2011-04-22 14:53:47,630 DEBUG nova.virt.xenapi.vm_utils [-] Size for image 5:4099360 from (pid=3736) _fetch_image_glance_disk /home/openstack/nova/nova/virt/xenapi/vm_utils.py:442
2011-04-22 14:53:48,626 DEBUG nova.virt.xenapi.vm_utils [-] Created VDI OpaqueRef:8908e5b2-fe39-070c-4d6c-fba396a7ecfb (Glance image 5, 4099360, False) on OpaqueRef:506f6691-e685-d001-c84d-d9e900c78e36. from (pid=3736) create_vdi /home/openstack/nova/nova/virt/xenapi/vm_utils.py:277
2011-04-22 14:53:48,627 ERROR nova.compute.manager [6ORAHQW0FV8ZJUL5MBTF client1 appli1] Instance '1' failed to spawn. Is virtualization enabled in the BIOS?
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/compute/manager.py", line 234, in run_instance
(nova.compute.manager): TRACE: self.driver.spawn(instance_ref)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi_conn.py", line 188, in spawn
(nova.compute.manager): TRACE: self._vmops.spawn(instance)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vmops.py", line 117, in spawn
(nova.compute.manager): TRACE: vdi_uuid = self._create_disk(instance)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vmops.py", line 113, in _create_disk
(nova.compute.manager): TRACE: instance.image_id, user, project, disk_image_type)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vm_utils.py", line 382, in fetch_image
(nova.compute.manager): TRACE: access, image_type)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vm_utils.py", line 537, in _fetch_image_glance
(nova.compute.manager): TRACE: session, instance_id, image, access, image_type)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vm_utils.py", line 452, in _fetch_image_glance_disk
(nova.compute.manager): TRACE: lambda dev:
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vm_utils.py", line 908, in with_vdi_attached_here
(nova.compute.manager): TRACE: this_vm_ref = get_this_vm_ref(session)
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vm_utils.py", line 989, in get_this_vm_ref
(nova.compute.manager): TRACE: return session.get_xenapi().VM.get_by_uuid(get_this_vm_uuid())
(nova.compute.manager): TRACE: File "/home/openstack/nova/nova/virt/xenapi/vm_utils.py", line 984, in get_this_vm_uuid
(nova.compute.manager): TRACE: with file('/sys/hypervisor/uuid') as f:
(nova.compute.manager): TRACE: IOError: [Errno 2] No such file or directory: '/sys/hypervisor/uuid'
(nova.compute.manager): TRACE:
2011-04-22 14:54:17,799 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenserver vm state -> |Running|
2011-04-22 14:54:17,799 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenapi power_state -> |1|
2011-04-22 14:54:20,243 INFO nova.compute.manager [-] Found instance 'instance-00000001' in DB but no VM. State=8, so setting state to shutoff.
2011-04-22 14:54:20,243 INFO nova.compute.manager [-] DB/VM state mismatch. Changing state from '8' to '5'
2011-04-22 14:55:20,570 INFO nova.virt.xenapi.vm_utils [-] (VM_UTILS) xenserver vm state -> |Running|

# file "uuid" exist on XenServer
root@6Cloud01:/home/openstack/nova/nova/virt/xenapi# ssh root@146.5.2.3 <CR>
root@146.5.2.3's password:
Last login: Wed Apr 20 22:35:29 2011 from 146.5.2.3
Type "xsconsole" for access to the management console.
[root@xenserver-01 ~]# ls -l /sys/hypervisor/uuid <CR>
-r--r--r-- 1 root root 4096 Apr 22 15:07 /sys/hypervisor/uuid
[root@xenserver-01 ~]# cat /sys/hypervisor/uuid <CR>
a618a457-0c72-4e45-8e78-e2ea9790394c
[root@xenserver-01 ~]#

# Other info (nova.conf, XenAPI test, ...)
root@6Cloud01:/home/openstack# cat nova.conf
--verbose
--nodaemon
--my_ip=146.5.2.2
--network_manager=nova.network.manager.FlatManager
--sql_connection=mysql://root:nova@localhost/nova
--image_service=nova.image.glance.GlanceImageService
--logdir=/var/log/nova
--auth_driver=nova.auth.dbdriver.DbDriver
--connection_type=xenapi
--xenapi_connection_url=https://146.5.2.3/
--xenapi_connection_username=root
--xenapi_connection_password=CloudOMC
--xenapi_inject_image=true
--rescue_timeout=86400
--allow_admin_api=true
root@6Cloud01:/home/openstack#

root@6Cloud01:/home/openstack# nova-manage --flagfile=/home/openstack/nova.conf shell python <CR>
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import XenAPI
>>> import nova.virt.xenapi_conn
>>> nova.virt.xenapi_conn.XenAPI = XenAPI
>>> x = nova.virt.xenapi_conn.XenAPIConnection("https://146.5.2.3", "root", "CloudOMC")
>>> x.list_instances()
['Ubuntu10']
>>> ^D
root@6Cloud01:/home/openstack#

Question information

Language:
French Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Laurent Borensky
Solved:
Last query:
Last reply:
Revision history for this message
Laurent Borensky (laurent-borensky) said : 		#1

Other information about XenServer initialization
=============================

root@6Cloud01:~# ssh root@146.5.2.3 <CR>
root@146.5.2.3's password:
Last login: Sun Apr 24 15:23:03 2011 from 146.5.2.53
Type "xsconsole" for access to the management console.

[root@xenserver-01 ~]# cd /etc/xapi.d <CR>
host-post-declare-dead plugins
[root@xenserver-01 /etc/xapi.d]# ls -l <CR>
total 8
drwxr-xr-x 2 root root 4096 Apr 19 22:53 host-post-declare-dead
drwxr-xr-x 2 root root 4096 Apr 20 10:23 plugins

[root@xenserver-01 xapi.d]# xe sr-list <CR>
uuid ( RO) : 7ed9b60c-367f-6c25-3ef7-dd492acd8996
          name-label ( RW): Local storage
    name-description ( RW):
                host ( RO): xenserver-01
                type ( RO): lvm
        content-type ( RO): user

uuid ( RO) : 1e268947-71b9-a98c-3d69-bceeb268a247
          name-label ( RW): XenServer Tools
    name-description ( RW): XenServer Tools ISOs
                host ( RO): xenserver-01
                type ( RO): iso
        content-type ( RO): iso

uuid ( RO) : 59d4ddf5-7ad0-a8ed-33e2-e3ffdeb76d19
          name-label ( RW): DVD drives
    name-description ( RW): Physical DVD drives
                host ( RO): xenserver-01
                type ( RO): udev
        content-type ( RO): iso

uuid ( RO) : eb02adf7-eb93-a558-aeae-ac077fc4c961
          name-label ( RW): NFS ISO library
    name-description ( RW): NFS ISO Library [146.5.2.97:/nfs/Data_Xen]
                host ( RO): xenserver-01
                type ( RO): iso
        content-type ( RO): iso

uuid ( RO) : f4e3e527-63d7-0daa-6a97-123226a4eb8b
          name-label ( RW): Removable storage
    name-description ( RW):
                host ( RO): xenserver-01
                type ( RO): udev
        content-type ( RO): disk
 <CR>
[root@xenserver-01 xapi.d]# xe sr-param-list uuid=7ed9b60c-367f-6c25-3ef7-dd492acd8996
uuid ( RO) : 7ed9b60c-367f-6c25-3ef7-dd492acd8996
              name-label ( RW): Local storage
        name-description ( RW):
                    host ( RO): xenserver-01
      allowed-operations (SRO): forget; VDI.create; VDI.snapshot; plug; update; destroy; VDI.destroy; scan; VDI.clone; VDI.resize; unplug
      current-operations (SRO):
                    VDIs (SRO): 32c6a12e-4d87-4ad4-951e-3b1f1a557be9; 24c6fd90-57eb-47c3-b290-d1bab2dc89ee; 08ef59b9-8061-4cd3-91d7-d952bb8e3f29; 1f1e9c9f-7d2c-466c-99bb-314e1dc1e5bb; 04589c1b-5522-4998-8410-dc54140b4170; fa05d438-117b-4f92-a1a4-a3c9908a067a; c821a620-da0b-4f39-a4a2-5f2ff7f085e2; 08e969f9-4178-46d1-b02a-a9341c9bc87d
                    PBDs (SRO): 7ebf167b-845b-c68a-96f1-82a17bbce036
      virtual-allocation ( RO): 6547308544
    physical-utilisation ( RO): 6559891456
           physical-size ( RO): 491501125632
                    type ( RO): lvm
            content-type ( RO): user
                  shared ( RW): false
            other-config (MRW): i18n-key: local-storage; i18n-original-value-name_label: Local storage
               sm-config (MRO): allocation: thick; use_vhd: true; devserial: scsi-SATA_Hitachi_HTS7250100902PCL400VLK8ST6J
                   blobs ( RO):
     local-cache-enabled ( RO): false
                    tags (SRW):

Revision history for this message
Laurent Borensky (laurent-borensky) said : 		#2

this code (extract of vm_utils.py) seems wrong because the code is for a local file.

def get_this_vm_uuid():
    with file('/sys/hypervisor/uuid') as f:
        return f.readline().strip()

def get_this_vm_ref(session):
    return session.get_xenapi().VM.get_by_uuid(get_this_vm_uuid())

Revision history for this message
Dan Prince (dan-prince) said : 		#3

Hi Laurent,

Are you running nova-compute as a guest VM on the actual XenServer it controls? The existing nova xenapi code requires that nova-compute be running as a Guest machine on the Xenserver that it controls.

It works something like this:

1) nova-compute is running in a guest VM (can be Ubuntu, Debian, etc.) The nova.conf file is configured so that it communicates with xenapi to XenServer host machine.

2) When handling a request to provision a new instance nova-compute sends a request to Xenserver over xenapi to create a new VDI. In doing this it checks '/sys/hypervisor/uuid' to obtain the id of the nova-compute guest VM (running on the actual XenServer). The uuid file should exist in XenServer guest VM automatically (you shouldn't have to create it).

3) XenServer then creates the VDI and configures a new device (via the Xenserver nova plugins) on the nova-compute guest VM that writes directly to it. A device at /dev/xvd should then show up in the nova-compute guest VM so that it can stream data to it from glance, etc.

Hope this helps.

Dan

Revision history for this message
Laurent Borensky (laurent-borensky) said : 		#4

Hi Dan,

No, nova-compute is running on another physical server with other services Ubuntu Nova. I must create a VM on XenServer for this service nova-compute? I have not read anything in the documentation (wiki) for this.

Than you.

Revision history for this message
Laurent Borensky (laurent-borensky) said : 		#5

Dan,

Where can I find documentation for installing "nova-compute"in a virtual machine? What will be the configuration file "nova.conf?

Revision history for this message
Laurent Borensky (laurent-borensky) said : 		#6

Thank you Dan.

I'll try With A VM "Nov-compute"on XenServer Host.

Revision history for this message
ROHIT DUMBRE (rohit-dumbre) said : 		#7

Hi Dan , I am facing the same error . Can you tell me a specific way to do this

Revision history for this message
ANBARI Zakaria (abualy4400) said : 		#8

Same error!!
Plz help :/

Revision history for this message
ANBARI Zakaria (abualy4400) said : 		#9

i think i found the problem source , it lies within the dom0 itself, as the xenstored process isnt running correctly, so when you start the domU where the opensatck is installed, you should stop the service with "pkill xenstored" the restart the process using the command "xenstored", all this in the dom0, now when you'll start the opensatck-nova-compue using "service opensatck-nova-compute start" and then use "service opensatck-nova-compute status" you'll notice that it's working, still it's not connected to the cloud controller so i'm working on this problem.