VMDK images will not start on ESXi 4.1 Reason: The file specified is not a virtual disk

Asked by Joel Eichelberger on 2011-10-13

Hello,

I am running the Cactus 2011.2 build on RHEL 5 using ESXi 4.1 for a hypervisor. I am using Glance for the image service. When I start an VM instance, it builds a empty VM instance on the ESXi host. It provisions the image size to the same size as the .vmdk image in Glance. However, its actual size is 0 KB, and when it tries to start I get the error The file specified is not a virtual disk. and "Cannot open the disk '/vmfs/volumes/a_unique_id/instance-00000062/instance-00000062.vmdk' or one of the snapshot disks it depends on."

The RHEL Nova Controller/Compute/Glance node is a VM on the ESXi host. The .vmdk image in question was created on the ESXi host, then moved into the Nova Controller/Compute VM. I have tried serving the image from a webserver not on the ESXi host (or on the RHEL VM) it is the same machine as I am using to serve the wsdl. To add the image to the Glance server I used:

glance add name="Small Image" disk_format=vmdk container_format=ovf is_public=true location=file:///opt/image/smallImage-flat.vmdk

Thank you in advance for the advice.

The stack trace from the nova-compute.log is:

2011-10-13 07:33:47,729 DEBUG suds.metrics [-] method 'RetrieveProperties' invoked: 155 (ms) from (pid=4739) invoke build/bdist.linux-x86_64/egg/suds/client.py:607
2011-10-13 07:33:47,729 DEBUG suds.sudsobject [-] ['value', '_type'] must be superset of [u'_type'], ordering ignored from (pid=4739) __keylist build/bdist.linux-x86_64/egg/suds/sudsobject.py:206
2011-10-13 07:33:47,729 WARNING nova.virt.vmwareapi_conn [-] Task [PowerOnVM_Task] (returnval){
   value = "haTask-400-vim.VirtualMachine.powerOn-5819"
   _type = "Task"
 } status: error Reason: The file specified is not a virtual disk.
2011-10-13 07:33:47,731 ERROR nova.compute.manager [IGPUYPWZ7710FXDYIPP4 root aProject] Instance '97' failed to spawn. Is virtualization enabled in the BIOS?
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/nova/compute/manager.py", line 234, in run_instance
(nova.compute.manager): TRACE: self.driver.spawn(instance_ref)
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/nova/virt/vmwareapi_conn.py", line 129, in spawn
(nova.compute.manager): TRACE: self._vmops.spawn(instance)
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/nova/virt/vmwareapi/vmops.py", line 323, in spawn
(nova.compute.manager): TRACE: _power_on_vm()
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/nova/virt/vmwareapi/vmops.py", line 321, in _power_on_vm
(nova.compute.manager): TRACE: self._session._wait_for_task(instance.id, power_on_task)
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/nova/virt/vmwareapi_conn.py", line 346, in _wait_for_task
(nova.compute.manager): TRACE: ret_val = done.wait()
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/eventlet-0.9.15-py2.6.egg/eventlet/event.py", line 116, in wait
(nova.compute.manager): TRACE: return hubs.get_hub().switch()
(nova.compute.manager): TRACE: File "/opt/openStack/lib/python2.6/site-packages/eventlet-0.9.15-py2.6.egg/eventlet/hubs/hub.py", line 177, in switch
(nova.compute.manager): TRACE: return self.greenlet.switch()
(nova.compute.manager): TRACE: Error: Reason: The file specified is not a virtual disk.
(nova.compute.manager): TRACE:
2011-10-13 07:33:47,803 DEBUG suds.resolver [-] searching schema for (ns0:SelectionSpec) from (pid=4739) root build/bdist.linux-x86_64/egg/suds/resolver.py:128
2011-10-13 07:33:47,804 DEBUG suds.xsd.query [-] ('SelectionSpec', urn:vim25), found as: <Complex:0x18488990 name="SelectionSpec" /> from (pid=4739) result build/bdist.linux-x86_64/egg/suds/xsd/query.py:85
2011-10-13 07:33:47,804 DEBUG suds.resolver [-] found (ns0:SelectionSpec) as (<Complex:0x18488990 name="SelectionSpec" />) from (pid=4739) root build/bdist.linux-x86_64/egg/suds/resolver.py:136

my nova-conf looks like:
--daemonize=1
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/opt/openStack/bin/nova-dhcpbridge
--lock_path=/tmp
--logdir=/var/log/nova
--state_path=/var/lib/nova
--verbose
--network_manager=nova.network.manager.FlatManager
--sql_connection=mysql://root:password@127.0.0.1/nova
--rabbit_host=127.0.0.1
--ec2_host=127.0.0.1
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=127.0.0.1:9292
--glance_host=127.0.0.1
--glance_port=9292
--use_s3=false
--connection_type=vmwareapi
--vmwareapi_host_ip=172.20.11.11
--vmwareapi_host_username=user
--vmwareapi_host_password=password
--vmwareapi_wsdl_loc=http://172.20.3.71:8080/vimService.wsdl
--console_manager=nova.console.vmrc_manager.ConsoleVMRCManager

The .vmx file created on the ESXi host during instance spawn is:
.encoding = "UTF-8"
config.version = "8"
virtualHW.version = "7"
pciBridge0.present = "true"
pciBridge4.present = "true"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "true"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "true"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "true"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
vmci0.present = "true"
nvram = "instance-00000062.nvram"
virtualHW.productCompatibility = "hosted"
powerType.powerOff = "soft"
powerType.powerOn = "hard"
powerType.suspend = "hard"
powerType.reset = "soft"
displayName = "instance-00000062"
extendedConfigFile = "instance-00000062.vmxf"
memsize = "2048"
ethernet0.present = "true"
ethernet0.networkName = "openstackNet"
ethernet0.addressType = "static"
ethernet0.address = "02:16:3e:13:98:cd"
guestOS = "other"
uuid.location = "56 4d cb 4a 4c 47 a4 40-41 1e 24 d8 ae 44 39 b0"
uuid.bios = "56 4d cb 4a 4c 47 a4 40-41 1e 24 d8 ae 44 39 b0"
vc.uuid = "52 96 0a ce 05 85 e1 f8-66 3e 23 21 9a 1a 4a 45"
machine.id = "02:16:3e:13:98:cd;172.20.11.35;255.255.255.252;172.20.11.33"
scsi0.present = "TRUE"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsilogic"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "instance-00000062.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
floppy0.present = "FALSE"
replay.supported = "FALSE"
sched.swap.derivedName = "/vmfs/volumes/4e8f12f2-d4a05d60-e474-782bcb357be8/instance-00000062/instance-00000062-5b59b3fd.vswp"
replay.filename = ""
scsi0:0.redo = ""

Question information

Language:
English Edit question
Status:
Expired
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Last query:
2011-10-20
Last reply:
2011-11-05
Sateesh (sateesh-chodapuneedi) said : #1

What is the Operating system installed in the image?

Sateesh, Hello.

I've tried a couple of OSs. I tried an Ubuntu 10.10 server, and a RHEL5. Though, the RHEL 5 was built with VMWare Workstation 7. I am going to build an image on the ESXi host of RHEL 5 quick and test that.

-Joel

Oh, and both OSs are 64 bit.

I just double checked, and a RHEL 5 vmdk image created on the ESXi host does't work either. Thanks again.

Sateesh (sateesh-chodapuneedi) said : #5

The images to be used on ESX(i) need be uploaded to Glance with following properties,
vmware_adaptertype (specify the value 'lsiLogic')
vmware_ostype (the value depends on OS inside the image)

For RHEL 5 64 bit guest, the upload command looks like,
glance add name="Small Image" disk_format=raw container_format=bare is_public=true location=file:///opt/image/smallImage-flat.vmdk vmware_adaptertype='lsiLogic' vmware_ostype="rhel5_64Guest"

For Ubuntu 10.10 64 bit guest, vmware_ostype would be "ubuntu64Guest"

Hope that helps.

Sateesh,

I just deleted the instance from Glance, and then added it back. I get the same error. Here are the details of that image.

Thanks for the help!

[root@localhost ~]# glance show 23
URI: http://0.0.0.0/images/23
Id: 23
Public: Yes
Name: rhelSmall
Size: 0
Location: file:///opt/tmp/rhel_vsphere/JOEL_RHEL_TEST-flat.vmdk
Disk format: raw
Container format: bare
Property 'vmware_ostype': rhel5_64Guest
Property 'vmware_adaptertype': lsiLogic
[root@localhost ~]#

Sateesh (sateesh-chodapuneedi) said : #7

Hi Joel,

>>Name: rhelSmall
>>Size: 0

Image size shows up as 0. Can you try after a re-uploading the image?

So, I've tried doing that a number of times, and it is always has a size 0. Images which are RAW show up with sizes. When the ESXi host creates the VM-instance.vmdk, that file is correctly provisioned. However, it doesn't have an actual size. I was originally concerned by the size being 0 as well, but I assumed, since it was being provisioned the exact size of the image in Glance, it was actually getting size information from somewhere and the displaying of size was just a bug.

Thanks again.

Sateesh,

Good morning. Do you have any other thoughts on this?

Hi Joel,

Nova looks up image metadata before attempting to provision an empty virtual disk of the size found in image metadata. So essentially glance meta data should point to correct size of the image. Otherwise disk provision fails, which results in a spawn failure. Can you please try to upload the image again and see if "glance show ##" does show the correct image size?

Launchpad Janitor (janitor) said : #11

This question was expired because it remained in the 'Open' state without activity for the last 15 days.