OpenStack Compute (nova)

Parted cannot find device when using glance images in xenapi

Bug #705790 reported by Salvatore Orlando
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Salvatore Orlando
OpenStack Compute (nova) 2011.1 "bexar"

Bug Description

When spawning an image on a xenapi backend, this operation sometimes fails when attempting to run parted for writing the partition table. Parted cannot find the device where the image is going to be streamed.
Failure does not occur when spawning raw images.

For traceback see http://paste.openstack.org/show/532

The 'offending device' was just been mounted by xapi using VBD.plug on the VM where the compute node is running.
A possible explanation is a race condition between xapi returning success on the VBD.plug operation and the device being actually available in the /dev filesystem.

Executing udevsettle before running parted should fix this.

Related branches

lp:~rconradharris/nova/bug705790
Jay Pipes (community): Approve
Ed Leafe (community): Approve
Devin Carlen (community): Approve
Josh Kearney (community): Approve
Diff: 81 lines (+38/-1)
2 files modified
nova/virt/xenapi/vm_utils.py (+30/-1)
nova/virt/xenapi_conn.py (+8/-0)
Salvatore Orlando (salvatore-orlando)
Changed in nova:
assignee: nobody → Salvatore Orlando (salvatore-orlando)
Revision history for this message
Antony Messerli (antonym) wrote :

It appears that Maverick has changed their kernel to use sdX instead of xvdX like lucid did. We can see it being attached in the domU as sdb but it's trying to find xvdb.

[ 227.351327] blkfront device/vbd/51728 num-ring-pages 1 nr_ents 32.
[ 227.355281] blkfront: sdb: barriers enabled
[ 227.355481] sdb: unknown partition table
[57361.504505] blkfront device/vbd/51728 num-ring-pages 1 nr_ents 32.
[57361.507225] blkfront: sdb: barriers enabled
[57361.507437] sdb: unknown partition table

Thierry Carrez (ttx)
Changed in nova:
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Ewan Mellor (ewanmellor) wrote :

Thanks Ant. I wasn't aware of this, but our Xen.org team was, as there was a thread about it on xen-devel. This is a daft thing for Ubuntu to have done, and it has been reverted:

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/684875
http://www.gossamer-threads.com/lists/xen/devel/192003

Do we need workaround code in the meantime? We could maybe autodetect the broken versions of Ubuntu, and fix up the problem. An alternative would be for you to add a udev rule that symlinked xvdX to sdX when it was hotplugged. That would work I think.

I don't like the idea of unconditionally assuming that it's sdX if xvdX isn't there -- we'll destroy someone's root filesystem that way.

Revision history for this message
Josh Kearney (jk0) wrote :

I agree that we need to go with the auto-detect approach. Is this something Citrix has a fix for (that we could work in ASAP) or do we still need to figure out the best way to implement this?

Revision history for this message
Thierry Carrez (ttx) wrote :

Marking FixCommitted since branch was merged, please reopen bug if there was more to it.

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
Salvatore Orlando (salvatore-orlando) wrote :

Antony & Rick,

thanks for sorting this out. I was completely unaware of this change in maverick.
Nevertheless, I still think udevsettle should be executed before trying to open the just-plugged device, in order to make sure the device is actually available and avoid errors due to race conditions.
I will open a new bug report for this.

Thierry Carrez (ttx)
Changed in nova:
milestone: none → 2011.1
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.