Network discovery failure after launching a test job

Asked by Harshavardhana

Error during network discovery.

snip from "nova-manage"

------------------------------------------------
migrate/versioning/repository.py:82
2011-04-21 16:25:53,077 DEBUG migrate.versioning.util [-] Disposing SQLAlchemy engine Engine(mysql://root:nova@openstack1/nova) from (pid=30657) with_engine /usr/lib/pymodules/python2.6/migrate/versioning/util/__init__.py:163
2011-04-21 16:25:53,967 DEBUG nova.utils [-] backend <module 'nova.db.sqlalchemy.api' from '/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/api.pyc'> from (pid=30703) __get_backend /usr/lib/pymodules/python2.6/nova/utils.py:427
2011-04-21 16:25:54,053 CRITICAL nova [-] Network with cidr 10.0.0.0/28 already exists
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/bin/nova-manage", line 1122, in <module>
(nova): TRACE: main()
(nova): TRACE: File "/usr/bin/nova-manage", line 1111, in main
(nova): TRACE: fn(*argv)
(nova): TRACE: File "/usr/bin/nova-manage", line 546, in create
(nova): TRACE: label=label)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 584, in create_networks
(nova): TRACE: cidr))
(nova): TRACE: ValueError: Network with cidr 10.0.0.0/28 already exists
(nova): TRACE:
---------------------------------------------

snip from nova-network

---------------------------------------------
2011-04-21 16:35:26,825 AUDIT nova [-] Starting network node (version 2011.2-workspace:tarmac-20110415024701-a9bdb77vaatk99lh)
2011-04-21 16:35:26,839 DEBUG nova.utils [-] Attempting to grab semaphore "iptables" for method "apply"... from (pid=20052) inner /usr/lib/pymodules/python2.6/nova/utils.py:594
2011-04-21 16:35:26,839 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "apply"... from (pid=20052) inner /usr/lib/pymodules/python2.6/nova/utils.py:599
2011-04-21 16:35:26,840 CRITICAL nova [-] failed to create /usr/lib/pymodules/python2.6/openstack1.MainThread-20052
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/bin/nova-network", line 48, in <module>
(nova): TRACE: service.serve()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/service.py", line 290, in serve
(nova): TRACE: x.start()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/service.py", line 84, in start
(nova): TRACE: self.manager.init_host()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 499, in init_host
(nova): TRACE: super(VlanManager, self).init_host()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 128, in init_host
(nova): TRACE: self.driver.init_host()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 378, in init_host
(nova): TRACE: iptables_manager.apply()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 606, in inner
(nova): TRACE: with lock:
(nova): TRACE: File "/usr/lib/pymodules/python2.6/lockfile.py", line 223, in __enter__
(nova): TRACE: self.acquire()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/lockfile.py", line 239, in acquire
(nova): TRACE: raise LockFailed("failed to create %s" % self.unique_name)
(nova): TRACE: LockFailed: failed to create /usr/lib/pymodules/python2.6/openstack1.MainThread-20052
(nova): TRACE:

Not sure if this is a bug or a configuration mistake. Please advise.

Using packages from
deb http://ppa.launchpad.net/nova-core/release/ubuntu maverick main
deb-src http://ppa.launchpad.net/nova-core/release/ubuntu maverick main

root@openstack1:~# uname -a
Linux openstack1 2.6.35-22-server #33-Ubuntu SMP Sun Sep 19 20:48:58 UTC 2010 x86_64 GNU/Linux
root@openstack1:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.10
DISTRIB_CODENAME=maverick
DISTRIB_DESCRIPTION="Ubuntu 10.10"

----------------------------------------------------------

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Vish Ishaya
Solved:
Last query:
Last reply:
Revision history for this message
Best Vish Ishaya (vishvananda) said :
#1

The first error looks like you tried to create the same set of networks twice.

the second one is due to a missing flag
--lock_path=/some/writable/directory (usually it is /var/lock/nova)

If you installed from packages this should be set in the default nova.conf Did you change it?
On Apr 21, 2011, at 4:47 PM, Harshavardhana wrote:

> New question #153795 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/153795
>
>
> Error during network discovery.
>
> snip from "nova-manage"
>
> ------------------------------------------------
> migrate/versioning/repository.py:82
> 2011-04-21 16:25:53,077 DEBUG migrate.versioning.util [-] Disposing SQLAlchemy engine Engine(mysql://root:nova@openstack1/nova) from (pid=30657) with_engine /usr/lib/pymodules/python2.6/migrate/versioning/util/__init__.py:163
> 2011-04-21 16:25:53,967 DEBUG nova.utils [-] backend <module 'nova.db.sqlalchemy.api' from '/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/api.pyc'> from (pid=30703) __get_backend /usr/lib/pymodules/python2.6/nova/utils.py:427
> 2011-04-21 16:25:54,053 CRITICAL nova [-] Network with cidr 10.0.0.0/28 already exists
> (nova): TRACE: Traceback (most recent call last):
> (nova): TRACE: File "/usr/bin/nova-manage", line 1122, in <module>
> (nova): TRACE: main()
> (nova): TRACE: File "/usr/bin/nova-manage", line 1111, in main
> (nova): TRACE: fn(*argv)
> (nova): TRACE: File "/usr/bin/nova-manage", line 546, in create
> (nova): TRACE: label=label)
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 584, in create_networks
> (nova): TRACE: cidr))
> (nova): TRACE: ValueError: Network with cidr 10.0.0.0/28 already exists
> (nova): TRACE:
> ---------------------------------------------
>
> snip from nova-network
>
> ---------------------------------------------
> 2011-04-21 16:35:26,825 AUDIT nova [-] Starting network node (version 2011.2-workspace:tarmac-20110415024701-a9bdb77vaatk99lh)
> 2011-04-21 16:35:26,839 DEBUG nova.utils [-] Attempting to grab semaphore "iptables" for method "apply"... from (pid=20052) inner /usr/lib/pymodules/python2.6/nova/utils.py:594
> 2011-04-21 16:35:26,839 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "apply"... from (pid=20052) inner /usr/lib/pymodules/python2.6/nova/utils.py:599
> 2011-04-21 16:35:26,840 CRITICAL nova [-] failed to create /usr/lib/pymodules/python2.6/openstack1.MainThread-20052
> (nova): TRACE: Traceback (most recent call last):
> (nova): TRACE: File "/usr/bin/nova-network", line 48, in <module>
> (nova): TRACE: service.serve()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/service.py", line 290, in serve
> (nova): TRACE: x.start()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/service.py", line 84, in start
> (nova): TRACE: self.manager.init_host()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 499, in init_host
> (nova): TRACE: super(VlanManager, self).init_host()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 128, in init_host
> (nova): TRACE: self.driver.init_host()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 378, in init_host
> (nova): TRACE: iptables_manager.apply()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 606, in inner
> (nova): TRACE: with lock:
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/lockfile.py", line 223, in __enter__
> (nova): TRACE: self.acquire()
> (nova): TRACE: File "/usr/lib/pymodules/python2.6/lockfile.py", line 239, in acquire
> (nova): TRACE: raise LockFailed("failed to create %s" % self.unique_name)
> (nova): TRACE: LockFailed: failed to create /usr/lib/pymodules/python2.6/openstack1.MainThread-20052
> (nova): TRACE:
>
>
> Not sure if this is a bug or a configuration mistake. Please advise.
>
>
> Using packages from
> deb http://ppa.launchpad.net/nova-core/release/ubuntu maverick main
> deb-src http://ppa.launchpad.net/nova-core/release/ubuntu maverick main
>
> root@openstack1:~# uname -a
> Linux openstack1 2.6.35-22-server #33-Ubuntu SMP Sun Sep 19 20:48:58 UTC 2010 x86_64 GNU/Linux
> root@openstack1:~# cat /etc/lsb-release
> DISTRIB_ID=Ubuntu
> DISTRIB_RELEASE=10.10
> DISTRIB_CODENAME=maverick
> DISTRIB_DESCRIPTION="Ubuntu 10.10"
>
>
> ----------------------------------------------------------
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
Harshavardhana (harsha-gluster) said :
#2

Thanks Vish, No i haven't changed nova.conf. I will look for the --lock_path and report back if it worked.

Revision history for this message
Harshavardhana (harsha-gluster) said :
#3

Checked just now, seems like the directory exists but still the nova-network thread is failing. is it permissions?

root@openstack1:/var/lock/nova# cd ..
root@openstack1:/var/lock# ls
lvm nova
root@openstack1:/var/lock# stat nova
  File: `nova'
  Size: 40 Blocks: 0 IO Block: 4096 directory
Device: 12h/18d Inode: 59146 Links: 2
Access: (0755/drwxr-xr-x) Uid: ( 106/ nova) Gid: ( 0/ root)
Access: 2011-04-22 13:13:47.025973002 -0700
Modify: 2011-04-21 15:53:34.969018005 -0700
Change: 2011-04-22 13:13:55.935973002 -0700
root@openstack1:/var/lock# ls -ltr
total 0
drwxr-xr-x 2 nova root 40 2011-04-21 15:53 nova
drwx------ 2 root root 40 2011-04-21 16:40 lvm
root@openstack1:/var/lock#

Revision history for this message
Harshavardhana (harsha-gluster) said :
#4

and my nova.conf

--sql_connection=mysql://root:nova@openstack1/nova
--s3_host=openstack2
--rabbit_host=172.17.30.151
--cc_host=172.17.30.151
--ec2_url=http://172.17.30.151:8773/services/Cloud

--daemonize=1

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge

--FAKE_subdomain=ec2

--ca_path=/var/lib/nova/CA
--keys_path=/var/lib/nova/keys
--networks_path=/var/lib/nova/networks
--instances_path=/var/lib/nova/instances
--images_path=/var/lib/nova/images
--buckets_path=/var/lib/nova/buckets

--libvirt_type=kvm

--network_manager=nova.network.manager.VlanManager

--vlan_interface=eth0

--logdir=/var/log/nova
--verbose
--volume_group=openstack1
--fixed_range=172.17.30.0/24
--network_size=64

Revision history for this message
Vish Ishaya (vishvananda) said :
#5

I don't see --lock_path in there

Vish

On Apr 22, 2011, at 1:16 PM, Harshavardhana wrote:

> Question #153795 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153795
>
> Harshavardhana posted a new comment:
> and my nova.conf
>
>
> --sql_connection=mysql://root:nova@openstack1/nova
> --s3_host=openstack2
> --rabbit_host=172.17.30.151
> --cc_host=172.17.30.151
> --ec2_url=http://172.17.30.151:8773/services/Cloud
>
> --daemonize=1
>
> --dhcpbridge_flagfile=/etc/nova/nova.conf
> --dhcpbridge=/usr/bin/nova-dhcpbridge
>
> --FAKE_subdomain=ec2
>
> --ca_path=/var/lib/nova/CA
> --keys_path=/var/lib/nova/keys
> --networks_path=/var/lib/nova/networks
> --instances_path=/var/lib/nova/instances
> --images_path=/var/lib/nova/images
> --buckets_path=/var/lib/nova/buckets
>
> --libvirt_type=kvm
>
> --network_manager=nova.network.manager.VlanManager
>
> --vlan_interface=eth0
>
> --logdir=/var/log/nova
> --verbose
> --volume_group=openstack1
> --fixed_range=172.17.30.0/24
> --network_size=64
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
Harshavardhana (harsha-gluster) said :
#6

After adding --lock_path the error seems to have gone away, but i didn't understand since i didn't add anything manually. I was just using NovaDeploymentTool it never was added in nova.conf file.

Anyways after that a simple launch wouldn't still work

root@openstack2:~# euca-describe-instances
RESERVATION r-l5rhsgfr IRT default
INSTANCE i-0000000a ami-51539ae1 networking mykey (IRT, openstack4) 0 m1.tiny 2011-04-22T22:39:01Z nova
root@openstack2:~#

Relevant log section from openstack4 , this would just hang.

011-04-22 15:37:56,911 DEBUG nova.rpc [-] Initing the Adapter Consumer for compute from (pid=1632) __init__ /usr/lib/pymodules/python2.6/nova/rpc.py:148
2011-04-22 15:39:01,244 DEBUG nova.rpc [-] received {'_context_request_id': 'T7P5R9SGSEPWHNA7O6Z7', '_context_read_deleted': False, 'args': {'instance_id': 10, 'injected_files': None, 'availability_zone': None}, '_context_is_admin': True, '_context_timestamp': '2011-04-22T22:39:00Z', '_context_user': 'anne', 'method': 'run_instance', '_context_project': 'IRT', '_context_remote_address': '172.17.30.151'} from (pid=1632) _receive /usr/lib/pymodules/python2.6/nova/rpc.py:167
2011-04-22 15:39:01,245 DEBUG nova.rpc [-] unpacked context: {'timestamp': '2011-04-22T22:39:00Z', 'remote_address': '172.17.30.151', 'project': 'IRT', 'is_admin': True, 'user': 'anne', 'request_id': 'T7P5R9SGSEPWHNA7O6Z7', 'read_deleted': False} from (pid=1632) _unpack_context /usr/lib/pymodules/python2.6/nova/rpc.py:331
2011-04-22 15:39:01,316 AUDIT nova.compute.manager [T7P5R9SGSEPWHNA7O6Z7 anne IRT] instance 10: starting...
2011-04-22 15:39:01,471 DEBUG nova.rpc [-] Making asynchronous call on network.openstack1 ... from (pid=1632) call /usr/lib/pymodules/python2.6/nova/rpc.py:350
2011-04-22 15:39:01,471 DEBUG nova.rpc [-] MSG_ID is a211c46a89f44f77888e775b410e1671 from (pid=1632) call /usr/lib/pymodules/python2.6/nova/rpc.py:353

Do i need to open a new question for this resolving current question?

Thanks Vish for all the help.

Revision history for this message
Harshavardhana (harsha-gluster) said :
#7

Problem could be that NovaDeploymentTool has a template of nova.conf as nova.conf.erb which doesn't have --lock_path

See below, i guess it needs to be added :-

--sql_connection=mysql://root:nova@<%= mysql %>/nova
--s3_host=<%= nova_objectstore %>
--rabbit_host=<%= nova_api %>
--cc_host=<%= nova_api %>
--ec2_url=http://<%= nova_api %>:8773/services/Cloud

--daemonize=1

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge

--FAKE_subdomain=ec2

--ca_path=/var/lib/nova/CA
--keys_path=/var/lib/nova/keys
--networks_path=/var/lib/nova/networks
--instances_path=/var/lib/nova/instances
--images_path=/var/lib/nova/images
--buckets_path=/var/lib/nova/buckets

--libvirt_type=<%= libvirt_type %>

--network_manager=<%= network_manager %>

--vlan_interface=<%= network_interface %>

--logdir=/var/log/nova
--verbose

Revision history for this message
Harshavardhana (harsha-gluster) said :
#8

Thanks Vish Ishaya, that solved my question.

Revision history for this message
Vish Ishaya (vishvananda) said :
#9

Yes i don't think that nova deployment tool has been updated for cactus yet. Looks like it is making a request to network that isn't coming back. I would check the nova-network log

On Apr 22, 2011, at 3:43 PM, Harshavardhana wrote:

> Question #153795 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153795
>
> Harshavardhana posted a new comment:
> After adding --lock_path the error seems to have gone away, but i didn't
> understand since i didn't add anything manually. I was just using
> NovaDeploymentTool it never was added in nova.conf file.
>
> Anyways after that a simple launch wouldn't still work
>
>
> root@openstack2:~# euca-describe-instances
> RESERVATION r-l5rhsgfr IRT default
> INSTANCE i-0000000a ami-51539ae1 networking mykey (IRT, openstack4) 0 m1.tiny 2011-04-22T22:39:01Z nova
> root@openstack2:~#
>
> Relevant log section from openstack4 , this would just hang.
>
> 011-04-22 15:37:56,911 DEBUG nova.rpc [-] Initing the Adapter Consumer for compute from (pid=1632) __init__ /usr/lib/pymodules/python2.6/nova/rpc.py:148
> 2011-04-22 15:39:01,244 DEBUG nova.rpc [-] received {'_context_request_id': 'T7P5R9SGSEPWHNA7O6Z7', '_context_read_deleted': False, 'args': {'instance_id': 10, 'injected_files': None, 'availability_zone': None}, '_context_is_admin': True, '_context_timestamp': '2011-04-22T22:39:00Z', '_context_user': 'anne', 'method': 'run_instance', '_context_project': 'IRT', '_context_remote_address': '172.17.30.151'} from (pid=1632) _receive /usr/lib/pymodules/python2.6/nova/rpc.py:167
> 2011-04-22 15:39:01,245 DEBUG nova.rpc [-] unpacked context: {'timestamp': '2011-04-22T22:39:00Z', 'remote_address': '172.17.30.151', 'project': 'IRT', 'is_admin': True, 'user': 'anne', 'request_id': 'T7P5R9SGSEPWHNA7O6Z7', 'read_deleted': False} from (pid=1632) _unpack_context /usr/lib/pymodules/python2.6/nova/rpc.py:331
> 2011-04-22 15:39:01,316 AUDIT nova.compute.manager [T7P5R9SGSEPWHNA7O6Z7 anne IRT] instance 10: starting...
> 2011-04-22 15:39:01,471 DEBUG nova.rpc [-] Making asynchronous call on network.openstack1 ... from (pid=1632) call /usr/lib/pymodules/python2.6/nova/rpc.py:350
> 2011-04-22 15:39:01,471 DEBUG nova.rpc [-] MSG_ID is a211c46a89f44f77888e775b410e1671 from (pid=1632) call /usr/lib/pymodules/python2.6/nova/rpc.py:353
>
>
> Do i need to open a new question for this resolving current question?
>
> Thanks Vish for all the help.
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
Harshavardhana (harsha-gluster) said :
#10

Nova network log says :-

Command: sudo -E dnsmasq --strict-order --bind-interfaces --conf-file= --domain=novalocal --pid-file=/var/lib/nova/networks/nova-br100.pid --listen-address=10.0.0.1 --except-interface=lo --dhcp-range=10.0.0.3,static,120s --dhcp-hostsfile=/var/lib/nova/networks/nova-br100.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
Exit code: 2
Stdout: ''
Stderr: '\ndnsmasq: failed to bind listening socket for 10.0.0.1: Address already in use\n'
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/bin/nova-network", line 48, in <module>
(nova): TRACE: service.serve()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/service.py", line 290, in serve
(nova): TRACE: x.start()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/service.py", line 84, in start
(nova): TRACE: self.manager.init_host()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 499, in init_host
(nova): TRACE: super(VlanManager, self).init_host()
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 134, in init_host
(nova): TRACE: self._on_set_network_host(ctxt, network['id'])
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 629, in _on_set_network_host
(nova): TRACE: self.driver.update_dhcp(context, network_id)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 607, in inner
(nova): TRACE: retval = f(*args, **kwargs)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 588, in update_dhcp
(nova): TRACE: _execute(*command, addl_env=env)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 666, in _execute
(nova): TRACE: return utils.execute(*cmd, **kwargs)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 173, in execute
(nova): TRACE: cmd=' '.join(cmd))
(nova): TRACE: ProcessExecutionError: Unexpected error while running command.
(nova): TRACE: Command: sudo -E dnsmasq --strict-order --bind-interfaces --conf-file= --domain=novalocal --pid-file=/var/lib/nova/networks/nova-br100.pid --listen-address=10.0.0.1 --except-interface=lo --dhcp-range=10.0.0.3,static,120s --dhcp-hostsfile=/var/lib/nova/networks/nova-br100.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
(nova): TRACE: Exit code: 2
(nova): TRACE: Stdout: ''
(nova): TRACE: Stderr: '\ndnsmasq: failed to bind listening socket for 10.0.0.1: Address already in use\n'
(nova): TRACE:

Revision history for this message
Harshavardhana (harsha-gluster) said :
#11

Alright looks like init started another instance of dnsmasq, stopped it so now the ip is assigned.

but still the instance seems to stay in "networking"

root@openstack2:~# euca-describe-instances
RESERVATION r-l25vihnf IRT default
INSTANCE i-0000000b ami-51539ae1 10.0.0.3 10.0.0.3 networking mykey (IRT, openstack3) 0 m1.tiny 2011-04-25T19:04:26Z nova
root@openstack2:~#

is there any logs that i should look out for?

Revision history for this message
Harshavardhana (harsha-gluster) said :
#12

ok now the openstack3 has a nova-compute log

2011-04-25 12:04:12,178 DEBUG nova.linux_net [-] Starting VLAN inteface vlan100 from (pid=1321) ensure_vlan /usr/lib/pymodules/
python2.6/nova/network/linux_net.py:444
2011-04-25 12:04:12,178 DEBUG nova.utils [-] Running cmd (subprocess): sudo vconfig set_name_type VLAN_PLUS_VID_NO_PAD from (pi
d=1321) execute /usr/lib/pymodules/python2.6/nova/utils.py:150
2011-04-25 12:04:12,240 DEBUG nova.utils [-] Result was 1 from (pid=1321) execute /usr/lib/pymodules/python2.6/nova/utils.py:16
6
2011-04-25 12:04:12,241 ERROR nova.exception [-] Uncaught exception
(nova.exception): TRACE: Traceback (most recent call last):
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 120, in _wrap
(nova.exception): TRACE: return f(*args, **kw)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 225, in run_instance
(nova.exception): TRACE: instance_id)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/manager.py", line 532, in setup_compute_network
(nova.exception): TRACE: network_ref['bridge'])
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 436, in ensure_vlan_bridge
(nova.exception): TRACE: interface = ensure_vlan(vlan_num)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 445, in ensure_vlan
(nova.exception): TRACE: _execute('sudo', 'vconfig', 'set_name_type', 'VLAN_PLUS_VID_NO_PAD')
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/network/linux_net.py", line 666, in _execute
(nova.exception): TRACE: return utils.execute(*cmd, **kwargs)
(nova.exception): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 173, in execute
(nova.exception): TRACE: cmd=' '.join(cmd))
(nova.exception): TRACE: ProcessExecutionError: Unexpected error while running command.
(nova.exception): TRACE: Command: sudo vconfig set_name_type VLAN_PLUS_VID_NO_PAD
(nova.exception): TRACE: Exit code: 1
(nova.exception): TRACE: Stdout: ''
(nova.exception): TRACE: Stderr: 'sudo: vconfig: command not found\n'
(nova.exception): TRACE:
2011-04-25 12:04:12,263 ERROR nova [-] Exception during message handling
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/rpc.py", line 188, in _receive
(nova): TRACE: rval = node_func(context=ctxt, **node_args)
(nova): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 126, in _wrap
(nova): TRACE: raise Error(str(e))
(nova): TRACE: Error: Unexpected error while running command.
(nova): TRACE: Command: sudo vconfig set_name_type VLAN_PLUS_VID_NO_PAD
(nova): TRACE: Exit code: 1
(nova): TRACE: Stdout: ''
(nova): TRACE: Stderr: 'sudo: vconfig: command not found\n'
(nova): TRACE:
2011-04-25 12:04:36,653 INFO nova.compute.manager [-] Found instance 'instance-0000000b' in DB but no VM. State=0, so assuming spawn is in progress.
2011-04-25 12:05:36,693 INFO nova.compute.manager [-] Found instance 'instance-0000000b' in DB but no VM. State=0, so assuming spawn is in progress.
2011-04-25 12:06:36,733 INFO nova.compute.manager [-] Found instance 'instance-0000000b' in DB but no VM. State=0, so assuming spawn is in progress.

Revision history for this message
Harshavardhana (harsha-gluster) said :
#13

ok now i have installed vlan package.

ib/pymodules/python2.6/nova/utils.py:150
2011-04-25 12:10:18,297 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=1310) execute /usr/lib/pymodules/python2.6/nova/utils.py:150
2011-04-25 12:10:18,334 DEBUG nova.utils [-] Running cmd (subprocess): mkdir -p /var/lib/nova/instances/instance-0000000c/ from (pid=1310) execute /usr/lib/pymodules/python2.6/nova/utils.py:150
2011-04-25 12:10:18,340 INFO nova.virt.libvirt_conn [-] instance instance-0000000c: Creating image
2011-04-25 12:10:18,465 DEBUG nova.utils [-] Attempting to grab semaphore "592fa047" for method "call_if_not_exists"... from (pid=1310) inner /usr/lib/pymodules/python2.6/nova/utils.py:594
2011-04-25 12:10:18,515 ERROR nova.compute.manager [98IQV0BGREU0JHQ272IK anne IRT] Instance '12' failed to spawn. Is virtualization enabled in the BIOS?
(nova.compute.manager): TRACE: Traceback (most recent call last):
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 234, in run_instance
(nova.compute.manager): TRACE: self.driver.spawn(instance_ref)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 120, in _wrap
(nova.compute.manager): TRACE: return f(*args, **kw)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 616, in spawn
(nova.compute.manager): TRACE: self._create_image(instance, xml, network_info)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 841, in _create_image
(nova.compute.manager): TRACE: project=project)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 776, in _cache_image
(nova.compute.manager): TRACE: call_if_not_exists(base, fn, *args, **kwargs)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 607, in inner
(nova.compute.manager): TRACE: retval = f(*args, **kwargs)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 774, in call_if_not_exists
(nova.compute.manager): TRACE: fn(target=base, *args, **kwargs)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 787, in _fetch_image
(nova.compute.manager): TRACE: images.fetch(image_id, target, user, project)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/images.py", line 51, in fetch
(nova.compute.manager): TRACE: metadata = image_service.get(elevated, image_id, image_file)
(nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/image/local.py", line 116, in get
(nova.compute.manager): TRACE: raise exception.NotFound
(nova.compute.manager): TRACE: NotFound: None
(nova.compute.manager): TRACE:
2011-04-25 12:10:51,678 INFO nova.compute.manager [-] Found instance 'instance-0000000c' in DB but no VM. State=8, so setting state to shutoff.
2011-04-25 12:10:51,679 INFO nova.compute.manager [-] DB/VM state mismatch. Changing state from '8' to '5'
(END)

Seems like it doesn't recognize the BIOS kvm

root@openstack4:~# ls -ltr /dev/kvm
crw-rw---- 1 root kvm 10, 232 2011-04-23 15:16 /dev/kvm
root@openstack4:~#

root@openstack4:~# kvm
pci_add_option_rom: failed to find romfile "pxe-rtl8139.bin"
Could not initialize SDL - exiting

Revision history for this message
Vish Ishaya (vishvananda) said :
#14

It looks like you are trying to use the local image service on multiple
machines. You need to use glance or share the images dir for multiple
machines.
On Apr 25, 2011 12:13 PM, "Harshavardhana" <
<email address hidden>> wrote:
> Question #153795 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153795
>
> Harshavardhana posted a new comment:
> ok now i have installed vlan package.
>
> ib/pymodules/python2.6/nova/utils.py:150
> 2011-04-25 12:10:18,297 DEBUG nova.utils [-] Running cmd (subprocess):
sudo iptables-restore from (pid=1310) execute
/usr/lib/pymodules/python2.6/nova/utils.py:150
> 2011-04-25 12:10:18,334 DEBUG nova.utils [-] Running cmd (subprocess):
mkdir -p /var/lib/nova/instances/instance-0000000c/ from (pid=1310) execute
/usr/lib/pymodules/python2.6/nova/utils.py:150
> 2011-04-25 12:10:18,340 INFO nova.virt.libvirt_conn [-] instance
instance-0000000c: Creating image
> 2011-04-25 12:10:18,465 DEBUG nova.utils [-] Attempting to grab semaphore
"592fa047" for method "call_if_not_exists"... from (pid=1310) inner
/usr/lib/pymodules/python2.6/nova/utils.py:594
> 2011-04-25 12:10:18,515 ERROR nova.compute.manager [98IQV0BGREU0JHQ272IK
anne IRT] Instance '12' failed to spawn. Is virtualization enabled in the
BIOS?
> (nova.compute.manager): TRACE: Traceback (most recent call last):
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 234, in
run_instance
> (nova.compute.manager): TRACE: self.driver.spawn(instance_ref)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/exception.py", line 120, in _wrap
> (nova.compute.manager): TRACE: return f(*args, **kw)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 616, in spawn
> (nova.compute.manager): TRACE: self._create_image(instance, xml,
network_info)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 841, in
_create_image
> (nova.compute.manager): TRACE: project=project)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 776, in
_cache_image
> (nova.compute.manager): TRACE: call_if_not_exists(base, fn, *args,
**kwargs)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/utils.py", line 607, in inner
> (nova.compute.manager): TRACE: retval = f(*args, **kwargs)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 774, in
call_if_not_exists
> (nova.compute.manager): TRACE: fn(target=base, *args, **kwargs)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 787, in
_fetch_image
> (nova.compute.manager): TRACE: images.fetch(image_id, target, user,
project)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/virt/images.py", line 51, in fetch
> (nova.compute.manager): TRACE: metadata = image_service.get(elevated,
image_id, image_file)
> (nova.compute.manager): TRACE: File
"/usr/lib/pymodules/python2.6/nova/image/local.py", line 116, in get
> (nova.compute.manager): TRACE: raise exception.NotFound
> (nova.compute.manager): TRACE: NotFound: None
> (nova.compute.manager): TRACE:
> 2011-04-25 12:10:51,678 INFO nova.compute.manager [-] Found instance
'instance-0000000c' in DB but no VM. State=8, so setting state to shutoff.
> 2011-04-25 12:10:51,679 INFO nova.compute.manager [-] DB/VM state
mismatch. Changing state from '8' to '5'
> (END)
>
> Seems like it doesn't recognize the BIOS kvm
>
> root@openstack4:~# ls -ltr /dev/kvm
> crw-rw---- 1 root kvm 10, 232 2011-04-23 15:16 /dev/kvm
> root@openstack4:~#
>
> root@openstack4:~# kvm
> pci_add_option_rom: failed to find romfile "pxe-rtl8139.bin"
> Could not initialize SDL - exiting
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).