neutron

Network problem of the new launched instances

Asked by Salman

Hi Guys,

I am having trouble with launching instances. The launch fails on networking task with status error. Here is the output (at nova-compute screen session) on launching instance :

2012-05-02 06:41:52 TRACE nova.rpc.amqp RemoteError: Remote error: QuantumNotFoundException (u'Quantum entity not found: %s', '{"QuantumError": {"message": "Unable to find a network with the specified identifier.", "type": "NetworkNotFound", "detail": "Network 5f227bba-eb3b-4dba-ad4c-b47c80aaffd7 could not be found"}}')

And with this error no tap device is created for the device.

Dan was saying that running a fresh stack.sh script might resolve the error, but error is persistent even after running stack.sh again.

I have just a single node setup in a VM that has eth0 on NAT (ip= 10.0.3.15) and eth1 is on host-only-network (ip=10.0.0.10).
The way I launch instances: 1) either using dashboard Or 2) Using : euca-run-instances ami-00000001 -k mykey-$tenant0 -t m1.tiny
============================
my nova.conf:
-------------------------------------------------
[DEFAULT]
verbose=True
auth_strategy=keystone
allow_resize_to_same_host=True
root_helper=sudo /usr/local/bin/nova-rootwrap
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
dhcpbridge_flagfile=/etc/nova/nova.conf
fixed_range=10.0.0.0/24
s3_host=10.0.3.15
s3_port=3333
network_manager=nova.network.quantum.manager.QuantumManager
quantum_connection_host=localhost
quantum_connection_port=9696
osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
my_ip=10.0.3.15
public_interface=br-int
vlan_interface=eth0
flat_network_bridge=br-int
flat_interface=eth1
sql_connection=mysql://root:nova@localhost/nova?charset=utf8
libvirt_type=qemu
instance_name_template=instance-%08x
novncproxy_base_url=http://10.0.3.15:6080/vnc_auto.html
xvpvncproxy_base_url=http://10.0.3.15:6081/console
vncserver_listen=127.0.0.1
vncserver_proxyclient_address=127.0.0.1
api_paste_config=/etc/nova/api-paste.ini
image_service=nova.image.glance.GlanceImageService
ec2_dmz_host=10.0.3.15
rabbit_host=localhost
rabbit_password=nova
glance_api_servers=10.0.3.15:9292
force_dhcp_release=True
connection_type=libvirt
libvirt_ovs_bridge=br-int

#----------------From Ryu example config--------------------
use_deprecated-auth=true
firewall_driver=quantum.plugins.ryu.nova.firewall.NopFirewallDriver
libvirt_ovs_integration_bridge=br-int
libvirt_vif_type=ethernet
libvirt_vif_driver=quantum.plugins.ryu.nova.vif.LibvirtOpenVswitchOFPRyuDriver
libvirt_ovs_ryu_api_host=127.0.0.1:8080
linuxnet_interface_driver=quantum.plugins.ryu.nova.linux_net.LinuxOVSRyuInterfaceDriver
linuxnet_ovs_ryu_api_host=127.0.0.1:8080
quantum_use_dhcp=true

Any ideas ? speculations?
I have been stuck with this issue for some time now. Any help will be much appreciated.

Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
neutron Edit question
Assignee:
No assignee Edit question
Solved by:
Somik Behera
Solved:
Last query:
Last reply:
Revision history for this message
Somik Behera (somikbehera) said : 		#1

Hi Salman,

Open vSwitch and linux bridge are the more mature plugins that most folks use for testing and gating Quantum features. I would try and see if you can get the set-up working with Open vSwitch using Quantum's documentation in the Open vSwitch plugin README file.

If that works, I would go ahead and open a bug on Ryu plugin under bugs.

Thanks,
Somik

Revision history for this message
Salman (salmanmk) said : 		#2

Hi Somik,

Thanks for the quick reply. I had the instances running with OVS plugin and want to experiment with Ryu.
Do you have any advise on what might be wrong ?

And I have pinged the Ryu developers as well, and they are preparing a VM for easy testing of Ryu. But I am not sure how long it would take, so it would be great if I can figure out whats going wrong. I suspect a problem with my configuration is more probable than a bug in ryu.

Thoughts?

Revision history for this message
dan wendlandt (danwent) said : 		#3

If things work with the OVS plugin, and don't work with the Ryu plugin, its
probably something specific to the Ryu plugin.

On Wed, May 16, 2012 at 9:35 PM, Salman <
<email address hidden>> wrote:

> Question #197527 on quantum changed:
> https://answers.launchpad.net/quantum/+question/197527
>
> Status: Answered => Open
>
> Salman is still having a problem:
> Hi Somik,
>
> Thanks for the quick reply. I had the instances running with OVS plugin
> and want to experiment with Ryu.
> Do you have any advise on what might be wrong ?
>
> And I have pinged the Ryu developers as well, and they are preparing a
> VM for easy testing of Ryu. But I am not sure how long it would take, so
> it would be great if I can figure out whats going wrong. I suspect a
> problem with my configuration is more probable than a bug in ryu.
>
> Thoughts?
>
> --
> You received this question notification because you are an answer
> contact for quantum.
>

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Revision history for this message
Salman (salmanmk) said : 		#4

Hmm... little description to my problem above:

The network id that the quantum was complaining above was related to a public network (10.0.0./24 created by devstack by default). So new instances by default get an IP on all public networks (does nova attach VNICs to the instances in propprtion to public networks present? ), and since this network was not associated with any project, I believe thats why quantum didn't know about this network ?

So I worked around this by modifying that public network and assigned it to a temporary project, now the instance launches up but there is a new error that I see in instance's log (previously I was unable to access this log and the instance status would just show an error in the networking task):

ip: SIOCGIFFLAGS: No such device

and with that there is no tap interface in the wireshark either.

==========================
nova-manage network quantum_list :
----------------------------------------------
uuid project priority cidr_v4 , cidr_v6
09d21e3b-c1dd-48cc-a275-35835ce77ce3 temp_project None 10.0.0.0/24 , None
a65ec76b-c4ba-4f19-b03f-f9da3f2bb9cf ryu_tenant1 0 11.0.0.0/25 , None

===========================
ifconfig:
----------------------------------------
br-int Link encap:Ethernet HWaddr 08:00:27:16:d5:09
          inet6 addr: fe80::a00:27ff:fe16:d509/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:3865 errors:0 dropped:0 overruns:0 frame:0
          TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:963003 (963.0 KB) TX bytes:5616 (5.6 KB)

eth0 Link encap:Ethernet HWaddr 08:00:27:7a:ff:65
          inet addr:10.0.3.15 Bcast:10.0.3.255 Mask:255.255.255.0
          inet6 addr: fe80::a00:27ff:fe7a:ff65/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:44543 errors:0 dropped:0 overruns:0 frame:0
          TX packets:27121 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:48437762 (48.4 MB) TX bytes:3598443 (3.5 MB)

eth1 Link encap:Ethernet HWaddr 08:00:27:16:d5:09
          inet6 addr: fe80::a00:27ff:fe16:d509/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:11912 errors:0 dropped:0 overruns:0 frame:0
          TX packets:796 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2881210 (2.8 MB) TX bytes:103606 (103.6 KB)

gw-09d21e3b-c1 Link encap:Ethernet HWaddr fa:16:3e:09:e4:8b
          inet addr:11.0.0.1 Bcast:11.255.255.255 Mask:255.0.0.0 <=======Note: I set this IP manually (it was 10.0.0./24 earlier)
          inet6 addr: fe80::f816:3eff:fe09:e48b/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
          RX packets:552 errors:0 dropped:0 overruns:0 frame:0
          TX packets:238 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:135665 (135.6 KB) TX bytes:41489 (41.4 KB)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:197692 errors:0 dropped:0 overruns:0 frame:0
          TX packets:197692 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:159523785 (159.5 MB) TX bytes:159523785 (159.5 MB)

virbr0 Link encap:Ethernet HWaddr 6a:45:b9:a1:ae:ef
          inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
================================
ovs-ofctl show br-int
-----------------------------------------------------
OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000008002716d509
n_tables:1, n_buffers:256
features: capabilities:0x87, actions:0xfff
 1(gw-09d21e3b-c1): addr:fa:16:3e:09:e4:8b
     config: 0
     state: 0
 3(eth1): addr:08:00:27:16:d5:09
     config: 0
     state: 0
     current: 1GB-FD COPPER AUTO_NEG
     advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
     supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
 LOCAL(br-int): addr:08:00:27:16:d5:09
     config: 0 <================= Note: I did an ifconfig br-int up to change their status
     state: 0
OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0

Is there something wrong?

Revision history for this message
dan wendlandt (danwent) said : 		#5

See: http://docs.openstack.org/trunk/openstack-network/admin/content/Net-Create-dle455.html

Specifically:

"To create a network using Quantum Manager, run:

nova-manage network create --label=public --fixed_range_v4=8.8.8.0/24

When this command is invoked, the Quantum Manager will contact the Quantum Service to create a corresponding Quantum network, and likewise will create an IPAM subnet using the specified fixed range.

By default, networks are "global" in the sense that they are shared by all tenants (i.e. projects in Nova). To create a tenant-specific network, specify the "--project_id" argument when creating the network. For example:

nova-manage network create --label=tenant2-private --fixed_range_v4=10.0.0.0/24 --project_id=10daaef1-b481-42a6-8653-7a42e982e409

This project_id should be the tenant UUID from keystone. To view all keystone tenant UUIDs, run the following command as a keystone admin user:

keystone tenant-list

When a new VM is created, it is given a vNIC for each global network, as well as a vNIC for each network owned by the tenant associated with the VM. This lets users create hybrid scenarios where a VM has a vNIC both on a shared public network and on a private tenant-specific network."

Also, you mention:

"So I worked around this by modifying that public network and assigned it to a temporary project, "

I'm not sure whether you did this with nova-manage or via the database, but this won't work, b/c you are only changing the project in Nova, not in Quantum. You cannot change the association of a network and a project (or the lack of a project) after creation. I've created a bug to add this to the documentation: https://bugs.launchpad.net/quantum/+bug/1000837

As I mentioned, I can't really answer questions that are specific to the Ryu plugin.

Revision history for this message
Best Somik Behera (somikbehera) said : 		#6

Since Ryu Plugin isn't maintained by the core members of the Quantum team, I suggest that you send an email to mailing list about Ryu plugin specific issues.

Thanks,
Somik

Revision history for this message
Salman (salmanmk) said : 		#7

Thanks Somik Behera, that solved my question.