Multi node: Can't ping and ssh to vm in Vlan mode

Asked by KiRaDo

I have two nodes, one (ubuntu2) running api, network and compute, volume and one (ubuntu1) just a compute node. I using vlan network mode

$ sudo nova-manage service list
ubuntu2 nova-network enabled :-) 2011-04-07 03:44:39
ubuntu2 nova-volume enabled :-) 2011-04-07 03:44:40
ubuntu2 nova-compute disabled :-) 2011-04-07 03:44:42
ubuntu2 nova-scheduler enabled :-) 2011-04-07 03:44:39
ubuntu1 nova-compute enabled :-) 2011-04-07 03:44:37

when i launch instances:

RESERVATION r-c0bvqxjx project1 default
INSTANCE i-00000013 ami-7beeeafe 10.0.1.15 10.0.1.15 running key01 (project1, ubuntu2) 0 m1.tiny 2011-04-07T03:53:03Z nova

RESERVATION r-n4wehu4g project1 default
INSTANCE i-00000011 ami-7beeeafe 10.0.1.14 10.0.1.14 running key01 (project1, ubuntu1) 0 m1.tiny 2011-04-07T03:50:39Z nova

Problem is, i can ping and ssh to 10.0.1.15 but can't ping and ssh to 10.0.1.14

-------------------------------------------------------------------------------------------------------------------------------
nova.conf

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--s3_host=10.2.14.205
--rabbit_host=10.2.14.205
--cc_host=10.2.14.205
--ec2_url=http://10.2.14.205:8773/services/Cloud
--fixed_range=10.0.0.0/8
--network_size=256
--FAKE_subdomain=ec2
--routing_source_ip=10.2.14.205
--sql_connection=mysql://root:nova@10.2.14.205/nova
-------------------------------------

+ on ubuntu2:

interface
-------------------------------------

br100 Link encap:Ethernet HWaddr 60:eb:69:6a:46:fd
          inet6 addr: fe80::9cf4:1dff:fef3:c7a0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:139 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:29752 (29.7 KB) TX bytes:468 (468.0 B)

eth0 Link encap:Ethernet HWaddr 60:eb:69:6a:46:fd
          inet addr:10.2.14.203 Bcast:10.2.14.255 Mask:255.255.255.0
          inet6 addr: fe80::62eb:69ff:fe6a:46fd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:556182 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1045803 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:59938722 (59.9 MB) TX bytes:102892260 (102.8 MB)
          Interrupt:16

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:292 errors:0 dropped:0 overruns:0 frame:0
          TX packets:292 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:28059 (28.0 KB) TX bytes:28059 (28.0 KB)

virbr0 Link encap:Ethernet HWaddr a2:69:7a:19:b1:ba
          inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:2964 (2.9 KB)

vlan100 Link encap:Ethernet HWaddr 60:eb:69:6a:46:fd
          inet6 addr: fe80::62eb:69ff:fe6a:46fd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:150 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:32544 (32.5 KB)

vnet0 Link encap:Ethernet HWaddr fe:16:3e:09:bc:eb
          inet6 addr: fe80::fc16:3eff:fe09:bceb/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:25 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:6966 (6.9 KB) TX bytes:468 (468.0 B)
---------------------------------

brclt show

bridge name bridge id STP enabled interfaces
br100 8000.60eb696a46fd no vlan100
                                                        vnet0
virbr0 8000.000000000000 yes
---------------------------------

+ On ubuntu2

interface
---------------------------

br100 Link encap:Ethernet HWaddr 00:1d:60:e8:75:38
          inet addr:10.0.1.1 Bcast:10.0.1.255 Mask:255.255.255.0
          inet6 addr: fe80::9006:7dff:fe4b:846b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:237 errors:0 dropped:0 overruns:0 frame:0
          TX packets:241 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:22774 (22.7 KB) TX bytes:36985 (36.9 KB)

eth0 Link encap:Ethernet HWaddr 00:1d:60:e8:75:38
          inet addr:10.2.14.205 Bcast:10.2.14.255 Mask:255.255.255.0
          inet6 addr: fe80::21d:60ff:fee8:7538/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:1133227 errors:0 dropped:0 overruns:0 frame:0
          TX packets:555018 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:110538175 (110.5 MB) TX bytes:73426894 (73.4 MB)
          Interrupt:18

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING MTU:16436 Metric:1
          RX packets:3192648 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3192648 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:262066647 (262.0 MB) TX bytes:262066647 (262.0 MB)

virbr0 Link encap:Ethernet HWaddr 8a:98:8b:82:4a:ec
          inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:2977 (2.9 KB)

vlan100 Link encap:Ethernet HWaddr 00:1d:60:e8:75:38
          inet6 addr: fe80::21d:60ff:fee8:7538/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:6196 (6.1 KB)

vnet0 Link encap:Ethernet HWaddr fe:16:3e:10:3a:18
          inet6 addr: fe80::fc16:3eff:fe10:3a18/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:237 errors:0 dropped:0 overruns:0 frame:0
          TX packets:218 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:26092 (26.0 KB) TX bytes:32775 (32.7 KB)
--------------------------
brctl show

bridge name bridge id STP enabled interfaces
br100 8000.001d60e87538 no vlan100
                                                        vnet0
virbr0 8000.000000000000 yes
-----------------------

Please give me some advices.
Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Vish Ishaya
Solved:
Last query:
Last reply:
Revision history for this message
Vish Ishaya (vishvananda) said :
#1

You have no received packets on vlan100 which means the vlan isn't enabled on your switch. To use vlan mode, you need to enable vlan trunking on the vlans that are used by nova. In the default case this will be vlan100 for the first project, and will start counting up by one for each additional project.

On Apr 6, 2011, at 9:19 PM, KiRaDo wrote:

> New question #151941 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/151941
>
> I have two nodes, one (ubuntu2) running api, network and compute, volume and one (ubuntu1) just a compute node. I using vlan network mode
>
> $ sudo nova-manage service list
> ubuntu2 nova-network enabled :-) 2011-04-07 03:44:39
> ubuntu2 nova-volume enabled :-) 2011-04-07 03:44:40
> ubuntu2 nova-compute disabled :-) 2011-04-07 03:44:42
> ubuntu2 nova-scheduler enabled :-) 2011-04-07 03:44:39
> ubuntu1 nova-compute enabled :-) 2011-04-07 03:44:37
>
> when i launch instances:
>
> RESERVATION r-c0bvqxjx project1 default
> INSTANCE i-00000013 ami-7beeeafe 10.0.1.15 10.0.1.15 running key01 (project1, ubuntu2) 0 m1.tiny 2011-04-07T03:53:03Z nova
>
> RESERVATION r-n4wehu4g project1 default
> INSTANCE i-00000011 ami-7beeeafe 10.0.1.14 10.0.1.14 running key01 (project1, ubuntu1) 0 m1.tiny 2011-04-07T03:50:39Z nova
>
>
> Problem is, i can ping and ssh to 10.0.1.15 but can't ping and ssh to 10.0.1.14
>
> -------------------------------------------------------------------------------------------------------------------------------
> nova.conf
>
> --dhcpbridge_flagfile=/etc/nova/nova.conf
> --dhcpbridge=/usr/bin/nova-dhcpbridge
> --logdir=/var/log/nova
> --state_path=/var/lib/nova
> --lock_path=/var/lock/nova
> --verbose
> --s3_host=10.2.14.205
> --rabbit_host=10.2.14.205
> --cc_host=10.2.14.205
> --ec2_url=http://10.2.14.205:8773/services/Cloud
> --fixed_range=10.0.0.0/8
> --network_size=256
> --FAKE_subdomain=ec2
> --routing_source_ip=10.2.14.205
> --sql_connection=mysql://root:nova@10.2.14.205/nova
> -------------------------------------
>
> + on ubuntu2:
>
> interface
> -------------------------------------
>
> br100 Link encap:Ethernet HWaddr 60:eb:69:6a:46:fd
> inet6 addr: fe80::9cf4:1dff:fef3:c7a0/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:139 errors:0 dropped:0 overruns:0 frame:0
> TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:29752 (29.7 KB) TX bytes:468 (468.0 B)
>
> eth0 Link encap:Ethernet HWaddr 60:eb:69:6a:46:fd
> inet addr:10.2.14.203 Bcast:10.2.14.255 Mask:255.255.255.0
> inet6 addr: fe80::62eb:69ff:fe6a:46fd/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:556182 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1045803 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:59938722 (59.9 MB) TX bytes:102892260 (102.8 MB)
> Interrupt:16
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:292 errors:0 dropped:0 overruns:0 frame:0
> TX packets:292 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:28059 (28.0 KB) TX bytes:28059 (28.0 KB)
>
> virbr0 Link encap:Ethernet HWaddr a2:69:7a:19:b1:ba
> inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:2964 (2.9 KB)
>
> vlan100 Link encap:Ethernet HWaddr 60:eb:69:6a:46:fd
> inet6 addr: fe80::62eb:69ff:fe6a:46fd/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:150 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:32544 (32.5 KB)
>
> vnet0 Link encap:Ethernet HWaddr fe:16:3e:09:bc:eb
> inet6 addr: fe80::fc16:3eff:fe09:bceb/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:25 errors:0 dropped:0 overruns:0 frame:0
> TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:500
> RX bytes:6966 (6.9 KB) TX bytes:468 (468.0 B)
> ---------------------------------
>
> brclt show
>
> bridge name bridge id STP enabled interfaces
> br100 8000.60eb696a46fd no vlan100
> vnet0
> virbr0 8000.000000000000 yes
> ---------------------------------
>
> + On ubuntu2
>
> interface
> ---------------------------
>
> br100 Link encap:Ethernet HWaddr 00:1d:60:e8:75:38
> inet addr:10.0.1.1 Bcast:10.0.1.255 Mask:255.255.255.0
> inet6 addr: fe80::9006:7dff:fe4b:846b/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:237 errors:0 dropped:0 overruns:0 frame:0
> TX packets:241 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:22774 (22.7 KB) TX bytes:36985 (36.9 KB)
>
> eth0 Link encap:Ethernet HWaddr 00:1d:60:e8:75:38
> inet addr:10.2.14.205 Bcast:10.2.14.255 Mask:255.255.255.0
> inet6 addr: fe80::21d:60ff:fee8:7538/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:1133227 errors:0 dropped:0 overruns:0 frame:0
> TX packets:555018 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:110538175 (110.5 MB) TX bytes:73426894 (73.4 MB)
> Interrupt:18
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:3192648 errors:0 dropped:0 overruns:0 frame:0
> TX packets:3192648 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:262066647 (262.0 MB) TX bytes:262066647 (262.0 MB)
>
> virbr0 Link encap:Ethernet HWaddr 8a:98:8b:82:4a:ec
> inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:2977 (2.9 KB)
>
> vlan100 Link encap:Ethernet HWaddr 00:1d:60:e8:75:38
> inet6 addr: fe80::21d:60ff:fee8:7538/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:42 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:6196 (6.1 KB)
>
> vnet0 Link encap:Ethernet HWaddr fe:16:3e:10:3a:18
> inet6 addr: fe80::fc16:3eff:fe10:3a18/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:237 errors:0 dropped:0 overruns:0 frame:0
> TX packets:218 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:500
> RX bytes:26092 (26.0 KB) TX bytes:32775 (32.7 KB)
> --------------------------
> brctl show
>
> bridge name bridge id STP enabled interfaces
> br100 8000.001d60e87538 no vlan100
> vnet0
> virbr0 8000.000000000000 yes
> -----------------------
>
> Please give me some advices.
> Thanks!
>
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
KiRaDo (cuonghoangce) said :
#2

Thanks Vish,
I only created a project. I can't see: "vlan isn't enabled on your switch" and "you need to enable vlan trunking on the vlans that are used by nova".
Could you explain that in more detail?

Revision history for this message
Best Vish Ishaya (vishvananda) said :
#3

The switch that connects your host machines needs to allow the hosts to add vlan tags to the packets they send and pass them unmodified. This is generally called VLAN trunking. You generally have to configure the switch with specific vlan numbers for this to work.

If your switch doesn't support vlan trunking or you can't figure out how to turn it on, you will probably have more luck using FlatDHCPManager instead of VlanManager.

Vish

On Apr 7, 2011, at 7:40 PM, KiRaDo wrote:

> Question #151941 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/151941
>
> Status: Answered => Open
>
> KiRaDo is still having a problem:
> Thanks Vish,
> I only created a project. I can't see: "vlan isn't enabled on your switch" and "you need to enable vlan trunking on the vlans that are used by nova".
> Could you explain that in more detail?
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
KiRaDo (cuonghoangce) said :
#4

Thanks Vish Ishaya, that solved my question.