neutron

configure Neutron with 2 external networks using linuxbridge plugin / agent

Asked by Andreas Schröder

I have 2 VLANs on the physical interface of my single-node OpenStack Havana setup based on Ubuntu Linux 12.04 LTS x64. The interface itself is a bond device of 2 10 GE links connected to a switch via LACP.
I would like to use the linuxbridge plugin/agent instead of openvswitch in order to try to keep things less complex.

The goal is to have multiple tenants with their own private networks and floating IPs of either the first or the second external interface. No tenant shall be connected to both external VLANs at the same time.

What I read so far sounds like this should be possible if I would use linuxbridge in vlan mode and create 2 provider networks with the known VLAN IDs. For each provider network a single L3 (and metadata) agent would be needed. Then I would create a router for each tenant network and connect it to one of the 2 provider networks.
With the quota setting for the number of routers I could enforce that each tenant is only connected to the first or the second provider network but not both at the same time.

So far I didn't have success with running 2 L3 agents at the same time. I also set the host variable in each config file to virtual hostnames (aliases to public IP in /etc/hosts).
When starting the second L3 agent the bridge configuration seems to get messed up. Restarting both L3 agents multiple times also doesn't work.

Question information

Language:
English Edit question
Status:
Solved
For:
neutron Edit question
Assignee:
No assignee Edit question
Solved by:
Andreas Schröder
Solved:
Last query:
Last reply:
Revision history for this message
ZhiQiang Fan (aji-zqfan) said : 		#1

are the namespaces, which are created by the first l3, flushed?

Revision history for this message
Andreas Schröder (andreas-schroeder-s) said : 		#2

changes to neutron.conf :
core_plugin = neutron.plugins.linuxbridge.lb_neutron_plugin.LinuxBridgePluginV2

changes to dhcp_agent.ini:
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
use_namespaces = True

changes to l3_agent.ini:
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
use_namespaces = True
gateway_external_network_id = ff6f097a-3efd-4bd7-9a1a-dd477df3cef6
handle_internal_only_routers = True
external_network_bridge =
host = l3agent-vlan100

changes to l3_agent_VLAN200.ini:
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver
use_namespaces = True
gateway_external_network_id = 54d5d0fb-f039-4ce0-bad7-423c79cb1e16
handle_internal_only_routers = False
external_network_bridge =
host = l3agent-vlan200

changes in plugins/linuxbridge/linuxbridge_conf.ini:
[VLANS]
network_vlan_ranges=physnet1:100:100,physnet1:200:200,physnet1:1000:1100
tenant_network_type=vlan

[SECURITYGROUP]
firewall_driver=neutron.agent.linux.iptables_firewall.IptablesFirewallDriver

[LINUX_BRIDGE]
physical_interface_mappings=physnet1:bond0

Revision history for this message
Andreas Schröder (andreas-schroeder-s) said : 		#3

The namespaces are not flushed, but it seems to try the delete them.
I deleted all routers and networks and started from the ground up. First, I stopped the L3 agents and added both provider networks. Then I updated the network IDs in the L3 agent configuration and started the first L3 agent.
After that I added a tenant network and a router. After connecting the router to the first provider network I started the second L3 agent.

## after adding router and router interface to tenant network:

# ip netns
qrouter-8d3522f4-2b1a-4049-9239-f5a906208484
# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.000000000000 no
brq29325e60-4d 8000.00304864eebd no bond0.1000
       tap0f21ae71-05

## after setting gateway on router:

# ip netns
qrouter-8d3522f4-2b1a-4049-9239-f5a906208484
# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.000000000000 no
brq29325e60-4d 8000.00304864eebd no bond0.1000
       tap0f21ae71-05
brq4fd0d5eb-4b 8000.00304864eebd no bond0.100
       tap529b2f60-18

## after starting second l3-agent:

# ip netns
qrouter-8d3522f4-2b1a-4049-9239-f5a906208484
# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.000000000000 no

==> /var/log/neutron/neutron-netns-cleanup.log <==
2013-10-22 18:00:04.504 5407 ERROR neutron.agent.netns_cleanup_util [-] Error unable to destroy namespace: qrouter-8d3522f4-2b1a-4049-9239-f5a906208484
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util Traceback (most recent call last):
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/netns_cleanup_util.py", line 135, in destroy_namespace
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util ip.garbage_collect_namespace()
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 141, in garbage_collect_namespace
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util self.netns.delete(self.namespace)
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 442, in delete
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util self._as_root('delete', name, use_root_namespace=True)
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 208, in _as_root
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util kwargs.get('use_root_namespace', False))
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 65, in _as_root
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util namespace)
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ip_lib.py", line 76, in _execute
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util root_helper=root_helper)
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util File "/usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py", line 62, in execute
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util raise RuntimeError(m)
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util RuntimeError:
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'delete', 'qrouter-8d3522f4-2b1a-4049-9239-f5a906208484']
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util Exit code: 1
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util Stdout: ''
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util Stderr: 'Cannot remove /var/run/netns/qrouter-8d3522f4-2b1a-4049-9239-f5a906208484: Device or resource busy\n'
2013-10-22 18:00:04.504 5407 TRACE neutron.agent.netns_cleanup_util

Revision history for this message
Andreas Schröder (andreas-schroeder-s) said : 		#4

Setting 'handle_internal_only_routers = False' for both L3 agents seems to fix this.
I also added 'metadata_port = 9698' to the second L3 agent.

This is what it looks like now with 2 routers and 2 running VMs:

# brctl show
bridge name bridge id STP enabled interfaces
br100 8000.000000000000 no
brq29325e60-4d 8000.00304864eebd no bond0.1000
       tap0f21ae71-05
       tapee8ffdda-66
       tapf85d12c2-2c
brq4fd0d5eb-4b 8000.00304864eebd no bond0.100
       tap529b2f60-18
brq5ed48ee6-20 8000.00304864eebd no bond0.1001
       tap3dac45ff-c5
       tapbd688c3a-0e
       tapf3ce8462-f9
brq60cdeba9-03 8000.00304864eebd no bond0.200
       tap4b3f218b-63

# ip netns
qrouter-b14d8e5d-506f-43f2-bcc0-5252bda9f0f2
qdhcp-29325e60-4dee-470c-b669-824a8f7503a8
qdhcp-5ed48ee6-20d8-40bf-9075-49ca03bca37c
qrouter-8d3522f4-2b1a-4049-9239-f5a906208484