Quantum L3 on multinode

ovs_quantum_plugin.ini:
network_vlan_ranges = default:1:100,physnet1
bridge_mappings = default:br-default,physnet1:br-eth0

l3_agent.ini:
external_network_bridge = br-ex

on controller node:
we can run quantum-server --config-file /etc/quantum/quantum.conf --config-file/etc/quantum/ovs_quantum_plugin.ini

we assume two computes nodes: hosta and hostb:
on hosta with two nics, dummy0 and eth0:
A create ovs switch to public network:
sudo ovs-vsctl add-br br-ex
sudo ovs-vsctl add-port br-ex dummy0
B. create ovs switch to connect hostb with via vlan bridge, assuming vlanid 1000:
sudo ovs-vsctl add-br br-eth0
sudo ovs-vsctl add-port br-eth0 eth0.1000 tag=1000 -- set interface eth0.1000 type=internal
C. create ovs bridge for default (we have to do it because we must provide physical binding for virtual network)
sudo ovs-vsctl add-br br-default

C: run binaries:
quantum-l3-agent --config-file /etc/quantum/quantum.conf --config-file/etc/quantum/l3_agent.ini
quantum-openvswitch-agent --config-file /etc/quantum/quantum.conf --config-file/etc/quantum/ovs_quantum_plugin.ini

on hostb with one nic: eth0:
A. create ovs switch to connect hosta with via vlan bridge, assuming vlanid 1000:
1. sudo ovs-vsctl add-br br-eth0
2. sudo ovs-vsctl add-port br-eth0 eth0.1000 tag=1000 -- set interface eth0.1000 type=internal
B: binaries:
quantum-openvswitch-agent --config-file /etc/quantum/quantum.conf --config-file/etc/quantum/ovs_quantum_plugin.ini

all below steps are on hosta since we will use sudo ip netns to check:

create networks:
A. provider network for VMs
quantum net-create private_net --provider:network_type vlan --provider:physical_network physnet1 --provider:segmentation_id 1000
quantum subnet-create private_net 10.0.1.0/24 --name private_subnet
B. external network for floating and router gateway:
quantum net-create external_net --router:external true
quantum subnet-create external_net 8.0.1.0/24 --enable_dhcp false

playing with servers:
b6d25722-b21d-47c0-940f-e140bd5fecfa is id of private_net
nova boot --image 39fc8570-13b7-484d-8ebd-377104e3e1d1 --flavor 1 myserver1 --nic net-id=b6d25722-b21d-47c0-940f-e140bd5fecfa
sudo ip netns exec qdhcp-b6d25722-b21d-47c0-940f-e140bd5fecfa ping 10.0.1.3

playing with router:
quantum router-create myrouter
quantum router-gateway-set myrouter external_net
quantum router-interface-add myrouter private_subnet
in vm:
8.0.1.2 is ip of router gateway port.
ping 8.0.1.2

playing with floating:
4939a98b-4084-4fc0-9d28-0c3938f22f98 is id for router myrouter

quantum floatingip-create external_net
2932819a-c0be-49ec-a11c-97c5b60d643a is id for floating ip
e37987cb-4c06-4637-95bc-24b6985ac88f is id for myserver1's port
quantum floatingip-associate 2932819a-c0be-49ec-a11c-97c5b60d643a e37987cb-4c06-4637-95bc-24b6985ac88f
sudo ip netns exec qrouter-4939a98b-4084-4fc0-9d28-0c3938f22f98 ping 8.0.1.3

About HA, I don't know what u mean to do? which part you want to HA?

Thanks. Had similar idea.

For HA, don't we meed Router on each Node?

The model I had was they have private network to talk to each other, but when going on to internet, they would have a dedciated route on each node.

Otherwise one node with Routing becomes a Single Point of Failure.

Then how does the S NAT and D NAT Rules apply for Compute Node to talk to Internet - with remote Router Node as well as local Router Node.

I am keeping VLAN model from Nova Network in mind but with HA as an add on option. I hope all these are expalined.

Did not follow

C. create ovs bridge for default (we have to do it because we must provide physical binding for virtual network)
sudo ovs-vsctl add-br br-default

understood for br-ex and br-eth0.

each virtual network must have a corresponding physical network defined for it in VLAN mode. Since we have used br-eth0 ( which has only one vlan id 1000) for network private_net, we must provide another one for network external_net, which is br-default. But in fact, we are not going to make use of it for traffic since we will use br-ex to do public related traffic. (It looks like a default of current ovs plugin.)

Router routes traffic from a subnet to a external network. We can distribute router among hosts to avoid the single failure point. By now we have no nova-like multi-host implemented. Wait for nova-like multi-host implemented, we can make the router multi-host too.

router is using SNAT like:
-A quantum-l3-agent-POSTROUTING -s 10.0.1.0/24 -d 8.0.1.2/32 -j ACCEPT
-A quantum-l3-agent-snat -s 10.0.1.0/24 -j SNAT --to-source 8.0.1.2

floatingip is using DNAT and SNAT like:
sudo ip netns exec qrouter-4939a98b-4084-4fc0-9d28-0c3938f22f98 ip -4 addr add 8.0.1.3/32 brd 8.0.1.3 scope global dev {gw_iface}
OUTPUT -d 8.0.1.3/32 -j DNAT --to-destination 10.0.1.3
PREROUTING -d 8.0.1.3/32 -j DNAT --to-destination 10.0.1.3
float-snat -s 10.0.1.5/32 -j SNAT --to-source 8.0.1.3

you can use the command to see iptables:
 sudo ip netns exec qrouter-4939a98b-4084-4fc0-9d28-0c3938f22f98 iptables-save