How securitygroup works from one ls to another in OVN

Asked by Taoyunxiang on 2019-07-22

I delployed ovn with openstack .The vms are associated with default security groups.

situation 1:

If connect ls1(network1) to ls2(network2) by router1, everything gose fine.

situation 2:
If set ls1 as external network to router1, ls2 connect router1 as an internal network. When ls1(network1 ,which is external network) connect ls2(network2,which is internal network), traffic failes. If I add the ls1 to security group, everything gose fine.

I do not konw the difference between situation 1 and situation 2. Can you give me some explain?

Question information

Language:
English Edit question
Status:
Solved
For:
networking-ovn Edit question
Assignee:
No assignee Edit question
Solved by:
Numan Siddique
Solved:
2019-07-24
Last query:
2019-07-24
Last reply:
2019-07-24
Daniel Alvarez (dalvarezs) said : #1

Hi Taoyunxiang,

Can you please highlight the differences with some examples? ie. on both scenarios, which commands are you using, how are you testing connectivity, etc.

As the port security is handled per port, if the ports belong to the same security group, it should not affect whether the logical switch has a localnet port or not I believe and then it's clearly a bug,

Thanks,
Daniel

Numan Siddique (numansiddique) said : #2

If you have ML2OVS setup handy, is it possible to try out the same test and provide the results ?

I think, the security group has remote group id set.

As Daniel mentioned, can you please also share the output of "openstack security group show .." to see the rules.

Thanks
Numan

Taoyunxiang (taoyunxiang) said : #4

i will show commands I used.

situation 1:
                    #neutron router-gateway-set router1 ls1 (net1)
                    #neutron router-interface-add router1 subnet2
                    #nova boot --image magic-mirros --flavor bigger --nic net-name=net2 --availability-zone nova:compute3 vm1

#nova list
vm1 | ACTIVE | - | Running | net2=192.168.3.9, 10.142.18.25

Then I get a physical machine server1 , which is 10.142.18.11

192.168.3.9 to 10.142.18.11 can pass ,but reverse can not pass

situation 2:
                    #neutron router-interface-add router1 subnet1
                    #neutron router-interface-add router1 subnet2
                    #nova boot --image magic-mirros --flavor bigger --nic net-name=net1 --availability-zone nova:compute3 vm1
                    #nova boot --image magic-mirros --flavor bigger --nic net-name=net2 --availability-zone nova:compute3 vm2
#nova list

vm1 | ACTIVE | - | Running | net1=192.168.5.5
vm2 | ACTIVE | - | Running | net2=192.168.6.6

192.168.5.5 to 192.168.6.6 can pass, reverse can also pass

Taoyunxiang (taoyunxiang) said : #6

For situation 1,the ovn-trace result as follows,

[root@ovn1 ~]# ovn-trace vlan786 'inport == "provnet-fd175338-a168-4b0d-916e-9c739f83f241" && eth.src == ec:38:8f:6c:6c:43 && ip4.src == 10.142.18.11 && ip4.dst == 10.142.18.27 && eth.dst == fa:16:3e:d5:e1:c0 && icmp && ip.ttl == 254'
# icmp,reg14=0x1,vlan_tci=0x0000,dl_src=ec:38:8f:6c:6c:43,dl_dst=fa:16:3e:d5:e1:c0,nw_src=10.142.18.11,nw_dst=10.142.18.27,nw_tos=0,nw_ecn=0,nw_ttl=254,icmp_type=0,icmp_code=0

ingress(dp="vlan786", inport="provnet-fd1753")
----------------------------------------------
 0. ls_in_port_sec_l2 (ovn-northd.c:4060): inport == "provnet-fd1753", priority 50, uuid b7f40738
    next;
 3. ls_in_pre_acl (ovn-northd.c:3162): ip && inport == "provnet-fd1753", priority 110, uuid 583b7eca
    next;
11. ls_in_arp_rsp (ovn-northd.c:4092): inport == "provnet-fd1753", priority 100, uuid e1842f91
    next;
16. ls_in_l2_lkup (ovn-northd.c:4435): eth.dst == fa:16:3e:d5:e1:c0 && is_chassis_resident("cr-lrp-7a4778"), priority 50, uuid f720c9b5
    outport = "7a4778";
    output;

egress(dp="vlan786", inport="provnet-fd1753", outport="7a4778")
---------------------------------------------------------------
 1. ls_out_pre_acl (ovn-northd.c:3148): ip && outport == "7a4778", priority 110, uuid bb5a0ed9
    next;
 9. ls_out_port_sec_l2 (ovn-northd.c:4518): outport == "7a4778", priority 50, uuid b670b872
    output;
    /* output to "7a4778", type "patch" */

ingress(dp="tyx-router-ext2", inport="lrp-7a4778")
--------------------------------------------------
 0. lr_in_admission (ovn-northd.c:5064): eth.dst == fa:16:3e:d5:e1:c0 && inport == "lrp-7a4778" && is_chassis_resident("cr-lrp-7a4778"), priority 50, uuid 5d274aff
    next;
 3. lr_in_unsnat (ovn-northd.c:5770): ip && ip4.dst == 10.142.18.27 && inport == "lrp-7a4778" && is_chassis_resident("cr-lrp-7a4778"), priority 100, uuid a2c382fa
    ct_snat;

ct_snat /* assuming no un-snat entry, so no change */
-----------------------------------------------------
 4. lr_in_dnat (ovn-northd.c:5828): ip && ip4.dst == 10.142.18.27 && inport == "lrp-7a4778" && is_chassis_resident("cr-lrp-7a4778"), priority 100, uuid e7bf66ec
    ct_dnat(192.168.3.21);

ct_dnat(ip4.dst=192.168.3.21)
-----------------------------
 7. lr_in_ip_routing (ovn-northd.c:4646): ip4.dst == 192.168.3.0/24, priority 49, uuid 32f5ddec
    ip.ttl--;
    reg0 = ip4.dst;
    reg1 = 192.168.3.1;
    eth.src = fa:16:3e:07:f0:dd;
    outport = "lrp-2b02ae";
    flags.loopback = 1;
    next;
 8. lr_in_arp_resolve (ovn-northd.c:6382): outport == "lrp-2b02ae" && reg0 == 192.168.3.21, priority 100, uuid 5ea26eb2
    eth.dst = fa:16:3e:4f:96:f3;
    next;
10. lr_in_arp_request (ovn-northd.c:6560): 1, priority 0, uuid a84a897b
    output;

egress(dp="tyx-router-ext2", inport="lrp-7a4778", outport="lrp-2b02ae")
-----------------------------------------------------------------------
 3. lr_out_delivery (ovn-northd.c:6588): outport == "lrp-2b02ae", priority 100, uuid 048d4238
    output;
    /* output to "lrp-2b02ae", type "patch" */

ingress(dp="tyx-net3", inport="2b02ae")
---------------------------------------
 0. ls_in_port_sec_l2 (ovn-northd.c:4060): inport == "2b02ae", priority 50, uuid 8f0994ff
    next;
 3. ls_in_pre_acl (ovn-northd.c:3146): ip && inport == "2b02ae", priority 110, uuid 565cc767
    next;
16. ls_in_l2_lkup (ovn-northd.c:4393): eth.dst == fa:16:3e:4f:96:f3, priority 50, uuid 8383315f
    outport = "fb1925";
    output;

egress(dp="tyx-net3", inport="2b02ae", outport="fb1925")
--------------------------------------------------------
 1. ls_out_pre_acl (ovn-northd.c:3194): ip, priority 100, uuid 9b74a7e3
    reg0[0] = 1;
    next;
 2. ls_out_pre_stateful (ovn-northd.c:3321): reg0[0] == 1, priority 100, uuid f0cc5c5d
    ct_next;

ct_next(ct_state=est|trk /* default (use --ct to customize) */)
---------------------------------------------------------------
 4. ls_out_acl (ovn-northd.c:3560): ct.est && ct_label.blocked == 0 && (outport == @neutron_pg_drop && ip), priority 2001, uuid f8a51807
    ct_commit(ct_label=0x1/0x1);

Taoyunxiang (taoyunxiang) said : #7

For situaiton1 ,
when I create a new securitygroup wich pass 10.142.18.0/24 , and associate it to vm, traffic goes.

I do not know the difference between situation 1 and situation 2 for securitygroup.

Taoyunxiang (taoyunxiang) said : #8

[root@controller1 ~]# openstack security group show e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at | 2019-04-26T06:03:13Z |
| description | Default security group |
| id | e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4 |
| name | default |
| project_id | e9346688171a407e997174156599d4ff |
| revision_number | 4 |
| rules | created_at='2019-04-26T06:03:13Z', direction='egress', ethertype='IPv6', id='1051252b-d3c2-4738-9d60-ed1b39f3eac3', updated_at='2019-04-26T06:03:13Z' |
| | created_at='2019-04-26T06:03:13Z', direction='egress', ethertype='IPv4', id='369c0bd4-be20-46dc-9931-8e867e5f1c5c', updated_at='2019-04-26T06:03:13Z' |
| | created_at='2019-04-26T06:03:13Z', direction='ingress', ethertype='IPv6', id='bd33cbf5-e274-4003-88a3-184c97c84163', remote_group_id='e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4', updated_at='2019-04-26T06:03:13Z' |
| | created_at='2019-04-26T06:03:13Z', direction='ingress', ethertype='IPv4', id='fc678467-60a5-4b72-94f0-78e6516ab0d4', remote_group_id='e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4', updated_at='2019-04-26T06:03:13Z' |
| updated_at | 2019-04-26T06:03:13Z |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Best Numan Siddique (numansiddique) said : #9

I think situation 1 is working as expected. Any traffic originating from the VM will be allowed. But any traffic destined to the VM will be dropped if there is no security group rule to allow it.

I think for sutation 2, it is working because both your VMs - vm1 (192.168.5.5) and vm2 (192.168.6.6) have same security group.
If you see the below
***
| | created_at='2019-04-26T06:03:13Z', direction='ingress', ethertype='IPv4', id='fc678467-60a5-4b72-94f0-78e6516ab0d4', remote_group_id='e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4',
*****
notice remote_group_id is set. Which means it will allow traffic from any port which belongs to the security group - 'e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4.

In scenario 1, your physical machine with ip - 10.142.18.11 obviously is not part of the security group 'e6023b9b-8d42-4fa0-a78e-b6d3b322f1d4' and hence the traffic is dropped unless you add a scurity group rule to allow traffic.

To confirm what I said is correct or not, I would suggest you to change the the security group for the vm2 port to a different one. May be you cab create your own new security group and associate to the vm2 port.

Please test it out and let me know how it goes.

Thanks
Numan

Taoyunxiang (taoyunxiang) said : #10

Hi, Numan,
Thanks for your reply, I have got it.

And I finished the test as you said , when I change security group of vm2 to another one, it can not connect to vm1.

Thanks,
Yun

Taoyunxiang (taoyunxiang) said : #11

Thanks Numan Siddique, that solved my question.