Hi Vish , does it work one NIC on nova-network host

Asked by Hugo Kou

Hello Vish ,
This is Hugo

I remember you post a related issue before.

I'm working on separate nova-network into a single box.

I choose FlatDHCP mode .
There's only one NIC on this host. Provide two network , one for nova service[192.168.1.0/24] another for instance's network[192.168.2.0/24].
I found that instance can not get metadata , so I made some test about iptables.

While I add a rule
iptables -t nat -A PREROUTING -d 169.254.169.254/32 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.1:80
to use simple apache test page.
It did not work , if I bind these two network on one NIC.(btw, I tried bind on br100 and eth0 too)

But once I'm using two NICs on nova-network host. Bind networks on different NICs. It could curl 169.254.169.254 correctly.
So is that means , nova-network host must have at least two NICs. Due to the limitation of iptables?

or

Just bcz I lose some iptable rules?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Vish Ishaya
Solved:
Last query:
Last reply:
Revision history for this message
Soren Hansen (soren) said :
#1

I don't understand what you're saying about an "apache test page"?

Revision history for this message
Hugo Kou (tonytkdk) said :
#2

oh sorry .
 I just install apache2 on host 192.168.1.1
and to test iptables PREROUTING .

while I preroute to 192.168.1.1:80 (http default port)
one NIC failed to curl 169.254.169.254:80
but
two NICs can curl 169.254.169.254:80

Revision history for this message
Vish Ishaya (vishvananda) said :
#3

Below

On Apr 12, 2011, at 5:02 AM, Hugo Kou wrote:

> New question #152528 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/152528
>
> Hello Vish ,
> This is Hugo
>
> I remember you post a related issue before.
>
> I'm working on separate nova-network into a single box.
>
> I choose FlatDHCP mode .
> There's only one NIC on this host. Provide two network , one for nova service[192.168.1.0/24] another for instance's network[192.168.2.0/24].
> I found that instance can not get metadata , so I made some test about iptables.

I have had this working without any special iptables rules. I have noticed that tit gets the wrong ip address though. You probably need to add:
--my_ip=192.168.1.X where X is the ip of the host.

>
> While I add a rule
> iptables -t nat -A PREROUTING -d 169.254.169.254/32 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.1:80
> to use simple apache test page.
> It did not work , if I bind these two network on one NIC.(btw, I tried bind on br100 and eth0 too)
>
> But once I'm using two NICs on nova-network host. Bind networks on different NICs. It could curl 169.254.169.254 correctly.
> So is that means , nova-network host must have at least two NICs. Due to the limitation of iptables?

No it definitely works

>
> or
>
> Just bcz I lose some iptable rules?
>
>
>
>
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
Best Vish Ishaya (vishvananda) said :
#4

I have discovered that I had to put br100 in promiscuous mode to get it to forward properly in some scenarios with one nic:

ip link set dev br100 promisc on

Vish

On Apr 12, 2011, at 9:45 AM, Vish Ishaya wrote:

> Question #152528 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/152528
>
> Status: Needs information => Answered
>
> Vish Ishaya proposed the following answer:
> Below
>
> On Apr 12, 2011, at 5:02 AM, Hugo Kou wrote:
>
>> New question #152528 on OpenStack Compute (nova):
>> https://answers.launchpad.net/nova/+question/152528
>>
>> Hello Vish ,
>> This is Hugo
>>
>> I remember you post a related issue before.
>>
>> I'm working on separate nova-network into a single box.
>>
>> I choose FlatDHCP mode .
>> There's only one NIC on this host. Provide two network , one for nova service[192.168.1.0/24] another for instance's network[192.168.2.0/24].
>> I found that instance can not get metadata , so I made some test about iptables.
>
> I have had this working without any special iptables rules. I have noticed that tit gets the wrong ip address though. You probably need to add:
> --my_ip=192.168.1.X where X is the ip of the host.
>
>>
>> While I add a rule
>> iptables -t nat -A PREROUTING -d 169.254.169.254/32 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.1:80
>> to use simple apache test page.
>> It did not work , if I bind these two network on one NIC.(btw, I tried bind on br100 and eth0 too)
>>
>> But once I'm using two NICs on nova-network host. Bind networks on different NICs. It could curl 169.254.169.254 correctly.
>> So is that means , nova-network host must have at least two NICs. Due to the limitation of iptables?
>
> No it definitely works
>
>>
>> or
>>
>> Just bcz I lose some iptable rules?
>>
>>
>>
>>
>>
>> --
>> You received this question notification because you are a member of Nova
>> Core, which is an answer contact for OpenStack Compute (nova).
>
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
Hugo Kou (tonytkdk) said :
#5

>I have had this working without any special iptables rules. I have noticed that tit gets the wrong ip address though. You >probably need to add:
>--my_ip=192.168.1.X where X is the ip of the host.

linux_net.py will try to add a rule depends on --my_ip flag..the result is PREROUTING 169.254.169.254:80 to $my_ip:8773

If I set --my_ip= 192.168.1.2 (nova-network host) , the rule will be
-A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.2:8773
but this line is incorrect. bcz nova-api service is not living on 192.168.1.2

due to this reason , I set --my_ip= 192.168.1.1 (nova-api host)
-A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.1.1:8773

Revision history for this message
Hugo Kou (tonytkdk) said :
#6

I also add another rule on nova-network host , and it works fine now.....

-A nova-network-POSTROUTING -s 192.168.2.0/24 -d 192.168.1.0/24 -j ACCEPT

thanks for your suggestion...

after two weeks study , finally separate nova-network into single box success....

all parameter and topology is over there...

http://hugokuo-hugo.blogspot.com/2011/04/this-topology-works-fine-with-bzr-984.html

I'm going to test glance , nova-volume and various topology now

Thanks Vish & Soren

Revision history for this message
Hugo Kou (tonytkdk) said :
#7

Thanks Vish Ishaya, that solved my question.