How to add additional routing rule in intance during fire it up ?

Asked by Hugo Kou

Hi all , there's a consideration of our private cloud ....as the link
http://hugokuo-hugo.blogspot.com/2011/06/method-of-web-service-halb-in-openstack.html

Due to nova-network host is not easy to do HA and LB , the idea from NTT PF lab might works , but still only Active/passive HA .
It's quite waste the network resources.

In our plan , we don't need floating_ip. So I plan to route instance's traffic from two pure router to do HA and LB.
It's like App layer HA+LB.......The details could be found in my newest blog post.

Due to UEC image need to retrieve metadata from nova-api server , so that gateway from nova-network host is necessary. nova-network prerouting 169.254.169.254:80 request to nova-api server for instacne.

Is there any suggestion to set instance's default gateway from other address instead of nova-network? (not configure it manually)

DHCP seems impossible to do that , while instance get dhcp response . All network parameter is been set in instance . So is that possible to inject network configuration with FlatMode ?

Consideration...

1.how about reset gateway after instance get metadata successfully ....(modify cloud-init??)

2.How to customize network inject file ?

Cheers
Hugo Kuo

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Hugo Kou (tonytkdk) said :
#1

jump

Revision history for this message
Launchpad Janitor (janitor) said :
#2

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Vish Ishaya (vishvananda) said :
#3

You can force dnsmasq to give out another gateway with
dhcp-option=3,<gateway_ip>
in your dnsmasq config file. You can specify a custom config file with nova using:
--dnsmasq_config_file

Keep in mind that you will need to set up metadata forwarding rules manually on the gateway.

Vish

On Jun 22, 2011, at 12:35 AM, Hugo Kou wrote:

> New question #162303 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/162303
>
> Hi all , there's a consideration of our private cloud ....as the link
> http://hugokuo-hugo.blogspot.com/2011/06/method-of-web-service-halb-in-openstack.html
>
> Due to nova-network host is not easy to do HA and LB , the idea from NTT PF lab might works , but still only Active/passive HA .
> It's quite waste the network resources.
>
> In our plan , we don't need floating_ip. So I plan to route instance's traffic from two pure router to do HA and LB.
> It's like App layer HA+LB.......The details could be found in my newest blog post.
>
> Due to UEC image need to retrieve metadata from nova-api server , so that gateway from nova-network host is necessary. nova-network prerouting 169.254.169.254:80 request to nova-api server for instacne.
>
> Is there any suggestion to set instance's default gateway from other address instead of nova-network? (not configure it manually)
>
> DHCP seems impossible to do that , while instance get dhcp response . All network parameter is been set in instance . So is that possible to inject network configuration with FlatMode ?
>
> Consideration...
>
> 1.how about reset gateway after instance get metadata successfully ....(modify cloud-init??)
>
> 2.How to customize network inject file ?
>
>
> Cheers
> Hugo Kuo
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Revision history for this message
Hugo Kou (tonytkdk) said :
#4

Thanks Vish...
Perfect answer for me,

But ,what's the usage of User Data part while launch image on Dashboard?
I thought that's a script which users could execute after instance fire up .
My original consideration is delete default gateway(nova-network host ip) then add another gateway with User Data in the instance.

Is there any negative effect while change the gateway of instance with the approach by force dnsmasq ?

The question is from a critical mistake for me in our internal cloud today. After I shutdown the nova-network host(nova-volume+network+scheduler) for changing faster NIC , I lost all network connection with instances those I ran up before trun off nova-network host. From any network , even the instance network ( I connected laptop to same switch with instance)

But the cloud still works fine with fire up new instance.

How come? As I know , nova-network(FlatDHCP) responsible for
1. iptables for floating IP
2. assign ip for each instance
3. gateway for instance

which wil cause a critical network problem like this? I'm interesting in the reason.

Another confusing point is which component handle "security groups" .
While I checked the iptable rule , it seems handled by nova-compute node . Am I right?
What if a client in same network with instance private IP , is this client still limited by security group ?
I argued with my leader about this question all day long.

I'm sorry about such many questions . But it must be very useful to know about nova-network for every one :>

Thanks for your answer.

Hugo Kuo

Revision history for this message
Hugo Kou (tonytkdk) said :
#5

Something cool today , those instances lost connection yesterday . They could be connect now.

Very strange.....and very interesting ...

Revision history for this message
Vish Ishaya (vishvananda) said :
#6

On Jul 7, 2011, at 8:35 AM, Hugo Kou wrote:

> Question #162303 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/162303
>
> Status: Expired => Open
>
> Hugo Kou is still having a problem:
> Thanks Vish...
> Perfect answer for me,
>
> But ,what's the usage of User Data part while launch image on Dashboard?
> I thought that's a script which users could execute after instance fire up .
> My original consideration is delete default gateway(nova-network host ip) then add another gateway with User Data in the instance.

This seems a little dangerous. User data is supposed to be set by users.
>
>
> Is there any negative effect while change the gateway of instance with the approach by force dnsmasq ?

I don't know the answer to this.

>
> The question is from a critical mistake for me in our internal cloud
> today. After I shutdown the nova-network host(nova-
> volume+network+scheduler) for changing faster NIC , I lost all network
> connection with instances those I ran up before trun off nova-network
> host. From any network , even the instance network ( I connected laptop
> to same switch with instance)

The network is a little fragile at the moment, hence the addition of the stuff in ~vishvananda/nova/ha-net

I'm not sure exactly what went wrong here. It seems like existing instances would get new dhcp leases when the network came back. Are you sure they didn't crash?

>
> But the cloud still works fine with fire up new instance.
>
> How come? As I know , nova-network(FlatDHCP) responsible for
> 1. iptables for floating IP
> 2. assign ip for each instance
> 3. gateway for instance
>
> which wil cause a critical network problem like this? I'm interesting in
> the reason.
>
> Another confusing point is which component handle "security groups" .
> While I checked the iptable rule , it seems handled by nova-compute node . Am I right?
> What if a client in same network with instance private IP , is this client still limited by security group ?
> I argued with my leader about this question all day long.

Security groups are created on the host level by nova-compute. There is an automatic allow rule if --allow_project_net_traffic is true to allow all traffic from the project network, so instances from the same project can talk to each other. You can turn this off by changing the flag.

>
> I'm sorry about such many questions . But it must be very useful to know
> about nova-network for every one :>
>
> Thanks for your answer.
>
> Hugo Kuo
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Can you help with this problem?

Provide an answer of your own, or ask Hugo Kou for more information if necessary.

To post a message you must log in.