A cactus cluster of two computers need glance?

Glance on one computer should be sufficient with the following flags set on both computers:

--glance_host=<ip of glance computer>
--image_service=nova.image.glance.GlanceImageService

On Apr 19, 2011, at 11:45 AM, arturo lorenzo wrote:

> New question #153485 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/153485
>
> I have a couple of computers: computer-1 running all the cactus services and computer-2 just running nova-compute. I am using FlatDHCPManager and I can run instances on computer-1 but not on computer-2.
> Here is the error when I try to run an instance on computer-2, but I think the question is if I need to run glance on computer-1 and computer-2.
> =============================================================================
> 2011-04-19 13:24:42,312 DEBUG nova.utils [-] Running cmd (subprocess): mkdir -p /var/lib/nova/instances/instance-0000000b/ from (pid=982) execute /usr
> /lib/pymodules/python2.6/nova/utils.py:150
> 2011-04-19 13:24:42,354 INFO nova.virt.libvirt_conn [-] instance instance-0000000b: Creating image
> 2011-04-19 13:24:42,454 DEBUG nova.utils [-] Attempting to grab semaphore "021316eb" for method "call_if_not_exists"... from (pid=982) inner /usr/lib/
> pymodules/python2.6/nova/utils.py:594
> 2011-04-19 13:24:42,454 DEBUG nova.utils [-] Running cmd (subprocess): cp /var/lib/nova/instances/_base/021316eb /var/lib/nova/instances/instance-0000
> 000b/kernel from (pid=982) execute /usr/lib/pymodules/python2.6/nova/utils.py:150
> 2011-04-19 13:24:42,491 DEBUG nova.utils [-] Attempting to grab semaphore "06d801f3" for method "call_if_not_exists"... from (pid=982) inner /usr/lib/
> pymodules/python2.6/nova/utils.py:594
> 2011-04-19 13:24:42,537 ERROR nova.compute.manager [O-GNOG-GHWR-ELW-D3SC admin simple_cloud] Instance '11' failed to spawn. Is virtualization enabled
> in the BIOS?
> (nova.compute.manager): TRACE: Traceback (most recent call last):
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/compute/manager.py", line 234, in run_instance
> (nova.compute.manager): TRACE: self.driver.spawn(instance_ref)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/exception.py", line 120, in _wrap
> (nova.compute.manager): TRACE: return f(*args, **kw)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 616, in spawn
> (nova.compute.manager): TRACE: self._create_image(instance, xml, network_info)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 867, in _create_image
> (nova.compute.manager): TRACE: size=size)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 776, in _cache_image
> (nova.compute.manager): TRACE: call_if_not_exists(base, fn, *args, **kwargs)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/utils.py", line 607, in inner
> (nova.compute.manager): TRACE: retval = f(*args, **kwargs)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 774, in call_if_not_exists
> (nova.compute.manager): TRACE: fn(target=base, *args, **kwargs)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/libvirt_conn.py", line 787, in _fetch_image
> (nova.compute.manager): TRACE: images.fetch(image_id, target, user, project)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/virt/images.py", line 51, in fetch
> (nova.compute.manager): TRACE: metadata = image_service.get(elevated, image_id, image_file)
> (nova.compute.manager): TRACE: File "/usr/lib/pymodules/python2.6/nova/image/local.py", line 116, in get
> (nova.compute.manager): TRACE: raise exception.NotFound
> (nova.compute.manager): TRACE: NotFound: None
> (nova.compute.manager): TRACE:
> 2011-04-19 13:24:44,943 INFO nova.compute.manager [-] Found instance 'instance-0000000b' in DB but no VM. State=8, so setting state to shutoff.
> =============================================================================
> Here is my nova.conf on both computers
>
> --verbose
> --dhcpbridge_flagfile=/etc/nova/nova.conf
> --dhcpbridge=/usr/bin/nova-dhcpbridge
> --logdir=/var/log/nova
> --state_path=/var/lib/nova
> --s3_host=192.168.168.100
> --rabbit_host=192.168.168.100
> --cc_host=192.168.168.100
> --ec2_url=http://192.168.168.100:8773/services/Cloud
> --fixed_range=10.0.0.0/12
> --network_size=64
> --FAKE_subdomain=ec2
> --sql_connection=mysql://root:nova@192.168.168.100/nova
> --network_manager=nova.network.manager.FlatDHCPManager
> --public_interface=eth0
> --scheduler_driver=nova.scheduler.simple.SimpleScheduler
> --lock_path=/tmp
>
> Thanks for your help!
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Vish,
that helped to get the image correctly. Thanks so much! Now I can get the image but I am running into an old problem. I can run instances on computer-1 but when running on computer-2 I get this error on the instance console:
=============================================================================
[ 0.709455] Freeing unused kernel memory: 1612k freed
init: plymouth-splash main process (329) terminated with status 2
init: plymouth main process (49) killed by SEGV signal
cloud-init start-local running: Tue, 19 Apr 2011 22:04:21 +0000. up 6.06 seconds
no instance data found in start-local
init: cloud-init-local main process (292) terminated with status 1
cloud-init start running: Tue, 19 Apr 2011 22:05:20 +0000. up 65.34 seconds
2011-04-19 22:05:20,331 - DataSourceEc2.py[WARNING]: waiting for metadata service at http://169.254.169.254/2009-04-04/meta-data/instance-id

2011-04-19 22:05:20,333 - DataSourceEc2.py[WARNING]: 22:05:20 [ 1/100]: url error [[Errno 101] Network is unreachable]

2011-04-19 22:05:21,337 - DataSourceEc2.py[WARNING]: 22:05:21 [ 2/100]: url error [[Errno 101] Network is unreachable]

2011-04-19 22:05:22,342 - DataSourceEc2.py[WARNING]: 22:05:22 [ 3/100]: url error [[Errno 101] Network is unreachable]

2011-04-19 22:05:23,347 - DataSourceEc2.py[WARNING]: 22:05:23 [ 4/100]: url error [[Errno 101] Network is unreachable]

2011-04-19 22:05:24,353 - DataSourceEc2.py[WARNING]: 22:05:24 [ 5/100]: url error [[Errno 101] Network is unreachable]

2011-04-19 22:05:25,358 - DataSourceEc2.py[WARNING]: 22:05:25 [ 6/100]: url error [[Errno 101] Network is unreachable]

2011-04-19 22:05:27,365 - DataSourceEc2.py[WARNING]: 22:05:27 [ 7/100]: url error [[Errno 101] Network is unreachable]
=============================================================================

here is again my nova.conf

--verbose
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--s3_host=192.168.168.100
--rabbit_host=192.168.168.100
--cc_host=192.168.168.100
--ec2_url=http://192.168.168.100:8773/services/Cloud
--fixed_range=10.0.0.0/12
--network_size=64
--FAKE_subdomain=ec2
--sql_connection=mysql://root:nova@192.168.168.100/nova
--network_manager=nova.network.manager.FlatDHCPManager
--public_interface=eth0
--scheduler_driver=nova.scheduler.simple.SimpleScheduler
--lock_path=/tmp
--glance_host=192.168.168.100
--image_service=nova.image.glance.GlanceImageService
--------------------------------------------------------------------------------------------------
my br100 shows no ip and eth0 shows the ip and the output of brctl show is:
bridge name bridge id STP enabled interfaces
br100 8000.fe163e2e5918 no vnet0
virbr0 8000.000000000000 yes

and my iptables is good because I can access the cc host correctly using wget 169.254.169.254 on computer-2.

any ideas?

you have to set
--flat_interface=eth0
for flat dhcp to work with multiple machines

On Apr 19, 2011, at 3:13 PM, arturo lorenzo wrote:

> Question #153485 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153485
>
> Status: Answered => Open
>
> arturo lorenzo is still having a problem:
> Vish,
> that helped to get the image correctly. Thanks so much! Now I can get the image but I am running into an old problem. I can run instances on computer-1 but when running on computer-2 I get this error on the instance console:
> =============================================================================
> [ 0.709455] Freeing unused kernel memory: 1612k freed
> init: plymouth-splash main process (329) terminated with status 2
> init: plymouth main process (49) killed by SEGV signal
> cloud-init start-local running: Tue, 19 Apr 2011 22:04:21 +0000. up 6.06 seconds
> no instance data found in start-local
> init: cloud-init-local main process (292) terminated with status 1
> cloud-init start running: Tue, 19 Apr 2011 22:05:20 +0000. up 65.34 seconds
> 2011-04-19 22:05:20,331 - DataSourceEc2.py[WARNING]: waiting for metadata service at http://169.254.169.254/2009-04-04/meta-data/instance-id
>
> 2011-04-19 22:05:20,333 - DataSourceEc2.py[WARNING]: 22:05:20 [
> 1/100]: url error [[Errno 101] Network is unreachable]
>
> 2011-04-19 22:05:21,337 - DataSourceEc2.py[WARNING]: 22:05:21 [
> 2/100]: url error [[Errno 101] Network is unreachable]
>
> 2011-04-19 22:05:22,342 - DataSourceEc2.py[WARNING]: 22:05:22 [
> 3/100]: url error [[Errno 101] Network is unreachable]
>
> 2011-04-19 22:05:23,347 - DataSourceEc2.py[WARNING]: 22:05:23 [
> 4/100]: url error [[Errno 101] Network is unreachable]
>
> 2011-04-19 22:05:24,353 - DataSourceEc2.py[WARNING]: 22:05:24 [
> 5/100]: url error [[Errno 101] Network is unreachable]
>
> 2011-04-19 22:05:25,358 - DataSourceEc2.py[WARNING]: 22:05:25 [
> 6/100]: url error [[Errno 101] Network is unreachable]
>
> 2011-04-19 22:05:27,365 - DataSourceEc2.py[WARNING]: 22:05:27 [ 7/100]: url error [[Errno 101] Network is unreachable]
> =============================================================================
>
> here is again my nova.conf
>
> --verbose
> --dhcpbridge_flagfile=/etc/nova/nova.conf
> --dhcpbridge=/usr/bin/nova-dhcpbridge
> --logdir=/var/log/nova
> --state_path=/var/lib/nova
> --s3_host=192.168.168.100
> --rabbit_host=192.168.168.100
> --cc_host=192.168.168.100
> --ec2_url=http://192.168.168.100:8773/services/Cloud
> --fixed_range=10.0.0.0/12
> --network_size=64
> --FAKE_subdomain=ec2
> --sql_connection=mysql://root:nova@192.168.168.100/nova
> --network_manager=nova.network.manager.FlatDHCPManager
> --public_interface=eth0
> --scheduler_driver=nova.scheduler.simple.SimpleScheduler
> --lock_path=/tmp
> --glance_host=192.168.168.100
> --image_service=nova.image.glance.GlanceImageService
> --------------------------------------------------------------------------------------------------
> my br100 shows no ip and eth0 shows the ip and the output of brctl show is:
> bridge name bridge id STP enabled interfaces
> br100 8000.fe163e2e5918 no vnet0
> virbr0 8000.000000000000 yes
>
> and my iptables is good because I can access the cc host correctly using
> wget 169.254.169.254 on computer-2.
>
> any ideas?
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Vish, still with problems.
In Bexar: computer-1 was running all nova-components and nova.conf had public_interface=eth0 and flat_interface=eth2 FlatDHCPManager.
computer-2 was running nova-compute only with the same nova.conf.
computer-1 did not have the br100 configured on /etc/network/interfaces only static IP on eth0 and on eth1.
When nova-network starts will set the br100 with 10.0.0.1 subnet and eth0 with the current IP from interfaces.
computer-2 did not have the br100 configured on /etc/network/interfaces only static IP on eth0.
This bridge is set when an instance is being launched on this computer-2 and br100 will have not IP and eth0 the current IP.
physical connections: computer-1: eth0 to the public net, eth1 to a switch where computer-2 was connected and eth2 to the same switch
with no IP assigned.
computer-2: eth0 to the switch and eth2 to the same switch with no IP assigned.
I was able to run instances successfully on both nova-compute nodes.
Now in Cactus: I have the same configuration but I am not connecting eth2 to the switch and I am using flat_interface=eth0 and I have added glance on computer-1 to serve images.
I can launch instances from computer-1 OK but not from computer-2, which I get :
2011-04-19 22:05:20,333 - DataSourceEc2.py[WARNING]: 22:05:20 [ 1/100]: url error [[Errno 101] Network is unreachable]
I am able to get images from computer-1 (glance) into computer-2 OK.
Any ideas?
Thx

if you do a brctl show does it show br100 bridged in to eth0 on both machines?

Vish

On Apr 20, 2011, at 8:17 AM, arturo lorenzo wrote:

> Question #153485 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153485
>
> Status: Answered => Open
>
> arturo lorenzo is still having a problem:
> Vish, still with problems.
> In Bexar: computer-1 was running all nova-components and nova.conf had public_interface=eth0 and flat_interface=eth2 FlatDHCPManager.
> computer-2 was running nova-compute only with the same nova.conf.
> computer-1 did not have the br100 configured on /etc/network/interfaces only static IP on eth0 and on eth1.
> When nova-network starts will set the br100 with 10.0.0.1 subnet and eth0 with the current IP from interfaces.
> computer-2 did not have the br100 configured on /etc/network/interfaces only static IP on eth0.
> This bridge is set when an instance is being launched on this computer-2 and br100 will have not IP and eth0 the current IP.
> physical connections: computer-1: eth0 to the public net, eth1 to a switch where computer-2 was connected and eth2 to the same switch
> with no IP assigned.
> computer-2: eth0 to the switch and eth2 to the same switch with no IP assigned.
> I was able to run instances successfully on both nova-compute nodes.
> Now in Cactus: I have the same configuration but I am not connecting eth2 to the switch and I am using flat_interface=eth0 and I have added glance on computer-1 to serve images.
> I can launch instances from computer-1 OK but not from computer-2, which I get :
> 2011-04-19 22:05:20,333 - DataSourceEc2.py[WARNING]: 22:05:20 [ 1/100]: url error [[Errno 101] Network is unreachable]
> I am able to get images from computer-1 (glance) into computer-2 OK.
> Any ideas?
> Thx
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Vish,
yes on computer-1 is:
bridge name bridge id STP enabled interfaces
br100 8000.001d92a0f4ce no eth0
virbr0 8000.000000000000 yes

and on computer-2 is:
bridge name bridge id STP enabled interfaces
br100 8000.001d92a0f47a no eth0
                                                        vnet0
virbr0 8000.000000000000 yes

but in computer-2 only happens when launching an instance and br100 contains the IP address which it did not in the past.
Thanks for the help!

That is all correct.

I would check that you have ip forwarding enabled on computer-1 and you may have an incorrect setting for the forward rule.
You probably need to manually specify --my_ip on computer-1:
--my_ip=<the public ip of the box>

if you do iptables -t -nat -L -n -v | grep 169.254 you should see a rule that has the public ip in it. I'm guessing that it has the wrong ip there.
If so, flush your iptables rules
iptables -F
iptables -t nat -F

set --my_ip as above
restart all services on on compute1

verify the iptables rule is set properly
try running the instance again

you may also have to set:
ip link set promisc on dev br100
in order to get out to the public internet from instances on computer-2

Vish

On Apr 20, 2011, at 8:31 AM, arturo lorenzo wrote:

> Question #153485 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153485
>
> Status: Answered => Open
>
> arturo lorenzo is still having a problem:
> Vish,
> yes on computer-1 is:
> bridge name bridge id STP enabled interfaces
> br100 8000.001d92a0f4ce no eth0
> virbr0 8000.000000000000 yes
>
> and on computer-2 is:
> bridge name bridge id STP enabled interfaces
> br100 8000.001d92a0f47a no eth0
> vnet0
> virbr0 8000.000000000000 yes
>
> but in computer-2 only happens when launching an instance and br100 contains the IP address which it did not in the past.
> Thanks for the help!
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Vish,
I can run instances on computer-1 successfully. My main problem is when running the instances on computer-2.
if I do wget 169.254.169.254 on computer-2, I do get the information OK, meaning that it redirects it to my computer-1 port 8773 correctly. Here is the content of that call from computer-2:
=================================================
wget 169.254.169.254
--2011-04-20 13:43:25-- http://169.254.169.254/
Connecting to 169.254.169.254:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 92 [text/html]
Saving to: `index.html'

100%[====================================================================================================>] 92 --.-K/s in 0s

2011-04-20 13:43:25 (9.31 MB/s) - `index.html' saved [92/92]

cat index.html
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04
============================================================================
what I've noticed so far is that as soon as I run nova-network on computer-1 I loose my NAT access rule on computer-2 so I cannot access the internet or other computers on the net from computer-2. The only way I can get access again on computer-2 is if I delete the bridge on computer-1, restore my iptables rules and restart networking.
I think that is the main problem when running nova-network, the bridging to br100 stops communication with the other computers for NATTING purposes.
any suggestion?
thanks!

So computer-1 is the gateway for computer-2? Haven't tried that setup.

Have you tried putting br100 in promisc mode like I suggested?

Vish

On Apr 20, 2011, at 11:07 AM, arturo lorenzo wrote:

> Question #153485 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153485
>
> Status: Answered => Open
>
> arturo lorenzo is still having a problem:
> Vish,
> I can run instances on computer-1 successfully. My main problem is when running the instances on computer-2.
> if I do wget 169.254.169.254 on computer-2, I do get the information OK, meaning that it redirects it to my computer-1 port 8773 correctly. Here is the content of that call from computer-2:
> =================================================
> wget 169.254.169.254
> --2011-04-20 13:43:25-- http://169.254.169.254/
> Connecting to 169.254.169.254:80... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 92 [text/html]
> Saving to: `index.html'
>
> 100%[====================================================================================================>]
> 92 --.-K/s in 0s
>
> 2011-04-20 13:43:25 (9.31 MB/s) - `index.html' saved [92/92]
>
> cat index.html
> 1.0
> 2007-01-19
> 2007-03-01
> 2007-08-29
> 2007-10-10
> 2007-12-15
> 2008-02-01
> 2008-09-01
> 2009-04-04
> ============================================================================
> what I've noticed so far is that as soon as I run nova-network on computer-1 I loose my NAT access rule on computer-2 so I cannot access the internet or other computers on the net from computer-2. The only way I can get access again on computer-2 is if I delete the bridge on computer-1, restore my iptables rules and restart networking.
> I think that is the main problem when running nova-network, the bridging to br100 stops communication with the other computers for NATTING purposes.
> any suggestion?
> thanks!
>
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Yes, computer-1 has two enets: eth0 which is the public and eth1 where I connect computer-2 and using these 2 iptables cmds on computer-1:
iptables -t nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
iptables --append FORWARD --in-interface eth1 -j ACCEPT
This way computer-2 can access the everything outside OK.
I have set promisc on br100 on computer-1 and still no luck.
Here is the output of ip addr show on computer-1:
ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet 169.254.169.254/32 scope link lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1d:92:a0:f4:ce brd ff:ff:ff:ff:ff:ff
    inet192.168.1.1/24 brd 192.168.1.255 scope global eth0
    inet6 fe80::21d:92ff:fea0:f4ce/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1d:92:a0:f4:cf brd ff:ff:ff:ff:ff:ff
    inet 192.168.168.2/24 brd 192.168.168.255 scope global eth1
    inet6 fe80::21d:92ff:fea0:f4cf/64 scope link
       valid_lft forever preferred_lft forever
4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether a2:70:e5:c6:4e:2d brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
6: br100: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 00:1d:92:a0:f4:ce brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/26 brd 10.0.0.63 scope global br100
    inet192.168.1.1/24 brd 192.168.1.255 scope global br100
    inet6 fe80::2415:25ff:fef3:4894/64 scope link
       valid_lft forever preferred_lft forever
======================================
As you can see br100 is in promisc mode and has both IP subnets on it.
But when nova-network comes to play on computer-1, computer-2 cannot access the net.
Thanks for the help!

That masquerade rule is probably messing things up. You need to limit it to only the ips used by compute-2 and make sure it is the right place in the chain

Vish

On Apr 20, 2011, at 12:58 PM, arturo lorenzo wrote:

> Question #153485 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153485
>
> Status: Answered => Open
>
> arturo lorenzo is still having a problem:
> Yes, computer-1 has two enets: eth0 which is the public and eth1 where I connect computer-2 and using these 2 iptables cmds on computer-1:
> iptables -t nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
> iptables --append FORWARD --in-interface eth1 -j ACCEPT
> This way computer-2 can access the everything outside OK.
> I have set promisc on br100 on computer-1 and still no luck.
> Here is the output of ip addr show on computer-1:
> ip addr show
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> inet 169.254.169.254/32 scope link lo
> inet6 ::1/128 scope host
> valid_lft forever preferred_lft forever
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
> link/ether 00:1d:92:a0:f4:ce brd ff:ff:ff:ff:ff:ff
> inet192.168.1.1/24 brd 192.168.1.255 scope global eth0
> inet6 fe80::21d:92ff:fea0:f4ce/64 scope link
> valid_lft forever preferred_lft forever
> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
> link/ether 00:1d:92:a0:f4:cf brd ff:ff:ff:ff:ff:ff
> inet 192.168.168.2/24 brd 192.168.168.255 scope global eth1
> inet6 fe80::21d:92ff:fea0:f4cf/64 scope link
> valid_lft forever preferred_lft forever
> 4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
> link/ether a2:70:e5:c6:4e:2d brd ff:ff:ff:ff:ff:ff
> inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
> 6: br100: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
> link/ether 00:1d:92:a0:f4:ce brd ff:ff:ff:ff:ff:ff
> inet 10.0.0.1/26 brd 10.0.0.63 scope global br100
> inet192.168.1.1/24 brd 192.168.1.255 scope global br100
> inet6 fe80::2415:25ff:fef3:4894/64 scope link
> valid_lft forever preferred_lft forever
> ======================================
> As you can see br100 is in promisc mode and has both IP subnets on it.
> But when nova-network comes to play on computer-1, computer-2 cannot access the net.
> Thanks for the help!
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

wait a second. You need to specify flat_interface=eth1 on computer-1

br100 should be bridged into the internal network on both machines

On Apr 20, 2011, at 12:58 PM, arturo lorenzo wrote:

> Question #153485 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153485
>
> Status: Answered => Open
>
> arturo lorenzo is still having a problem:
> Yes, computer-1 has two enets: eth0 which is the public and eth1 where I connect computer-2 and using these 2 iptables cmds on computer-1:
> iptables -t nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
> iptables --append FORWARD --in-interface eth1 -j ACCEPT
> This way computer-2 can access the everything outside OK.
> I have set promisc on br100 on computer-1 and still no luck.
> Here is the output of ip addr show on computer-1:
> ip addr show
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> inet 127.0.0.1/8 scope host lo
> inet 169.254.169.254/32 scope link lo
> inet6 ::1/128 scope host
> valid_lft forever preferred_lft forever
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
> link/ether 00:1d:92:a0:f4:ce brd ff:ff:ff:ff:ff:ff
> inet192.168.1.1/24 brd 192.168.1.255 scope global eth0
> inet6 fe80::21d:92ff:fea0:f4ce/64 scope link
> valid_lft forever preferred_lft forever
> 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
> link/ether 00:1d:92:a0:f4:cf brd ff:ff:ff:ff:ff:ff
> inet 192.168.168.2/24 brd 192.168.168.255 scope global eth1
> inet6 fe80::21d:92ff:fea0:f4cf/64 scope link
> valid_lft forever preferred_lft forever
> 4: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
> link/ether a2:70:e5:c6:4e:2d brd ff:ff:ff:ff:ff:ff
> inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
> 6: br100: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
> link/ether 00:1d:92:a0:f4:ce brd ff:ff:ff:ff:ff:ff
> inet 10.0.0.1/26 brd 10.0.0.63 scope global br100
> inet192.168.1.1/24 brd 192.168.1.255 scope global br100
> inet6 fe80::2415:25ff:fef3:4894/64 scope link
> valid_lft forever preferred_lft forever
> ======================================
> As you can see br100 is in promisc mode and has both IP subnets on it.
> But when nova-network comes to play on computer-1, computer-2 cannot access the net.
> Thanks for the help!
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Vish, it is working!!! Thanks!!!!
I've always thought that the flat_interface should be the same on all the compute nodes and nova-network.
So the solution is that computer-1 keeps flat_interface=eth1 and computer-2 keeps flat_interface=eth0.
There is no bridge set up on computer-1 and there is a bridge setup on computer-2 on their network interfaces.

Thank you very much! and have a great day!