Network Injection not working

Flat Manager with uec images will probably have problems starting up due to cloud init. If you look at the console output you wil likely see errors about reaching metadata. You can add a special forwarding rule for metadata, but for one box deployment it is much easier to use FlatDHCPManager or VlanManager, which handle forwarding rules for you.

Vish

On Apr 1, 2011, at 7:06 AM, Mandeep Singh wrote:

> New question #151316 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/151316
>
> Hi,
> I am evaluating the nova on a single box with a br100 bridge and eth0 attached to it. I have given a static IP to the br100. I created a project network as follows:
> /usr/bin/nova-manage network create 192.168.11.0/24 1 256
>
> Following is the nova.conf for your reference:
>
> --dhcpbridge_flagfile=/etc/nova/nova.conf
> --dhcpbridge=/usr/bin/nova-dhcpbridge
> --logdir=/var/log/nova
> --state_path=/var/lib/nova
> --lock_path=/var/lock/nova
> --verbose
> --s3_host=192.168.11.11
> --rabbit_host=192.168.11.11
> --cc_host=192.168.11.11
> --ec2_url=http://192.168.11.11:8773/services/Cloud
> --fixed_range=192.168.0.0/16
> --network_size=8
> --FAKE_subdomain=ec2
> --routing_source_ip=192.168.11.11
> --sql_connection=mysql://root:password@192.168.11.11/nova
> --network_manager=nova.network.manager.FlatManager
> --glance_host=192.168.10.37
> --glance_port=9292
> --image_service=nova.image.glance.GlanceImageService
>
> When I launch the instances, I can see in the logs that the key is being injected during the launch but I can't see the network being injected and consequently I am not able to ping the instance. I am using the ubuntu 1386 image downloaded from Eucalyptus website.
>
> Is there any issues with the network configuration or am I missing something? Please suggest.
>
> Thanks
> Mandeep Singh
>
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Thanks for the response Vish. I forgot to mention earlier that I tried FlatDHCP and FlatVLAN as well but the same thing happening... I think the instance is launching pretty nicely and I can see it saying inject key into the image in the logs. I have attached the console output of recently launched instance for your reference. Please do let me know if your want any other information to guide me through...

...
...
...
Loading, please wait...^M
Couldnt get a file descriptor referring to the console^M
Begin: Loading essential drivers... ...^M
Done.^M
Begin: Running /scripts/init-premount ...^M
Done.^M
Begin: Mounting root file system... ...^M
Begin: Running /scripts/local-top ...^M
Done.^M
Begin: Waiting for root file system... ...^M
[ 2.090340] FDC 0 is a S82078B^M
[ 2.117343] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)^M
[ 2.118439] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10^M
[ 2.119228] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, high) -> IRQ 10^M
[ 2.123227] eth0: RTL-8139C+ at 0xe0976000, 02:16:3e:0c:aa:da, IRQ 10^M
[ 2.126156] 8139too Fast Ethernet driver 0.9.28^M
[ 2.130848] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11^M
[ 2.133447] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10^M
[ 2.134222] virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10^M
[ 2.481639] vda: unknown partition table^M
Done.^M
Begin: Running /scripts/local-premount ...^M
Begin: Waiting for resume device... ...^M
Done.^M
Done.^M
[ 7.657451] kjournald starting. Commit interval 5 seconds^M
[ 7.658522] EXT3-fs: mounted filesystem with ordered data mode.^M
Begin: Running /scripts/local-bottom ...^M
Done.^M
Done.^M
Begin: Running /scripts/init-bottom ...^M
Done.^M
 * Setting preliminary keymap... ^[[80G ^M^[[74G[ OK ]^M
 * Starting kernel event manager... ^[[80G [ 7.930645] udev: starting version 141^M
^M^[[74G[ OK ]^M
 * Loading hardware drivers... ^[[80G [ 8.046665] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0^M
[ 8.105642] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1^M
[ 8.128599] synaptics was reset on resume, see synaptics_resume_reset if you have trouble on resume^M
[ 8.330894] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input4^M
^M^[[74G[ OK ]^M
 * Loading kernel modules... ^[[80G * Loading manual drivers... ^[[80G ^M^[[74G[ OK ]^M
 * Setting kernel variables (/etc/sysctl.conf)... ^[[80G ^M^[[74G[ OK ]^M
 * Setting kernel variables (/etc/sysctl.d/10-console-messages.conf)... ^[[80G ^M^[[74G[ OK ]^M
 * Setting kernel variables (/etc/sysctl.d/10-network-security.conf)... ^[[80G ^M^[[74G[ OK ]^M
 * Activating swap... ^[[80G ^M^[[74G[ OK ]^M
 * Checking file systems... ^[[80G fsck 1.41.4 (27-Jan-2009)^M
^M^[[74G[ OK ]^M
 * Mounting local filesystems... ^[[80G [mntent]: warning: no final newline at the end of /etc/fstab^M
^M^[[74G[ OK ]^M
 * Activating swapfile swap... ^[[80G ^M^[[74G[ OK ]^M
 * Configuring network interfaces... ^[[80G ^M^[[74G[ OK ]^M
 * Setting up console font and keymap... ^[[80G ^M^[[74G[ OK ]^M
 * Starting system log daemon... ^[[80G chown: cannot access `/var/log/mail.warn': No such file or directory^M
chown: cannot access `/var/log/user.log': No such file or directory^M
chown: cannot access `/var/log/daemon.log': No such file or directory^M
chown: cannot access `/var/log/messages': No such file or directory^M
chown: cannot access `/var/log/auth.log': No such file or directory^M
chown: cannot access `/var/log/mail.err': No such file or directory^M
chown: cannot access `/var/log/syslog': No such file or directory^M
chown: cannot access `/var/log/mail.log': No such file or directory^M
chown: cannot access `/var/log/kern.log': No such file or directory^M
chown: cannot access `/var/log/lpr.log': No such file or directory^M
chown: cannot access `/var/log/mail.info': No such file or directory^M
^M^[[74G[ OK ]^M
 * Starting kernel log daemon... ^[[80G ^M^[[74G[ OK ]^M
 * Starting OpenBSD Secure Shell server sshd ^[[80G ^M^[[74G[ OK ]^M
chown: failed to get attributes of `/var/log/dmesg': No such file or directory^M
chmod: failed to get attributes of `/var/log/dmesg': No such file or directory^M
 * Restarting OpenBSD Secure Shell server sshd^M
   ...done.^M

Issues with the other managers is usually due to dhcp failing in some way. You definitely should not manually add br100 in the other two modes. Sometimes just a killall dnsmasq + try again is all you need. You may have some old data in the network table as well, so you may want to try deleting the db, and recreating your user, project and networks.

Vish

On Apr 1, 2011, at 11:45 AM, Mandeep Singh wrote:

> Question #151316 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/151316
>
> Status: Answered => Open
>
> Mandeep Singh is still having a problem:
> Thanks for the response Vish. I forgot to mention earlier that I tried
> FlatDHCP and FlatVLAN as well but the same thing happening... I think
> the instance is launching pretty nicely and I can see it saying inject
> key into the image in the logs. I have attached the console output of
> recently launched instance for your reference. Please do let me know if
> your want any other information to guide me through...
>
> ...
> ...
> ...
> Loading, please wait...^M
> Couldnt get a file descriptor referring to the console^M
> Begin: Loading essential drivers... ...^M
> Done.^M
> Begin: Running /scripts/init-premount ...^M
> Done.^M
> Begin: Mounting root file system... ...^M
> Begin: Running /scripts/local-top ...^M
> Done.^M
> Begin: Waiting for root file system... ...^M
> [ 2.090340] FDC 0 is a S82078B^M
> [ 2.117343] 8139cp: 10/100 PCI Ethernet driver v1.3 (Mar 22, 2004)^M
> [ 2.118439] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10^M
> [ 2.119228] 8139cp 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, high) -> IRQ 10^M
> [ 2.123227] eth0: RTL-8139C+ at 0xe0976000, 02:16:3e:0c:aa:da, IRQ 10^M
> [ 2.126156] 8139too Fast Ethernet driver 0.9.28^M
> [ 2.130848] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11^M
> [ 2.133447] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10^M
> [ 2.134222] virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10^M
> [ 2.481639] vda: unknown partition table^M
> Done.^M
> Begin: Running /scripts/local-premount ...^M
> Begin: Waiting for resume device... ...^M
> Done.^M
> Done.^M
> [ 7.657451] kjournald starting. Commit interval 5 seconds^M
> [ 7.658522] EXT3-fs: mounted filesystem with ordered data mode.^M
> Begin: Running /scripts/local-bottom ...^M
> Done.^M
> Done.^M
> Begin: Running /scripts/init-bottom ...^M
> Done.^M
> * Setting preliminary keymap... ^[[80G ^M^[[74G[ OK ]^M
> * Starting kernel event manager... ^[[80G [ 7.930645] udev: starting version 141^M
> ^M^[[74G[ OK ]^M
> * Loading hardware drivers... ^[[80G [ 8.046665] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0^M
> [ 8.105642] eth0: link up, 100Mbps, full-duplex, lpa 0x05E1^M
> [ 8.128599] synaptics was reset on resume, see synaptics_resume_reset if you have trouble on resume^M
> [ 8.330894] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input4^M
> ^M^[[74G[ OK ]^M
> * Loading kernel modules... ^[[80G * Loading manual drivers... ^[[80G ^M^[[74G[ OK ]^M
> * Setting kernel variables (/etc/sysctl.conf)... ^[[80G ^M^[[74G[ OK ]^M
> * Setting kernel variables (/etc/sysctl.d/10-console-messages.conf)... ^[[80G ^M^[[74G[ OK ]^M
> * Setting kernel variables (/etc/sysctl.d/10-network-security.conf)... ^[[80G ^M^[[74G[ OK ]^M
> * Activating swap... ^[[80G ^M^[[74G[ OK ]^M
> * Checking file systems... ^[[80G fsck 1.41.4 (27-Jan-2009)^M
> ^M^[[74G[ OK ]^M
> * Mounting local filesystems... ^[[80G [mntent]: warning: no final newline at the end of /etc/fstab^M
> ^M^[[74G[ OK ]^M
> * Activating swapfile swap... ^[[80G ^M^[[74G[ OK ]^M
> * Configuring network interfaces... ^[[80G ^M^[[74G[ OK ]^M
> * Setting up console font and keymap... ^[[80G ^M^[[74G[ OK ]^M
> * Starting system log daemon... ^[[80G chown: cannot access `/var/log/mail.warn': No such file or directory^M
> chown: cannot access `/var/log/user.log': No such file or directory^M
> chown: cannot access `/var/log/daemon.log': No such file or directory^M
> chown: cannot access `/var/log/messages': No such file or directory^M
> chown: cannot access `/var/log/auth.log': No such file or directory^M
> chown: cannot access `/var/log/mail.err': No such file or directory^M
> chown: cannot access `/var/log/syslog': No such file or directory^M
> chown: cannot access `/var/log/mail.log': No such file or directory^M
> chown: cannot access `/var/log/kern.log': No such file or directory^M
> chown: cannot access `/var/log/lpr.log': No such file or directory^M
> chown: cannot access `/var/log/mail.info': No such file or directory^M
> ^M^[[74G[ OK ]^M
> * Starting kernel log daemon... ^[[80G ^M^[[74G[ OK ]^M
> * Starting OpenBSD Secure Shell server sshd ^[[80G ^M^[[74G[ OK ]^M
> chown: failed to get attributes of `/var/log/dmesg': No such file or directory^M
> chmod: failed to get attributes of `/var/log/dmesg': No such file or directory^M
> * Restarting OpenBSD Secure Shell server sshd^M
> ...done.^M
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Hi Vish,
Thanks. I have tried FlatDHCP again on a fresh installation and this time I am able to access my instances. But the problem lies with my requirement to have a cluster of compute nodes and there are issues ( in FlatDHCP) for outbound access from inside instances.

Let me try the FlatNetwork again on a fresh setup... ( Question here: Is flat injection implemented for Redhat flavor instances which doesn't use /etc/network/interfaces for networking??)

Best
Mandeep Singh

Outbound access should work. If it doesn't, there is a bug.

FlatNetwork does not work with redhat, and requires a lot of manual setup.

Vish

On Apr 8, 2011, at 1:09 AM, Mandeep Singh wrote:

> Question #151316 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/151316
>
> Mandeep Singh posted a new comment:
> Hi Vish,
> Thanks. I have tried FlatDHCP again on a fresh installation and this time I am able to access my instances. But the problem lies with my requirement to have a cluster of compute nodes and there are issues ( in FlatDHCP) for outbound access from inside instances.
>
> Let me try the FlatNetwork again on a fresh setup... ( Question here: Is
> flat injection implemented for Redhat flavor instances which doesn't use
> /etc/network/interfaces for networking??)
>
> Best
> Mandeep Singh
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Actually, my test environment has two nodes as follows:

Node1 -> Running nova-api, nova-compute, nova-scheduler, nova-network, nova DB, rabbitMQ
Flat Interface: eth0
Public Interface: eth1

Node2 -> Running nova-compute & nova-network only
Flat Interface: eth0
Public Interface: eth1

The outbound access from instances launched on Node1 is perfect but the outbound access from instances on host Node2 is not working.

You only need to run nova-network on node1, not both. You probably have an issue where node2 has gotten an ip address on the bridge and/or has a bunch of incorrect iptables rules. On node2:
apt-get remove nova-network (or kill it if you are running from source)
iptables -F
iptables -t nat -F
remove any 10.x ips that may have gotten placed on br100
restart nova-compute

On Apr 8, 2011, at 2:49 AM, Mandeep Singh wrote:

> Question #151316 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/151316
>
> Mandeep Singh posted a new comment:
> Actually, my test environment has two nodes as follows:
>
> Node1 -> Running nova-api, nova-compute, nova-scheduler, nova-network, nova DB, rabbitMQ
> Flat Interface: eth0
> Public Interface: eth1
>
> Node2 -> Running nova-compute & nova-network only
> Flat Interface: eth0
> Public Interface: eth1
>
> The outbound access from instances launched on Node1 is perfect but the
> outbound access from instances on host Node2 is not working.
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

I have messed so much with the setup that I think its is now worth a nuke & rebuild. Anyways, I had some luck with VLAN manager with a cisco managed switch. Will post more after more tests.