After reboot the physical host, Quantum+OVS doesn't work!

Hi Havent,

thanks for reporting this problem.I don't see anything obviously wrong from
the data you posted.
I would probably check the firewall driver employed on the nova driver.
What might be happening is that it is blocking traffic on the ports used by
DHCP; this because of a limitation in the interaction between nova and
quantum that we are currently addressing. I am not personally following
this matter, but it seems that since Quantum is not returning to nova the
dhcp server address for a network, the firewall driver will be unable to
enable DHCP traffic.

If instead you're using the NoopFirewallDriver, then the issue would be
different and some tcpdump traces might help.

Regards,
Salvatore

On 17 July 2012 22:31, Havent <email address hidden> wrote:

> New question #203474 on quantum:
> https://answers.launchpad.net/quantum/+question/203474
>
> I have two physical hosts, and one of them is the
> controller+compute(nova-*, Quantum, OVS, domain name is cc401), the other
> is compute(nova-compute, Quantum, OVS, domain name is cc401). I can create
> network, create instances and lauch the instances.
> But when I restart the two hosts, I found that when I create instance, the
> instance cannot grap an IP address.
> These are some information below(eth2 on both of them used to be as OVS
> management interface):
> on cc401:
> # ifconfig
> eth0 Link encap:Ethernet HWaddr d4:ae:52:7e:aa:be
> inet addr:10.131.0.231 Bcast:10.131.255.255 Mask:255.252.0.0
> inet6 addr: fe80::d6ae:52ff:fe7e:aabe/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:136210 errors:0 dropped:0 overruns:0 frame:0
> TX packets:65821 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:26104719 (26.1 MB) TX bytes:30414927 (30.4 MB)
> Interrupt:36 Memory:d6000000-d6012800
>
> eth2 Link encap:Ethernet HWaddr d4:ae:52:7e:aa:c2
> inet6 addr: fe80::d6ae:52ff:fe7e:aac2/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:40467 errors:0 dropped:0 overruns:0 frame:0
> TX packets:688 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:2982466 (2.9 MB) TX bytes:225552 (225.5 KB)
> Interrupt:32 Memory:da000000-da012800
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:545895 errors:0 dropped:0 overruns:0 frame:0
> TX packets:545895 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:225216640 (225.2 MB) TX bytes:225216640 (225.2 MB)
>
> tap682c4721-60 Link encap:Ethernet HWaddr 2e:4a:c7:f1:fc:67
> inet6 addr: fe80::2c4a:c7ff:fef1:fc67/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:790 errors:0 dropped:0 overruns:0 frame:0
> TX packets:3406 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:500
> RX bytes:252916 (252.9 KB) TX bytes:246706 (246.7 KB)
>
> virbr0 Link encap:Ethernet HWaddr e2:ac:4f:bc:10:c2
> inet addr:192.168.122.1 Bcast:192.168.122.255
> Mask:255.255.255.0
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> # ps -ef | grep quantum
> root 9321 8657 0 13:22 pts/5 00:00:00 grep --color=auto quantum
> quantum 11732 1 0 11:13 ? 00:00:00 python
> /usr/sbin/quantum-server
> root 11757 11756 0 11:13 pts/1 00:00:49 python
> /usr/sbin/quantum-openvswitch-agent
> /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
>
> # nova-manage service list
> Binary Host Zone
> Status State Updated_At
> nova-scheduler cc401 nova
> enabled :-) 2012-07-18 05:22:22
> nova-cert cc401 nova
> enabled :-) 2012-07-18 05:22:22
> nova-consoleauth cc401 nova
> enabled :-) 2012-07-18 05:22:21
> nova-network cc401 nova
> enabled :-) 2012-07-18 05:22:16
> nova-compute cc401 nova
> enabled :-) 2012-07-18 05:22:23
> nova-compute cc402 nova
> enabled :-) 2012-07-18 05:22:15
>
> # ps -ef | grep dnsmasq
> 105 2231 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq -u
> libvirt-dnsmasq --strict-order --bind-interfaces
> --pid-file=/var/run/libvirt/network/default.pid --conf-file=
> --except-interface lo --listen-address 192.168.122.1 --dhcp-range
> 192.168.122.2,192.168.122.254
> --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases
> --dhcp-lease-max=253 --dhcp-no-override
> nobody 2748 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq
> --strict-order --bind-interfaces --conf-file= --domain=novalocal
> --pid-file=/var/lib/nova/networks/nova-gw-b0deb0ca-b2.pid
> --listen-address=192.168.102.1 --except-interface=lo
> --dhcp-range=192.168.102.2,static,120s --dhcp-lease-max=256
> --dhcp-hostsfile=/var/lib/nova/networks/nova-gw-b0deb0ca-b2.conf
> --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
> root 2749 2748 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq
> --strict-order --bind-interfaces --conf-file= --domain=novalocal
> --pid-file=/var/lib/nova/networks/nova-gw-b0deb0ca-b2.pid
> --listen-address=192.168.102.1 --except-interface=lo
> --dhcp-range=192.168.102.2,static,120s --dhcp-lease-max=256
> --dhcp-hostsfile=/var/lib/nova/networks/nova-gw-b0deb0ca-b2.conf
> --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
> nobody 2833 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq
> --strict-order --bind-interfaces --conf-file= --domain=novalocal
> --pid-file=/var/lib/nova/networks/nova-gw-8cde0e95-1e.pid
> --listen-address=192.168.103.1 --except-interface=lo
> --dhcp-range=192.168.103.2,static,120s --dhcp-lease-max=256
> --dhcp-hostsfile=/var/lib/nova/networks/nova-gw-8cde0e95-1e.conf
> --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
> root 2834 2833 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq
> --strict-order --bind-interfaces --conf-file= --domain=novalocal
> --pid-file=/var/lib/nova/networks/nova-gw-8cde0e95-1e.pid
> --listen-address=192.168.103.1 --except-interface=lo
> --dhcp-range=192.168.103.2,static,120s --dhcp-lease-max=256
> --dhcp-hostsfile=/var/lib/nova/networks/nova-gw-8cde0e95-1e.conf
> --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
> nobody 2918 1 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq
> --strict-order --bind-interfaces --conf-file= --domain=novalocal
> --pid-file=/var/lib/nova/networks/nova-gw-a24f3507-98.pid
> --listen-address=192.168.104.1 --except-interface=lo
> --dhcp-range=192.168.104.2,static,120s --dhcp-lease-max=256
> --dhcp-hostsfile=/var/lib/nova/networks/nova-gw-a24f3507-98.conf
> --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
> root 2919 2918 0 08:59 ? 00:00:00 /usr/sbin/dnsmasq
> --strict-order --bind-interfaces --conf-file= --domain=novalocal
> --pid-file=/var/lib/nova/networks/nova-gw-a24f3507-98.pid
> --listen-address=192.168.104.1 --except-interface=lo
> --dhcp-range=192.168.104.2,static,120s --dhcp-lease-max=256
> --dhcp-hostsfile=/var/lib/nova/networks/nova-gw-a24f3507-98.conf
> --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro
> root 10109 8657 0 13:23 pts/5 00:00:00 grep --color=auto dnsmasq
>
> # ovs-vsctl show
> 73268980-56ac-4980-9e88-16d5530584e7
> Bridge br-int
> Port "tap682c4721-60"
> tag: 2
> Interface "tap682c4721-60"
> Port "gw-8cde0e95-1e"
> tag: 2
> Interface "gw-8cde0e95-1e"
> type: internal
> Port "gw-a24f3507-98"
> tag: 3
> Interface "gw-a24f3507-98"
> type: internal
> Port "eth2"
> Interface "eth2"
> Port "gw-b0deb0ca-b2"
> tag: 1
> Interface "gw-b0deb0ca-b2"
> type: internal
> Port br-int
> Interface br-int
> type: internal
> ovs_version: "1.4.0+build0"
>
>
> on cc402:
> # ifconfig
> eth0 Link encap:Ethernet HWaddr d4:ae:52:7e:ab:54
> inet addr:10.131.0.193 Bcast:10.131.255.255 Mask:255.252.0.0
> inet6 addr: fe80::d6ae:52ff:fe7e:ab54/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:119258 errors:0 dropped:0 overruns:0 frame:0
> TX packets:81017 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:33613309 (33.6 MB) TX bytes:22515775 (22.5 MB)
> Interrupt:36 Memory:d6000000-d6012800
>
> eth2 Link encap:Ethernet HWaddr d4:ae:52:7e:ab:58
> inet6 addr: fe80::d6ae:52ff:fe7e:ab58/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:7741 errors:0 dropped:0 overruns:0 frame:0
> TX packets:133 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:598197 (598.1 KB) TX bytes:42402 (42.4 KB)
> Interrupt:32 Memory:da000000-da012800
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:2920 errors:0 dropped:0 overruns:0 frame:0
> TX packets:2920 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:418527 (418.5 KB) TX bytes:418527 (418.5 KB)
>
> tap27b1f8e8-87 Link encap:Ethernet HWaddr a2:ea:e7:82:23:40
> inet6 addr: fe80::a0ea:e7ff:fe82:2340/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:799 errors:0 dropped:0 overruns:0 frame:0
> TX packets:752 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:500
> RX bytes:255814 (255.8 KB) TX bytes:80720 (80.7 KB)
>
> virbr0 Link encap:Ethernet HWaddr 76:be:8a:a5:39:b2
> inet addr:192.168.122.1 Bcast:192.168.122.255
> Mask:255.255.255.0
> UP BROADCAST MULTICAST MTU:1500 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> # ovs-vsctl show
> a5dc8ed9-ec08-421e-b992-d5968b09836f
> Bridge br-int
> Port "eth2"
> Interface "eth2"
> Port br-int
> Interface br-int
> type: internal
> Port "tap27b1f8e8-87"
> tag: 2
> Interface "tap27b1f8e8-87"
> ovs_version: "1.4.0+build0"
>
> # ps -ef | grep quantum
> quantum 10001 1 0 11:13 ? 00:00:00 python
> /usr/sbin/quantum-server
> root 10008 10007 0 11:13 pts/0 00:00:30 python
> /usr/sbin/quantum-openvswitch-agent
> /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini
> root 29491 27902 0 13:25 pts/3 00:00:00 grep --color=auto quantum
>
>
> the tcpdump result shows that the DHCP resquests don't receive replies.
>
>
> --
> You received this question notification because you are a member of
> Netstack Core Developers, which is an answer contact for quantum.
>

Hi Salvatore,
    Thanks for your reply!
    I change my nova.conf configuration on item as below:
--firewall_driver=nova.virt.firewall.NoopFirewallDriver
both cc401 and cc402 change.
I restart all the relative services on both cc401 and cc402.
But the former instances still don't grab IP addresses.
And the tcpdump result also shows the DHCP requests don't receive replies.

Now I give the nova.conf configuration on my two hosts:
cc401:
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--libvirt_use_virtio_for_bridges
--root_helper=sudo nova-rootwrap
--ec2_private_dns_show_ip
--log_config=/etc/nova/logging.conf

# LOGS/STATE
--verbose=False

# AUTHENTICATION
--auth_strategy=keystone

# SCHEDULER
#--compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler

# VOLUMES
--volume_group=nova-volumes
--volume_name_template=volume-%08x
--iscsi_helper=tgtadm

# DATABASE
--sql_connection=mysql://nova:admin@10.131.0.231/nova

# COMPUTE
--libvirt_type=kvm
--connection_type=libvirt
--instance_name_template=instance-%08x
--api_paste_config=/etc/nova/api-paste.ini
#--allow_resize_to_same_host=True

# APIS
#--osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
#--ec2_dmz_host=192.168.206.130
#--s3_host=192.168.206.130

# RABBITMQ
--rabbit_host=10.131.0.231
--rabbit_password=guest
--rabbit_port=5672
--rabbit_userid=guest
--rabbit_virtual_host=/

# GLANCE
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=10.131.0.231:9292
--glance_host=10.131.0.231

# NETWORK-FLATDHCP
#--network_manager=nova.network.manager.FlatDHCPManager
#--flat_network_dhcp_start=10.132.0.2
#--flat_interface=eth2
#--fixed_range=10.132.0.0/24
#--floating_range=10.129.0.32/28
#--flat_network_bridge=br100

# NETWORK-VLAN
#--network_manager=nova.network.manager.VlanManager
#--vlan_interface=eth2

# NETWORK_FOR_QUANTUM
--network_manager=nova.network.quantum.manager.QuantumManager
--linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
--quantum_use_dhcp=True
--linuxnet_ovs_integration_bridge=br-int
--libvirt_ovs_bridge=br-int
--libvirt_vif_type=ethernet
--libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver

# NETWORK-COMMON
--network_host=10.131.0.231
--force_dhcp_release=True
#--firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
--firewall_driver=nova.virt.firewall.NoopFirewallDriver
--public_interface=eth0
#--multi_host=true
--routing_source_ip=0.0.0.0

# NOVNC CONSOLE
--vnc_enabled=true
--novncproxy_base_url=http://10.131.0.231:6080/vnc_auto.html
--novncproxy_port=6080
--novncproxy_host=10.131.0.231
#--vncserver_listen=10.131.0.231
#--vncserver_proxyclient_address=10.131.0.231
#IP address; Cloud controller host IP address
--my_ip=10.131.0.231

cc402:
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--libvirt_use_virtio_for_bridges
--root_helper=sudo nova-rootwrap
--ec2_private_dns_show_ip
--log_config=/etc/nova/logging.conf

# LOGS/STATE
--verbose=False

# AUTHENTICATION
--auth_strategy=keystone

# SCHEDULER
#--compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler

# VOLUMES
--volume_group=nova-volumes
--volume_name_template=volume-%08x
--iscsi_helper=tgtadm

# DATABASE
--sql_connection=mysql://nova:admin@10.131.0.231/nova

# COMPUTE
--libvirt_type=kvm
--connection_type=libvirt
--instance_name_template=instance-%08x
--api_paste_config=/etc/nova/api-paste.ini
#--allow_resize_to_same_host=True

# APIS
#--osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
#--ec2_dmz_host=192.168.206.130
#--s3_host=192.168.206.130

# RABBITMQ
--rabbit_host=10.131.0.231
--rabbit_password=guest
--rabbit_port=5672
--rabbit_userid=guest
--rabbit_virtual_host=/

# GLANCE
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=10.131.0.231:9292
--glance_host=10.131.0.231

# NETWORK-FLATDHCP
#--network_manager=nova.network.manager.FlatDHCPManager
#--flat_network_dhcp_start=10.132.0.2
#--flat_interface=eth2
#--fixed_range=10.132.0.0/24
#--floating_range=10.129.0.32/28
#--flat_network_bridge=br100

# NETWORK-VLAN
#--network_manager=nova.network.manager.VlanManager
#--vlan_interface=eth2

# NETWORK_FOR_QUANTUM
--network_manager=nova.network.quantum.manager.QuantumManager
--linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
--quantum_use_dhcp=True
--linuxnet_ovs_integration_bridge=br-int
--libvirt_ovs_bridge=br-int
--libvirt_vif_type=ethernet
--libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtOpenVswitchDriver

# NETWORK-COMMON
--network_host=10.131.0.231
--force_dhcp_release=True
#--firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
--firewall_driver=nova.virt.firewall.NoopFirewallDriver
--public_interface=eth0
#--multi_host=true

--routing_source_ip=0.0.0.0
# NOVNC CONSOLE
--vnc_enabled=true
--novncproxy_base_url=http://10.131.0.193:6080/vnc_auto.html
--novncproxy_port=6080
--novncproxy_host=10.131.0.193
#--vncserver_listen=10.131.0.193
#--vncserver_proxyclient_address=10.131.0.193
#IP address; Cloud controller host IP address
--my_ip=10.131.0.193

and the tcpdump results are as below:
10:05:14.465306 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 308)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:02:3c:39 (oui Unknown), length 280, xid 0x6871df02, Flags [none]
   Client-Ethernet-Address fa:16:3e:02:3c:39 (oui Unknown)
   Vendor-rfc1048 Extensions
     Magic Cookie 0x63825363
     DHCP-Message Option 53, length 1: Discover
     Client-ID Option 61, length 7: ether fa:16:3e:02:3c:39
     Vendor-Class Option 60, length 12: "udhcp 1.17.2"
     MSZ Option 57, length 2: 576
     Parameter-Request Option 55, length 7:
       Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
       Domain-Name, BR, NTP
10:05:14.831855 STP 802.1d, Config, Flags [none], bridge-id 8009.24:b6:57:f3:56:00.8007, length 42
 message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
 root-id 8009.24:b6:57:f3:56:00, root-pathcost 0

cc402:
10:05:10.161568 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 308)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:02:3c:39 (oui Unknown), length 280, xid 0x6871df02, Flags [none]
   Client-Ethernet-Address fa:16:3e:02:3c:39 (oui Unknown)
   Vendor-rfc1048 Extensions
     Magic Cookie 0x63825363
     DHCP-Message Option 53, length 1: Discover
     Client-ID Option 61, length 7: ether fa:16:3e:02:3c:39
     Vendor-Class Option 60, length 12: "udhcp 1.17.2"
     MSZ Option 57, length 2: 576
     Parameter-Request Option 55, length 7:
       Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
       Domain-Name, BR, NTP
10:05:10.208839 STP 802.1d, Config, Flags [none], bridge-id 8003.24:b6:57:f3:56:00.8009, length 42
 message-age 0.00s, max-age 20.00s, hello-time 2.00s, forwarding-delay 15.00s
 root-id 8003.24:b6:57:f3:56:00, root-pathcost 0

and I try to tcpdump the eth2,,tap*, br-int on cc402:
the results are:
eth2:
11:11:54.390015 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 308)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:66:74:94 (oui Unknown), length 280, xid 0x9e614554, Flags [none]
   Client-Ethernet-Address fa:16:3e:66:74:94 (oui Unknown)
   Vendor-rfc1048 Extensions
     Magic Cookie 0x63825363
     DHCP-Message Option 53, length 1: Discover
     Client-ID Option 61, length 7: ether fa:16:3e:66:74:94
     Vendor-Class Option 60, length 12: "udhcp 1.17.2"
     MSZ Option 57, length 2: 576
     Parameter-Request Option 55, length 7:
       Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
       Domain-Name, BR, NTP
11:11:54.390433 IP (tos 0x0, ttl 64, id 41020, offset 0, flags [none], proto UDP (17), length 347)
    0.0.0.0.1 > 192.168.103.9.bootpc: BOOTP/DHCP, Reply, length 319, xid 0x9e614554, Flags [none]
   Your-IP 192.168.103.9
   Server-IP 192.168.103.1
   Client-Ethernet-Address fa:16:3e:66:74:94 (oui Unknown)
   Vendor-rfc1048 Extensions
     Magic Cookie 0x63825363
     DHCP-Message Option 53, length 1: Offer
     Server-ID Option 54, length 4: 192.168.103.1
     Lease-Time Option 51, length 4: 120
     RN Option 58, length 4: 60
     RB Option 59, length 4: 105
     Subnet-Mask Option 1, length 4: 255.255.255.0
     BR Option 28, length 4: 192.168.103.255
     Default-Gateway Option 3, length 4: 192.168.103.1
     Domain-Name-Server Option 6, length 4: 192.168.103.1
     Domain-Name Option 15, length 9: "novalocal"
     Hostname Option 12, length 14: "nova-quantum-2"

tap:
11:12:49.450979 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 308)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from fa:16:3e:66:74:94 (oui Unknown), length 280, xid 0x8ceaf775, Flags [none]
   Client-Ethernet-Address fa:16:3e:66:74:94 (oui Unknown)
   Vendor-rfc1048 Extensions
     Magic Cookie 0x63825363
     DHCP-Message Option 53, length 1: Discover
     Client-ID Option 61, length 7: ether fa:16:3e:66:74:94
     Vendor-Class Option 60, length 12: "udhcp 1.17.2"
     MSZ Option 57, length 2: 576
     Parameter-Request Option 55, length 7:
       Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
       Domain-Name, BR, NTP
11:12:49.451467 IP (tos 0x0, ttl 64, id 41025, offset 0, flags [none], proto UDP (17), length 347)
    0.0.0.0.1 > 192.168.103.9.bootpc: BOOTP/DHCP, Reply, length 319, xid 0x8ceaf775, Flags [none]
   Your-IP 192.168.103.9
   Server-IP 192.168.103.1
   Client-Ethernet-Address fa:16:3e:66:74:94 (oui Unknown)
   Vendor-rfc1048 Extensions
     Magic Cookie 0x63825363
     DHCP-Message Option 53, length 1: Offer
     Server-ID Option 54, length 4: 192.168.103.1
     Lease-Time Option 51, length 4: 120
     RN Option 58, length 4: 60
     RB Option 59, length 4: 105
     Subnet-Mask Option 1, length 4: 255.255.255.0
     BR Option 28, length 4: 192.168.103.255
     Default-Gateway Option 3, length 4: 192.168.103.1
     Domain-Name-Server Option 6, length 4: 192.168.103.1
     Domain-Name Option 15, length 9: "novalocal"
     Hostname Option 12, length 14: "nova-quantum-2"
11:12:49.591797 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length

br-int:
DHCP requests have no reply.

But when I log on the VM with novnc tool, it has no IP address.
I don't know what happened.

IP tables on cc401:
# iptables-save
# Generated by iptables-save v1.4.12 on Fri Jul 20 13:01:48 2012
*mangle
:PREROUTING ACCEPT [643534:256698344]
:INPUT ACCEPT [642447:256651876]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [612822:270480299]
:POSTROUTING ACCEPT [612467:270357762]
-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Fri Jul 20 13:01:48 2012
# Generated by iptables-save v1.4.12 on Fri Jul 20 13:01:48 2012
*nat
:PREROUTING ACCEPT [619:45960]
:INPUT ACCEPT [176:31562]
:OUTPUT ACCEPT [6405:384802]
:POSTROUTING ACCEPT [6226:373560]
:nova-api-OUTPUT - [0:0]
:nova-api-POSTROUTING - [0:0]
:nova-api-PREROUTING - [0:0]
:nova-api-float-snat - [0:0]
:nova-api-snat - [0:0]
:nova-compute-OUTPUT - [0:0]
:nova-compute-POSTROUTING - [0:0]
:nova-compute-PREROUTING - [0:0]
:nova-compute-float-snat - [0:0]
:nova-compute-snat - [0:0]
:nova-network-OUTPUT - [0:0]
:nova-network-POSTROUTING - [0:0]
:nova-network-PREROUTING - [0:0]
:nova-network-float-snat - [0:0]
:nova-network-snat - [0:0]
:nova-postrouting-bottom - [0:0]
-A PREROUTING -j nova-network-PREROUTING
-A PREROUTING -j nova-api-PREROUTING
-A PREROUTING -j nova-compute-PREROUTING
-A OUTPUT -j nova-network-OUTPUT
-A OUTPUT -j nova-api-OUTPUT
-A OUTPUT -j nova-compute-OUTPUT
-A POSTROUTING -j nova-network-POSTROUTING
-A POSTROUTING -j nova-api-POSTROUTING
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
-A POSTROUTING -j nova-compute-POSTROUTING
-A POSTROUTING -j nova-postrouting-bottom
-A nova-api-snat -j nova-api-float-snat
-A nova-compute-snat -j nova-compute-float-snat
-A nova-network-POSTROUTING -s 10.0.0.0/8 -d 10.131.0.231/32 -j ACCEPT
-A nova-network-POSTROUTING -s 10.0.0.0/8 -d 10.128.0.0/24 -j ACCEPT
-A nova-network-POSTROUTING -s 10.0.0.0/8 -d 10.0.0.0/8 -m conntrack ! --ctstate DNAT -j ACCEPT
-A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.131.0.231:8775
-A nova-network-snat -j nova-network-float-snat
-A nova-network-snat -s 10.0.0.0/8 -j SNAT --to-source 0.0.0.0
-A nova-network-snat -s 192.168.102.0/24 -j SNAT --to-source 0.0.0.0
-A nova-network-snat -s 192.168.103.0/24 -j SNAT --to-source 0.0.0.0
-A nova-network-snat -s 192.168.104.0/24 -j SNAT --to-source 0.0.0.0
-A nova-postrouting-bottom -j nova-network-snat
-A nova-postrouting-bottom -j nova-api-snat
-A nova-postrouting-bottom -j nova-compute-snat
COMMIT
# Completed on Fri Jul 20 13:01:48 2012
# Generated by iptables-save v1.4.12 on Fri Jul 20 13:01:48 2012
*filter
:INPUT ACCEPT [196516:67986032]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [187116:72940121]
:nova-api-FORWARD - [0:0]
:nova-api-INPUT - [0:0]
:nova-api-OUTPUT - [0:0]
:nova-api-local - [0:0]
:nova-compute-FORWARD - [0:0]
:nova-compute-INPUT - [0:0]
:nova-compute-OUTPUT - [0:0]
:nova-compute-inst-14 - [0:0]
:nova-compute-local - [0:0]
:nova-compute-provider - [0:0]
:nova-compute-sg-fallback - [0:0]
:nova-filter-top - [0:0]
:nova-network-FORWARD - [0:0]
:nova-network-INPUT - [0:0]
:nova-network-OUTPUT - [0:0]
:nova-network-local - [0:0]
-A INPUT -j nova-network-INPUT
-A INPUT -j nova-api-INPUT
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A INPUT -j nova-compute-INPUT
-A INPUT -p gre -j ACCEPT
-A FORWARD -j nova-filter-top
-A FORWARD -j nova-network-FORWARD
-A FORWARD -j nova-api-FORWARD
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -j nova-compute-FORWARD
-A OUTPUT -j nova-filter-top
-A OUTPUT -j nova-network-OUTPUT
-A OUTPUT -j nova-api-OUTPUT
-A OUTPUT -j nova-compute-OUTPUT
-A nova-api-INPUT -d 10.131.0.231/32 -p tcp -m tcp --dport 8775 -j ACCEPT
-A nova-compute-inst-14 -m state --state INVALID -j DROP
-A nova-compute-inst-14 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A nova-compute-inst-14 -j nova-compute-provider
-A nova-compute-inst-14 -s 192.168.103.1/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A nova-compute-inst-14 -s 192.168.103.0/24 -j ACCEPT
-A nova-compute-inst-14 -p icmp -j ACCEPT
-A nova-compute-inst-14 -p tcp -m tcp --dport 22 -j ACCEPT
-A nova-compute-inst-14 -j nova-compute-sg-fallback
-A nova-compute-local -d 192.168.103.8/32 -j nova-compute-inst-14
-A nova-compute-sg-fallback -j DROP
-A nova-filter-top -j nova-network-local
-A nova-filter-top -j nova-api-local
-A nova-filter-top -j nova-compute-local
-A nova-network-INPUT -i gw-8cde0e95-1e -p udp -m udp --dport 67 -j ACCEPT
-A nova-network-INPUT -i gw-8cde0e95-1e -p tcp -m tcp --dport 67 -j ACCEPT
-A nova-network-INPUT -i gw-8cde0e95-1e -p udp -m udp --dport 53 -j ACCEPT
-A nova-network-INPUT -i gw-8cde0e95-1e -p tcp -m tcp --dport 53 -j ACCEPT
COMMIT
# Completed on Fri Jul 20 13:01:48 2012

IP tables on cc402:
# iptables-save
# Generated by iptables-save v1.4.12 on Fri Jul 20 13:02:32 2012
*nat
:PREROUTING ACCEPT [1673:175172]
:INPUT ACCEPT [597:129056]
:OUTPUT ACCEPT [195:13587]
:POSTROUTING ACCEPT [195:13587]
:nova-compute-OUTPUT - [0:0]
:nova-compute-POSTROUTING - [0:0]
:nova-compute-PREROUTING - [0:0]
:nova-compute-float-snat - [0:0]
:nova-compute-snat - [0:0]
:nova-postrouting-bottom - [0:0]
-A PREROUTING -j nova-compute-PREROUTING
-A OUTPUT -j nova-compute-OUTPUT
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
-A POSTROUTING -j nova-compute-POSTROUTING
-A POSTROUTING -j nova-postrouting-bottom
-A nova-compute-snat -j nova-compute-float-snat
-A nova-postrouting-bottom -j nova-compute-snat
COMMIT
# Completed on Fri Jul 20 13:02:32 2012
# Generated by iptables-save v1.4.12 on Fri Jul 20 13:02:32 2012
*mangle
:PREROUTING ACCEPT [79018:39845446]
:INPUT ACCEPT [77942:39799330]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [107292:30521250]
:POSTROUTING ACCEPT [107292:30521250]
-A POSTROUTING -o virbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Fri Jul 20 13:02:32 2012
# Generated by iptables-save v1.4.12 on Fri Jul 20 13:02:32 2012
*filter
:INPUT ACCEPT [77947:39799590]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [107293:30521990]
:nova-compute-FORWARD - [0:0]
:nova-compute-INPUT - [0:0]
:nova-compute-OUTPUT - [0:0]
:nova-compute-inst-15 - [0:0]
:nova-compute-local - [0:0]
:nova-compute-provider - [0:0]
:nova-compute-sg-fallback - [0:0]
:nova-filter-top - [0:0]
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A INPUT -j nova-compute-INPUT
-A INPUT -p gre -j ACCEPT
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -j nova-filter-top
-A FORWARD -j nova-compute-FORWARD
-A OUTPUT -j nova-filter-top
-A OUTPUT -j nova-compute-OUTPUT
-A nova-compute-inst-15 -m state --state INVALID -j DROP
-A nova-compute-inst-15 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A nova-compute-inst-15 -j nova-compute-provider
-A nova-compute-inst-15 -s 192.168.103.1/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A nova-compute-inst-15 -s 192.168.103.0/24 -j ACCEPT
-A nova-compute-inst-15 -p icmp -j ACCEPT
-A nova-compute-inst-15 -p tcp -m tcp --dport 22 -j ACCEPT
-A nova-compute-inst-15 -j nova-compute-sg-fallback
-A nova-compute-local -d 192.168.103.9/32 -j nova-compute-inst-15
-A nova-compute-sg-fallback -j DROP
-A nova-filter-top -j nova-compute-local
COMMIT
# Completed on Fri Jul 20 13:02:32 2012

This question was expired because it remained in the 'Open' state without activity for the last 15 days.