OpenStack Compute (nova)

Instance on the pure compute node cannot reach the meta data server

Asked by Hyunsun Moon

I deployed nova to the two servers, one runs api, network, scheduler, objectstore, and the other runs compute only.
Instance has been proceed to the running state successfully, however, console output shows that the instance failed to start up unable to reach metadata server.
See the console output below.

----------------------------------------------------------------------------------------

1 [ 0.000000] Initializing cgroup subsys cpuset^M
  2 [ 0.000000] Initializing cgroup subsys cpu^M
  3 [ 0.000000] Linux version 2.6.32-28-server (buildd@allspice) (gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) ) #55-Ubuntu SMP Mon Jan 10 23: 57:16 UTC 2011 (Ubuntu 2.6.32-28.55-server 2.6.32.27+drm33.12)^M
  4 [ 0.000000] Command line: root=/dev/vda console=ttyS0^M
  5 [ 0.000000] KERNEL supported cpus:^M
  6 [ 0.000000] Intel GenuineIntel^M
  7 [ 0.000000] AMD AuthenticAMD^M
  8 [ 0.000000] Centaur CentaurHauls^M
  9 [ 0.000000] BIOS-provided physical RAM map:^M
 10 [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009f000 (usable)^M
 11 [ 0.000000] BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)^M
 12 [ 0.000000] BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)^M
 13 [ 0.000000] BIOS-e820: 0000000000100000 - 000000001fffb000 (usable)^M
 14 [ 0.000000] BIOS-e820: 000000001fffb000 - 0000000020000000 (reserved)^M
 15 [ 0.000000] BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)^M
 16 [ 0.000000] DMI 2.4 present.^M
 17 [ 0.000000] last_pfn = 0x1fffb max_arch_pfn = 0x400000000^M
 18 [ 0.000000] x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106^M
 19 [ 0.000000] Scanning 1 areas for low memory corruption^M
 20 [ 0.000000] modified physical RAM map:^M
 21 [ 0.000000] modified: 0000000000000000 - 0000000000001000 (usable)^M
 22 [ 0.000000] modified: 0000000000001000 - 0000000000006000 (reserved)^M
 23 [ 0.000000] modified: 0000000000006000 - 000000000009f000 (usable)^M
 24 [ 0.000000] modified: 000000000009f000 - 00000000000a0000 (reserved)^M
 25 [ 0.000000] modified: 00000000000f0000 - 0000000000100000 (reserved)^M
 26 [ 0.000000] modified: 0000000000100000 - 000000001fffb000 (usable)^M
 27 [ 0.000000] modified: 000000001fffb000 - 0000000020000000 (reserved)^M
 28 [ 0.000000] modified: 00000000fffc0000 - 0000000100000000 (reserved)^M
 29 [ 0.000000] init_memory_mapping: 0000000000000000-000000001fffb000^M
 30 [ 0.000000] NX (Execute Disable) protection: active^M
 31 [ 0.000000] ACPI: RSDP 00000000000f8940 00014 (v00 BOCHS )^M
 32 [ 0.000000] ACPI: RSDT 000000001fffde30 00034 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)^M
 33 [ 0.000000] ACPI: FACP 000000001ffffe70 00074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)^M
 34 [ 0.000000] ACPI: DSDT 000000001fffdfd0 01E22 (v01 BXPC BXDSDT 00000001 INTL 20090123)^M
 35 [ 0.000000] ACPI: FACS 000000001ffffe00 00040^M
 36 [ 0.000000] ACPI: SSDT 000000001fffdf90 00037 (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)^M
 37 [ 0.000000] ACPI: APIC 000000001fffdeb0 00072 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)^M
 38 [ 0.000000] ACPI: HPET 000000001fffde70 00038 (v01 BOCHS BXPCHPET 00000001 BXPC 00000001)^M
 39 [ 0.000000] No NUMA configuration found^M
 40 [ 0.000000] Faking a node at 0000000000000000-000000001fffb000^M
 41 [ 0.000000] Bootmem setup node 0 0000000000000000-000000001fffb000^M
 42 [ 0.000000] NODE_DATA [0000000000009000 - 000000000000dfff]^M

 43 [ 0.000000] bootmap [000000000000e000 - 0000000000011fff] pages 4^M
 44 [ 0.000000] (6 early reservations) ==> bootmem [0000000000 - 001fffb000]^M
 45 [ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]^M
 46 [ 0.000000] #1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000]^M
 47 [ 0.000000] #2 [0001000000 - 0001a4faa4] TEXT DATA BSS ==> [0001000000 - 0001a4faa4]^M
 48 [ 0.000000] #3 [000009f000 - 0000100000] BIOS reserved ==> [000009f000 - 0000100000]^M
 49 [ 0.000000] #4 [0001a50000 - 0001a50071] BRK ==> [0001a50000 - 0001a50071]^M
 50 [ 0.000000] #5 [0000008000 - 0000009000] PGTABLE ==> [0000008000 - 0000009000]^M
 51 [ 0.000000] found SMP MP-table at [ffff8800000f8990] f8990^M
 52 [ 0.000000] Zone PFN ranges:^M
 53 [ 0.000000] DMA 0x00000000 -> 0x00001000^M
 54 [ 0.000000] DMA32 0x00001000 -> 0x00100000^M
 55 [ 0.000000] Normal 0x00100000 -> 0x00100000^M
 56 [ 0.000000] Movable zone start PFN for each node^M
 57 [ 0.000000] early_node_map[3] active PFN ranges^M
 58 [ 0.000000] 0: 0x00000000 -> 0x00000001^M
 59 [ 0.000000] 0: 0x00000006 -> 0x0000009f^M
 60 [ 0.000000] 0: 0x00000100 -> 0x0001fffb^M
 61 [ 0.000000] ACPI: PM-Timer IO Port: 0xb008^M
 62 [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)^M
 63 [ 0.000000] ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])^M
 64 [ 0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23^M
 65 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)^M
 66 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)^M
 67 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)^M
 68 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)^M
 69 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)^M
 70 [ 0.000000] Using ACPI (MADT) for SMP configuration information^M
 71 [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000^M
 72 [ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs^M
 73 [ 0.000000] PM: Registered nosave memory: 0000000000001000 - 0000000000006000^M
 74 [ 0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000^M
 75 [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000^M
 76 [ 0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000^M
 77 [ 0.000000] Allocating PCI resources starting at 20000000 (gap: 20000000:dffc0000)^M
 78 [ 0.000000] Booting paravirtualized kernel on bare hardware^M
 79 [ 0.000000] NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:1 nr_node_ids:1^M
 80 [ 0.000000] PERCPU: Embedded 30 pages/cpu @ffff880001c00000 s91544 r8192 d23144 u2097152^M
 81 [ 0.000000] pcpu-alloc: s91544 r8192 d23144 u2097152 alloc=1*2097152^M
 82 [ 0.000000] pcpu-alloc: [0] 0 ^M
 83 [ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 129072^M
 84 [ 0.000000] Policy zone: DMA32^M
 85 [ 0.000000] Kernel command line: root=/dev/vda console=ttyS0^M

 86 [ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)^M
 87 [ 0.000000] Initializing CPU#0^M
 88 [ 0.000000] Checking aperture...^M
 89 [ 0.000000] No AGP bridge found^M
 90 [ 0.000000] Memory: 504688k/524268k available (5515k kernel code, 408k absent, 19172k reserved, 3090k data, 800k init)^M
 91 [ 0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1^M
 92 [ 0.000000] Hierarchical RCU implementation.^M
 93 [ 0.000000] NR_IRQS:4352 nr_irqs:256^M
 94 [ 0.000000] Console: colour VGA+ 80x25^M
 95 [ 0.000000] console [ttyS0] enabled^M
 96 [ 0.000000] allocated 5242880 bytes of page_cgroup^M
 97 [ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups^M
 98 [ 0.000000] HPET: 3 timers in total, 0 timers will be used for per-cpu timer^M
 99 [ 0.000000] Fast TSC calibration using PIT^M
100 [ 0.000000] Detected 2991.717 MHz processor.^M
101 [ 0.020495] Calibrating delay loop (skipped), value calculated using timer frequency.. 5983.43 BogoMIPS (lpj=29917170)^M
102 [ 0.021782] Security Framework initialized^M
103 [ 0.023549] AppArmor: AppArmor initialized^M
104 [ 0.024865] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)^M
105 [ 0.026802] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)^M
106 [ 0.027786] Mount-cache hash table entries: 256^M
107 [ 0.034375] Initializing cgroup subsys ns^M
108 [ 0.034787] Initializing cgroup subsys cpuacct^M
109 [ 0.035046] Initializing cgroup subsys memory^M
110 [ 0.035662] Initializing cgroup subsys devices^M
111 [ 0.035908] Initializing cgroup subsys freezer^M
112 [ 0.036122] Initializing cgroup subsys net_cls^M
113 [ 0.037178] CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)^M
114 [ 0.037500] CPU: L2 Cache: 512K (64 bytes/line)^M
115 [ 0.037800] CPU 0/0x0 -> Node 0^M
116 [ 0.038156] mce: CPU supports 10 MCE banks^M
117 [ 0.038987] Performance Events: AMD PMU driver.^M
118 [ 0.039379] ... version: 0^M
119 [ 0.039558] ... bit width: 48^M
120 [ 0.039737] ... generic registers: 4^M
121 [ 0.039954] ... value mask: 0000ffffffffffff^M
122 [ 0.040066] ... max period: 00007fffffffffff^M
123 [ 0.040292] ... fixed-purpose events: 0^M
124 [ 0.040470] ... event mask: 000000000000000f^M
125 [ 0.040888] SMP alternatives: switching to UP code^M
126 [ 0.222104] Freeing SMP alternatives: 40k freed^M
127 [ 0.223161] ACPI: Core revision 20090903^M
128 [ 0.243130] ftrace: converting mcount calls to 0f 1f 44 00 00^M

129 [ 0.243472] ftrace: allocating 22837 entries in 90 pages^M
130 [ 0.255981] Setting APIC routing to flat^M
131 [ 0.260546] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1^M
132 [ 0.369537] CPU0: AMD QEMU Virtual CPU version 0.13.91 stepping 03^M
133 [ 0.370000] Brought up 1 CPUs^M
134 [ 0.370000] Total of 1 processors activated (5983.43 BogoMIPS).^M
135 [ 0.370000] devtmpfs: initialized^M
136 [ 0.397449] regulator: core version 0.5^M
137 [ 0.397884] Time: 7:08:16 Date: 04/19/11^M
138 [ 0.399020] NET: Registered protocol family 16^M
139 [ 0.403710] ACPI: bus type pci registered^M
140 [ 0.405327] PCI: Using configuration type 1 for base access^M
141 [ 0.416789] bio: create slab <bio-0> at 0^M
142 [ 0.449703] ACPI: Interpreter enabled^M
143 [ 0.449944] ACPI: (supports S0 S3 S4 S5)^M
144 [ 0.450704] ACPI: Using IOAPIC for interrupt routing^M
145 [ 0.483202] ACPI: No dock devices found.^M
146 [ 0.484688] ACPI: PCI Root Bridge [PCI0] (0000:00)^M
147 [ 0.487738] pci 0000:00:01.3: quirk: region b000-b03f claimed by PIIX4 ACPI^M
148 [ 0.488091] pci 0000:00:01.3: quirk: region b100-b10f claimed by PIIX4 SMB^M
149 [ 0.534028] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)^M
150 [ 0.535200] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)^M
151 [ 0.536173] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)^M
152 [ 0.537101] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)^M
153 [ 0.539895] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none^M
154 [ 0.540069] vgaarb: loaded^M
155 [ 0.541793] SCSI subsystem initialized^M
156 [ 0.543842] usbcore: registered new interface driver usbfs^M
157 [ 0.544355] usbcore: registered new interface driver hub^M
158 [ 0.544921] usbcore: registered new device driver usb^M
159 [ 0.547013] ACPI: WMI: Mapper loaded^M
160 [ 0.547213] PCI: Using ACPI for IRQ routing^M
161 [ 0.551759] NetLabel: Initializing^M
162 [ 0.551944] NetLabel: domain hash size = 128^M
163 [ 0.552162] NetLabel: protocols = UNLABELED CIPSOv4^M
164 [ 0.553168] NetLabel: unlabeled traffic allowed by default^M
165 [ 0.554355] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0^M
166 [ 0.554680] hpet0: 3 comparators, 64-bit 100.000000 MHz counter^M
167 [ 0.560664] Switching to clocksource tsc^M
168 [ 0.589565] AppArmor: AppArmor Filesystem Enabled^M
169 [ 0.590272] pnp: PnP ACPI init^M
170 [ 0.590613] ACPI: bus type pnp registered^M
171 [ 0.598044] pnp: PnP ACPI: found 6 devices^M

172 [ 0.598291] ACPI: ACPI bus type pnp unregistered^M
173 [ 0.606287] NET: Registered protocol family 2^M
174 [ 0.607819] IP route cache hash table entries: 4096 (order: 3, 32768 bytes)^M
175 [ 0.612077] TCP established hash table entries: 16384 (order: 6, 262144 bytes)^M
176 [ 0.613119] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)^M
177 [ 0.613982] TCP: Hash tables configured (established 16384 bind 16384)^M
178 [ 0.614327] TCP reno registered^M
179 [ 0.615681] NET: Registered protocol family 1^M
180 [ 0.616118] pci 0000:00:00.0: Limiting direct PCI/PCI transfers^M
181 [ 0.616463] pci 0000:00:01.0: PIIX3: Enabling Passive Release^M
182 [ 0.616891] pci 0000:00:01.0: Activating ISA DMA hang workarounds^M
183 [ 0.620264] Scanning for low memory corruption every 60 seconds^M
184 [ 0.622871] audit: initializing netlink socket (disabled)^M
185 [ 0.623553] type=2000 audit(1303196895.618:1): initialized^M
186 [ 0.659799] HugeTLB registered 2 MB page size, pre-allocated 0 pages^M
187 [ 0.675033] VFS: Disk quotas dquot_6.5.2^M
188 [ 0.675754] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)^M
189 [ 0.683995] fuse init (API version 7.13)^M
190 [ 0.685234] msgmni has been set to 985^M
191 [ 0.690133] alg: No test for stdrng (krng)^M
192 [ 0.691016] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)^M
193 [ 0.691433] io scheduler noop registered^M
194 [ 0.691644] io scheduler anticipatory registered^M
195 [ 0.691878] io scheduler deadline registered (default)^M
196 [ 0.692474] io scheduler cfq registered^M
197 [ 0.693724] pci_hotplug: PCI Hot Plug PCI Core version: 0.5^M
198 [ 0.694659] pciehp: PCI Express Hot Plug Controller Driver version: 0.4^M
199 [ 0.696576] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0^M
200 [ 0.697058] ACPI: Power Button [PWRF]^M
201 [ 0.703762] processor LNXCPU:00: registered as cooling_device0^M
202 [ 0.715736] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11^M
203 [ 0.716195] virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11^M
204 [ 0.718132] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10^M
205 [ 0.718537] virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10^M
206 [ 0.733872] Linux agpgart interface v0.103^M
207 [ 0.734881] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled^M
208 [ 0.736223] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A^M
209 [ 0.737061] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A^M
210 [ 0.748911] brd: module loaded^M
211 [ 0.753970] loop: module loaded^M
212 [ 0.758988] vda: unknown partition table^M

213 [ 0.768828] input: Macintosh mouse button emulation as /devices/virtual/input/input1^M
214 [ 0.773590] scsi0 : ata_piix^M
215 [ 0.774975] scsi1 : ata_piix^M
216 [ 0.775562] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14^M
217 [ 0.775895] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15^M
218 [ 0.780098] Fixed MDIO Bus: probed^M
219 [ 0.780777] PPP generic driver version 2.4.2^M
220 [ 0.781577] tun: Universal TUN/TAP device driver, 1.6^M
221 [ 0.781817] tun: (C) 1999-2004 Max Krasnyansky <email address hidden>^M
222 [ 0.783198] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver^M
223 [ 0.783671] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver^M
224 [ 0.784071] uhci_hcd: USB Universal Host Controller Interface driver^M
225 [ 0.784603] uhci_hcd 0000:00:01.2: PCI INT D -> Link[LNKD] -> GSI 11 (level, high) -> IRQ 11^M
226 [ 0.785163] uhci_hcd 0000:00:01.2: UHCI Host Controller^M
227 [ 0.786006] uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1^M
228 [ 0.786768] uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c020^M
229 [ 0.790792] usb usb1: configuration #1 chosen from 1 choice^M
230 [ 0.792053] hub 1-0:1.0: USB hub found^M
231 [ 0.792579] hub 1-0:1.0: 2 ports detected^M
232 [ 0.794986] PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12^M
233 [ 0.797220] serio: i8042 KBD port at 0x60,0x64 irq 1^M
234 [ 0.797634] serio: i8042 AUX port at 0x60,0x64 irq 12^M
235 [ 0.799031] mice: PS/2 mouse device common for all mice^M
236 [ 0.800808] rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0^M
237 [ 0.801476] rtc0: alarms up to one day, 114 bytes nvram, hpet irqs^M
238 [ 0.802995] device-mapper: uevent: version 1.0.3^M
239 [ 0.805806] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2^M
240 [ 0.810968] device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: <email address hidden>^M
241 [ 0.812021] device-mapper: multipath: version 1.1.0 loaded^M
242 [ 0.812387] device-mapper: multipath round-robin: version 1.0.0 loaded^M
243 [ 0.814492] cpuidle: using governor ladder^M
244 [ 0.814763] cpuidle: using governor menu^M
245 [ 0.817746] TCP cubic registered^M
246 [ 0.819585] NET: Registered protocol family 10^M
247 [ 0.825829] lo: Disabled Privacy Extensions^M
248 [ 0.828673] NET: Registered protocol family 17^M
249 [ 0.829397] powernow-k8: Processor cpuid 623 not supported^M
250 [ 0.831846] registered taskstats version 1^M
251 [ 0.833224] Magic number: 7:220:117^M
252 [ 0.834090] rtc_cmos 00:01: setting system clock to 2011-04-19 07:08:16 UTC (1303196896)^M
253 [ 0.834518] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found^M

254 [ 0.834820] EDD information not available.^M
255 [ 0.970513] md: Waiting for all devices to be available before autodetect^M
256 [ 0.970860] md: If you don't use raid, use raid=noautodetect^M
257 [ 0.973552] md: Autodetecting RAID arrays.^M
258 [ 0.973784] md: Scanned 0 and added 0 devices.^M
259 [ 0.974006] md: autorun ...^M
260 [ 0.974149] md: ... autorun DONE.^M
261 [ 0.990906] kjournald starting. Commit interval 5 seconds^M
262 [ 0.991905] EXT3-fs: mounted filesystem with ordered data mode.^M
263 [ 0.992903] VFS: Mounted root (ext3 filesystem) readonly on device 252:0.^M
264 [ 0.999796] devtmpfs: mounted^M
265 [ 1.000878] Freeing unused kernel memory: 800k freed^M
266 [ 1.016698] Write protecting the kernel read-only data: 7808k^M
267 init: plymouth main process (48) killed by SEGV signal^M
268 init: plymouth-splash main process (296) terminated with status 2^M
269 cloud-init running: Tue, 19 Apr 2011 07:09:37 +0000. up 82.22 seconds
270 waiting for metadata service at http://169.254.169.254/2009-04-04/meta-data/instance-id
271 07:09:37 [ 1/100]: url error [[Errno 101] Network is unreachable]
272 07:09:38 [ 2/100]: url error [[Errno 101] Network is unreachable]
273 07:09:39 [ 3/100]: url error [[Errno 101] Network is unreachable]
274 07:09:40 [ 4/100]: url error [[Errno 101] Network is unreachable]
275 07:09:41 [ 5/100]: url error [[Errno 101] Network is unreachable]
276 07:09:42 [ 6/100]: url error [[Errno 101] Network is unreachable]
277 07:09:44 [ 7/100]: url error [[Errno 101] Network is unreachable]
278 07:09:47 [ 8/100]: url error [[Errno 101] Network is unreachable]
279 07:09:49 [ 9/100]: url error [[Errno 101] Network is unreachable]
280 07:09:51 [10/100]: url error [[Errno 101] Network is unreachable]
281 07:09:53 [11/100]: url error [[Errno 101] Network is unreachable]
282 07:09:56 [12/100]: url error [[Errno 101] Network is unreachable]
283 07:09:59 [13/100]: url error [[Errno 101] Network is unreachable]
284 07:10:02 [14/100]: url error [[Errno 101] Network is unreachable]
285 07:10:05 [15/100]: url error [[Errno 101] Network is unreachable]
286 07:10:08 [16/100]: url error [[Errno 101] Network is unreachable]
287 07:10:12 [17/100]: url error [[Errno 101] Network is unreachable]
288 07:10:16 [18/100]: url error [[Errno 101] Network is unreachable]
289 07:10:20 [19/100]: url error [[Errno 101] Network is unreachable]
290 07:10:24 [20/100]: url error [[Errno 101] Network is unreachable]
291 07:10:28 [21/100]: url error [[Errno 101] Network is unreachable]
292 07:10:33 [22/100]: url error [[Errno 101] Network is unreachable]
293 07:10:38 [23/100]: url error [[Errno 101] Network is unreachable]

294 07:10:43 [24/100]: url error [[Errno 101] Network is unreachable]
295 07:10:48 [25/100]: url error [[Errno 101] Network is unreachable]
296 07:10:53 [26/100]: url error [[Errno 101] Network is unreachable]
297 07:10:59 [27/100]: url error [[Errno 101] Network is unreachable]
298 07:11:05 [28/100]: url error [[Errno 101] Network is unreachable]
299 07:11:11 [29/100]: url error [[Errno 101] Network is unreachable]
300 07:11:17 [30/100]: url error [[Errno 101] Network is unreachable]
301 07:11:23 [31/100]: url error [[Errno 101] Network is unreachable]
302 07:11:30 [32/100]: url error [[Errno 101] Network is unreachable]
303 07:11:37 [33/100]: url error [[Errno 101] Network is unreachable]
304 07:11:44 [34/100]: url error [[Errno 101] Network is unreachable]
305 07:11:51 [35/100]: url error [[Errno 101] Network is unreachable]
306 07:11:58 [36/100]: url error [[Errno 101] Network is unreachable]
307 07:12:06 [37/100]: url error [[Errno 101] Network is unreachable]
308 07:12:14 [38/100]: url error [[Errno 101] Network is unreachable]
309 07:12:22 [39/100]: url error [[Errno 101] Network is unreachable]
310 07:12:30 [40/100]: url error [[Errno 101] Network is unreachable]
311 07:12:38 [41/100]: url error [[Errno 101] Network is unreachable]
312 07:12:47 [42/100]: url error [[Errno 101] Network is unreachable]
313 07:12:56 [43/100]: url error [[Errno 101] Network is unreachable]
314 07:13:05 [44/100]: url error [[Errno 101] Network is unreachable]
315 07:13:14 [45/100]: url error [[Errno 101] Network is unreachable]
316 07:13:23 [46/100]: url error [[Errno 101] Network is unreachable]
317 07:13:33 [47/100]: url error [[Errno 101] Network is unreachable]
318 07:13:43 [48/100]: url error [[Errno 101] Network is unreachable]
319 07:13:53 [49/100]: url error [[Errno 101] Network is unreachable]
320 07:14:03 [50/100]: url error [[Errno 101] Network is unreachable]
321 07:14:13 [51/100]: url error [[Errno 101] Network is unreachable]
322 07:14:24 [52/100]: url error [[Errno 101] Network is unreachable]
323 07:14:35 [53/100]: url error [[Errno 101] Network is unreachable]
324 07:14:46 [54/100]: url error [[Errno 101] Network is unreachable]
325 07:14:57 [55/100]: url error [[Errno 101] Network is unreachable]
326 07:15:08 [56/100]: url error [[Errno 101] Network is unreachable]
327 07:15:20 [57/100]: url error [[Errno 101] Network is unreachable]
328 07:15:32 [58/100]: url error [[Errno 101] Network is unreachable]
329 07:15:44 [59/100]: url error [[Errno 101] Network is unreachable]
330 07:15:56 [60/100]: url error [[Errno 101] Network is unreachable]
331 07:16:08 [61/100]: url error [[Errno 101] Network is unreachable]
332 07:16:21 [62/100]: url error [[Errno 101] Network is unreachable]
333 07:16:34 [63/100]: url error [[Errno 101] Network is unreachable]
334 07:16:47 [64/100]: url error [[Errno 101] Network is unreachable]
335 07:17:00 [65/100]: url error [[Errno 101] Network is unreachable]

336 07:17:13 [66/100]: url error [[Errno 101] Network is unreachable]
337 07:17:27 [67/100]: url error [[Errno 101] Network is unreachable]
338 07:17:41 [68/100]: url error [[Errno 101] Network is unreachable]
339 07:17:55 [69/100]: url error [[Errno 101] Network is unreachable]
340 07:18:09 [70/100]: url error [[Errno 101] Network is unreachable]
341 07:18:23 [71/100]: url error [[Errno 101] Network is unreachable]
342 07:18:38 [72/100]: url error [[Errno 101] Network is unreachable]
343 07:18:53 [73/100]: url error [[Errno 101] Network is unreachable]
344 07:19:08 [74/100]: url error [[Errno 101] Network is unreachable]
345 07:19:24 [75/100]: url error [[Errno 101] Network is unreachable]
346 07:19:39 [76/100]: url error [[Errno 101] Network is unreachable]
347 07:19:55 [77/100]: url error [[Errno 101] Network is unreachable]
348 07:20:11 [78/100]: url error [[Errno 101] Network is unreachable]
349 07:20:27 [79/100]: url error [[Errno 101] Network is unreachable]
350 07:20:43 [80/100]: url error [[Errno 101] Network is unreachable]
351 07:20:59 [81/100]: url error [[Errno 101] Network is unreachable]
352 07:21:16 [82/100]: url error [[Errno 101] Network is unreachable]
353 07:21:33 [83/100]: url error [[Errno 101] Network is unreachable]
354 07:21:50 [84/100]: url error [[Errno 101] Network is unreachable]
355 07:22:07 [85/100]: url error [[Errno 101] Network is unreachable]
356 07:22:24 [86/100]: url error [[Errno 101] Network is unreachable]
357 07:22:42 [87/100]: url error [[Errno 101] Network is unreachable]
358 07:23:00 [88/100]: url error [[Errno 101] Network is unreachable]
359 07:23:18 [89/100]: url error [[Errno 101] Network is unreachable]
360 07:23:36 [90/100]: url error [[Errno 101] Network is unreachable]
361 07:23:54 [91/100]: url error [[Errno 101] Network is unreachable]
362 07:24:13 [92/100]: url error [[Errno 101] Network is unreachable]
363 07:24:32 [93/100]: url error [[Errno 101] Network is unreachable]
364 07:24:51 [94/100]: url error [[Errno 101] Network is unreachable]
365 07:25:10 [95/100]: url error [[Errno 101] Network is unreachable]
366 07:25:29 [96/100]: url error [[Errno 101] Network is unreachable]
367 07:25:49 [97/100]: url error [[Errno 101] Network is unreachable]
368 07:26:09 [98/100]: url error [[Errno 101] Network is unreachable]
369 07:26:29 [99/100]: url error [[Errno 101] Network is unreachable]
370 07:26:49 [100/100]: url error [[Errno 101] Network is unreachable]
371 giving up on md after 1051 seconds
372 Could not find data source
373 Failed to get instance data
374 init: cloud-init main process (351) terminated with status 1^M
375 mountall: Event failed
376 mountall: Plymouth command failed

377 mountall: Plymouth command failed
378 mountall: Plymouth command failed
379 mountall: Plymouth command failed
380 mountall: Disconnected from Plymouth
381 init: plymouth-log main process (394) terminated with status 1^M
382 * Starting AppArmor profiles ^[[80G ^M^[[74G[ OK ]
383 Traceback (most recent call last):
384 Traceback (most recent call last):
385 File "/usr/bin/cloud-init-cfg", line 56, in <module>
386 main()
387 File "/usr/bin/cloud-init-cfg", line 43, in main
388 cc = cloudinit.CloudConfig.CloudConfig(cfg_path)
389 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 42, in __init__
390 File "/usr/bin/cloud-init-cfg", line 56, in <module>
391 main()
392 File "/usr/bin/cloud-init-cfg", line 43, in main
393 cc = cloudinit.CloudConfig.CloudConfig(cfg_path)
394 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 42, in __init__
395 self.cfg = self.get_config_obj(cfgfile)
396 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 53, in get_config_obj
397 f=file(cfgfile)
398 IOError: [Errno 2] No such file or directory: '/var/lib/cloud/data/cloud-config.txt'
399 self.cfg = self.get_config_obj(cfgfile)
400 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 53, in get_config_obj
401 f=file(cfgfile)
402 IOError: [Errno 2] No such file or directory: '/var/lib/cloud/data/cloud-config.txt'
403 Traceback (most recent call last):
404 File "/usr/bin/cloud-init-cfg", line 56, in <module>
405 main()
406 File "/usr/bin/cloud-init-cfg", line 43, in main
407 cc = cloudinit.CloudConfig.CloudConfig(cfg_path)
408 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 42, in __init__
409 self.cfg = self.get_config_obj(cfgfile)
410 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 53, in get_config_obj
411 f=file(cfgfile)
412 IOError: [Errno 2] No such file or directory: '/var/lib/cloud/data/cloud-config.txt'
413 Traceback (most recent call last):
414 File "/usr/bin/cloud-init-cfg", line 56, in <module>
415 main()
416 File "/usr/bin/cloud-init-cfg", line 43, in main
417 cc = cloudinit.CloudConfig.CloudConfig(cfg_path)

418 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 42, in __init__
419 self.cfg = self.get_config_obj(cfgfile)
420 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 53, in get_config_obj
421 f=file(cfgfile)
422 IOError: [Errno 2] No such file or directory: '/var/lib/cloud/data/cloud-config.txt'
423 Traceback (most recent call last):
424 File "/usr/bin/cloud-init-cfg", line 56, in <module>
425 main()
426 File "/usr/bin/cloud-init-cfg", line 43, in main
427 cc = cloudinit.CloudConfig.CloudConfig(cfg_path)
428 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 42, in __init__
429 self.cfg = self.get_config_obj(cfgfile)
430 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 53, in get_config_obj
431 f=file(cfgfile)
432 IOError: [Errno 2] No such file or directory: '/var/lib/cloud/data/cloud-config.txt'
433 Traceback (most recent call last):
434 File "/usr/bin/cloud-init-cfg", line 56, in <module>
435 main()
436 File "/usr/bin/cloud-init-cfg", line 43, in main
437 cc = cloudinit.CloudConfig.CloudConfig(cfg_path)
438 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 42, in __init__
439 self.cfg = self.get_config_obj(cfgfile)
440 File "/usr/lib/python2.6/dist-packages/cloudinit/CloudConfig.py", line 53, in get_config_obj
441 f=file(cfgfile)
442 IOError: [Errno 2] No such file or directory: '/var/lib/cloud/data/cloud-config.txt'
443 landscape-client is not configured, please run landscape-config.

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Hyunsun Moon
Solved:
Last query:
Last reply:

This question was originally filed as bug #766697.

Revision history for this message
Hyunsun Moon (hyunsun-moon) said : 		#1

I was using VLAN mode and did iptables -t nat -A PREROUTING for the 169.254.169.254 to the API node.

Revision history for this message
Hyunsun Moon (hyunsun-moon) said : 		#2

Here's iptables-save result of the compute node.

# Generated by iptables-save v1.4.4 on Tue Apr 19 20:27:13 2011
*nat
:PREROUTING ACCEPT [229:43562]
:POSTROUTING ACCEPT [10:622]
:OUTPUT ACCEPT [11:693]
:nova-compute-OUTPUT - [0:0]
:nova-compute-POSTROUTING - [0:0]
:nova-compute-PREROUTING - [0:0]
:nova-compute-floating-snat - [0:0]
:nova-compute-snat - [0:0]
:nova-postrouting-bottom - [0:0]
-A PREROUTING -j nova-compute-PREROUTING
-A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 121.166.195.54:8773
-A POSTROUTING -j nova-compute-POSTROUTING
-A POSTROUTING -j nova-postrouting-bottom
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535
-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE
-A OUTPUT -j nova-compute-OUTPUT
-A nova-compute-snat -j nova-compute-floating-snat
-A nova-postrouting-bottom -j nova-compute-snat
COMMIT
# Completed on Tue Apr 19 20:27:13 2011
# Generated by iptables-save v1.4.4 on Tue Apr 19 20:27:13 2011
*filter
:INPUT ACCEPT [475311:42792648]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [928238:74704384]
:nova-compute-FORWARD - [0:0]
:nova-compute-INPUT - [0:0]
:nova-compute-OUTPUT - [0:0]
:nova-compute-inst-18 - [0:0]
:nova-compute-local - [0:0]
:nova-compute-sg-fallback - [0:0]
:nova-filter-top - [0:0]
-A INPUT -j nova-compute-INPUT
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A FORWARD -j nova-filter-top
-A FORWARD -j nova-compute-FORWARD
-A FORWARD -d 192.168.122.0/24 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT
-A FORWARD -i virbr0 -o virbr0 -j ACCEPT
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j nova-filter-top
-A OUTPUT -j nova-compute-OUTPUT
-A nova-compute-FORWARD -i br100 -j ACCEPT
-A nova-compute-FORWARD -o br100 -j ACCEPT
-A nova-compute-inst-18 -m state --state INVALID -j DROP
-A nova-compute-inst-18 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A nova-compute-inst-18 -s 10.0.0.1/32 -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A nova-compute-inst-18 -s 10.0.0.0/26 -j ACCEPT
-A nova-compute-inst-18 -j nova-compute-sg-fallback
-A nova-compute-local -d 10.0.0.3/32 -j nova-compute-inst-18
-A nova-compute-sg-fallback -j DROP
-A nova-filter-top -j nova-compute-local
COMMIT
# Completed on Tue Apr 19 20:27:13 2011

Revision history for this message
Hyunsun Moon (hyunsun-moon) said : 		#3

It's odd because compute node can get the meta data.

# GET http://169.254.169.254/
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04

----------------------------------------------------

I wonder, how instance can get the IP address from DHCP server on network node, with the private IP like 10.0.0.1?

Revision history for this message
Hugo Kou (tonytkdk) said : 		#4

Vlan mode , Instance should get IP from DHCP .

nova-network will bind 10.0.0.1 on nova-network host .....

Revision history for this message
Hyunsun Moon (hyunsun-moon) said : 		#5

I found that DHCPDISCOVERY of VM is passed to br100, but failed to be forwarded to eth0 and further network hosts in LAN.
FlatDHCP has the same problem.

Revision history for this message
Hyunsun Moon (hyunsun-moon) said : 		#6

Hi,
this was not a bug but missing a flag, --flat_interface=eth0.
Thanks!

Revision history for this message
Hyunsun Moon (hyunsun-moon) said : 		#7

Solved.