Using VlanManager, no network access to instances running on another nova-compute host.
I have a setup with two hosts, one running as management/compute node and the other as a pure compute node. I am able to run instances on both hosts and have network access to any instances that are running on the combined master/compute node. The instances that are running on the pure compute node however are inaccessible from the network.
Both hosts have two active nics, eth0 is the public interface on a 10.169.30.128/25 network and eth1 is a 172.16.0.0/16 private network.
I have defined an virtual network for the project to run in as follows:
nova-manage network create --label=public --fixed_
Here is the configuration that nova-manage reports:
--storage_
--ca_file=
--ec2_dmz_
--fixed_
--compute_
--dmz_mask=
--fixed_
--glance_
--rabbit_
--user_
--s3_dmz=
--quota_ram=51200
--find_
--aws_access_
--vncserver_
--network_size=1024
--enable_
--my_ip=
--live_
--lockout_
--credential_
--quota_
--zone_
--logdir=
--sqlite_
--nouse_
--cpuinfo_
--num_networks=1
--boot_
--live_
--notification_
--osapi_
--rabbit_port=5672
--s3_access_
--rabbit_
--noresume_
--ajax_
--injected_
--network_
--snapshot_
--vncproxy_url=http://
--s3_secret_
--ajax_
--minimum_
--quota_cores=20
--nouse_project_ca
--rabbit_
--volume_
--volume_
--lock_
--live_
--flat_
--live_
--connection_
--noupdate_
--default_
--s3_port=3333
--logfile_mode=420
--logging_
--instance_
--ec2_host=$my_ip
--credential_
--vpn_cert_
--logging_
--stub_
--console_
--rpc_backend=
--default_
--osapi_scheme=http
--credential_
--sql_connectio
--console_
--instances_
--flat_injected
--use_local_volumes
--host=csvirt-1
--fixed_
--console_
--quota_
--quota_
--libvirt_
--floating_
--nomulti_host
--lockout_window=15
--db_backend=
--credentials_
--dmz_net=10.0.0.0
--sql_retry_
--vpn_start=1000
--volume_
--crl_file=crl.pem
--rpc_conn_
--s3_host=
--qemu_img=qemu-img
--max_nbd_
--vlan_
--scheduler_
--verbose
--sql_max_
--default_
--firewall_
--password_
--libvirt_type=kvm
--image_
--vpn_key_
--use_cow_images
--block_
--null_
--libvirt_
--vpn_client_
--credential_
--service_
--default_
--nopublish_errors
--quota_
--allowed_
--logging_
--enabled_
--quota_
--scheduler_
--ec2_port=8773
--rescue_
--osapi_port=8774
--auth_
--quota_volumes=10
--libvirt_uri=
--ec2_scheme=http
--keys_
--vpn_image_id=0
--host_
--noauto_
--quota_
--nofake_call
--state_
--sql_idle_
--vpn_ip=$my_ip
--default_
--aws_secret_
--nouse_ipv6
--key_file=
--nofake_network
--osapi_
--quota_
--region_list=
--auth_
--network_
--noenable_
--osapi_host=$my_ip
--zone_name=nova
--rescue_
--logging_
--timeout_nbd=10
--compute_
--libvirt_
--nofake_rabbit
--rabbit_
--vnc_keymap=en-us
--rescue_timeout=0
--ca_path=
--nouse_syslog
--superuser_
--osapi_path=/v1.0/
--ec2_path=
--allow_
--norabbit_use_ssl
--rabbit_
--node_
--lockout_
--db_driver=
--create_
--ajaxterm_
--volume_
--nostart_
--vlan_start=100
--rpc_thread_
--ipv6_
--vnc_enabled
--global_
--rabbit_
--rescue_
--network_
--ajax_
--project_
--image_
--control_
--cnt_vpn_clients=0
--vncproxy_
--compute_
--network_
Here is my nova.conf:
# RabbitMQ
--rabbit_
# MySQL
--sql_connectio
# Networking
--network_
--vlan_
--public_
--network_
--routing_
--fixed_
--network_size=1024
--dhcpbridge_
--dhcpbridge=
--user_ipv6=false
# Virtualization
--libvirt_type=kvm
# Volumes
--iscsi_
--num_targets=100
# APIs
--auth_
--cc_host=
--ec2_url=http://
--s3_host=
--s3_dmz=
# Image service
#--glance_
--glance_
--image_
# Misc
--logdir=
--state_
--lock_
--verbose
# VNC Console
--vnc_enabled=true
--vncproxy_url=http://
--vnc_console_
here is the output to /var/log/
2011-08-05 13:47:04,737 INFO nova.rpc [-] Created "compute_fanout" fanout exchange with "compute" routing key
2011-08-05 13:47:04,737 DEBUG nova.rpc [-] Initing the Adapter Consumer for compute from (pid=1303) __init__ /usr/lib/
2011-08-05 13:48:04,745 INFO nova.compute.
2011-08-05 13:48:23,561 DEBUG nova.rpc [-] received {u'_context_roles': [u'cloudadmin', u'projectmanager'], u'_context_
2011-08-05 13:48:23,561 DEBUG nova.rpc [-] unpacked context: {'user_id': u'cscloud', 'roles': [u'cloudadmin', u'projectmanager'], 'timestamp': u'2011-
2011-08-05 13:48:23,632 AUDIT nova.compute.
2011-08-05 13:48:23,869 DEBUG nova.rpc [-] Making asynchronous call on network ... from (pid=1303) multicall /usr/lib/
2011-08-05 13:48:23,869 DEBUG nova.rpc [-] MSG_ID is aff1348ef9f6428
2011-08-05 13:48:23,869 DEBUG nova.rpc [-] Creating new connection from (pid=1303) create /usr/lib/
2011-08-05 13:48:24,470 DEBUG nova.compute.
2011-08-05 13:48:24,744 DEBUG nova.virt.
2011-08-05 13:48:24,745 DEBUG nova.virt.
2011-08-05 13:48:24,745 DEBUG nova.utils [-] Attempting to grab semaphore "ensure_vlan" for method "ensure_vlan"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:24,745 DEBUG nova.utils [-] Attempting to grab file lock "ensure_vlan" for method "ensure_vlan"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:24,745 DEBUG nova.utils [-] Running cmd (subprocess): ip link show dev vlan1 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,749 DEBUG nova.utils [-] Result was 255 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,750 DEBUG nova.linux_net [-] Starting VLAN inteface vlan1 from (pid=1303) ensure_vlan /usr/lib/
2011-08-05 13:48:24,750 DEBUG nova.utils [-] Running cmd (subprocess): sudo vconfig set_name_type VLAN_PLUS_
2011-08-05 13:48:24,890 DEBUG nova.utils [-] Running cmd (subprocess): sudo vconfig add eth1 1 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,899 DEBUG nova.utils [-] Running cmd (subprocess): sudo ip link set vlan1 up from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,909 DEBUG nova.utils [-] Attempting to grab semaphore "ensure_bridge" for method "ensure_bridge"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:24,909 DEBUG nova.utils [-] Attempting to grab file lock "ensure_bridge" for method "ensure_bridge"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:24,909 DEBUG nova.utils [-] Running cmd (subprocess): ip link show dev br1 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,913 DEBUG nova.utils [-] Result was 255 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,914 DEBUG nova.linux_net [-] Starting Bridge interface for vlan1 from (pid=1303) ensure_bridge /usr/lib/
2011-08-05 13:48:24,914 DEBUG nova.utils [-] Running cmd (subprocess): sudo brctl addbr br1 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,922 DEBUG nova.utils [-] Running cmd (subprocess): sudo brctl setfd br1 0 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,931 DEBUG nova.utils [-] Running cmd (subprocess): sudo brctl stp br1 off from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,939 DEBUG nova.utils [-] Running cmd (subprocess): sudo ip link set br1 up from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,949 DEBUG nova.utils [-] Running cmd (subprocess): sudo route -n from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,958 DEBUG nova.utils [-] Running cmd (subprocess): sudo ip addr show dev vlan1 scope global from (pid=1303) execute /usr/lib/
2011-08-05 13:48:24,985 DEBUG nova.utils [-] Running cmd (subprocess): sudo brctl addif br1 vlan1 from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,059 DEBUG nova.virt.
2011-08-05 13:48:25,059 INFO nova [-] called setup_basic_
2011-08-05 13:48:25,059 INFO nova [-] ensuring static filters
2011-08-05 13:48:25,090 DEBUG nova.virt.
2011-08-05 13:48:25,090 DEBUG nova.utils [-] Attempting to grab semaphore "iptables" for method "_do_refresh_
2011-08-05 13:48:25,090 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "_do_refresh_
2011-08-05 13:48:25,093 DEBUG nova.utils [-] Attempting to grab semaphore "iptables" for method "apply"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:25,093 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "apply"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:25,093 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t filter from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,102 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,112 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t nat from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,121 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,141 DEBUG nova.virt.
2011-08-05 13:48:25,141 DEBUG nova.virt.
2011-08-05 13:48:25,141 DEBUG nova.utils [-] Attempting to grab semaphore "iptables" for method "apply"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:25,141 DEBUG nova.utils [-] Attempting to grab file lock "iptables" for method "apply"... from (pid=1303) inner /usr/lib/
2011-08-05 13:48:25,142 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t filter from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,151 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,160 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-save -t nat from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,169 DEBUG nova.utils [-] Running cmd (subprocess): sudo iptables-restore from (pid=1303) execute /usr/lib/
2011-08-05 13:48:25,179 DEBUG nova.utils [-] Running cmd (subprocess): mkdir -p /var/lib/
2011-08-05 13:48:25,297 INFO nova.virt.
2011-08-05 13:48:25,298 DEBUG nova.utils [-] Attempting to grab semaphore "00000001" for method "call_if_
2011-08-05 13:48:25,725 DEBUG nova.utils [-] Running cmd (subprocess): cp /var/lib/
2011-08-05 13:48:25,745 DEBUG nova.utils [-] Attempting to grab semaphore "da4b9237bacccd
2011-08-05 13:48:52,492 DEBUG nova.utils [-] Running cmd (subprocess): truncate -s 10737418240 /var/lib/
2011-08-05 13:48:52,505 DEBUG nova.utils [-] Running cmd (subprocess): e2fsck -fp /var/lib/
2011-08-05 13:49:04,794 INFO nova.compute.
2011-08-05 13:49:09,914 DEBUG nova.utils [-] Running cmd (subprocess): resize2fs /var/lib/
2011-08-05 13:49:15,011 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img create -f qcow2 -o cluster_
2011-08-05 13:49:15,165 DEBUG nova.utils [-] Attempting to grab semaphore "local_20" for method "call_if_
2011-08-05 13:49:15,165 DEBUG nova.utils [-] Running cmd (subprocess): truncate /var/lib/
2011-08-05 13:49:15,169 DEBUG nova.utils [-] Running cmd (subprocess): qemu-img create -f qcow2 -o cluster_
2011-08-05 13:49:15,312 INFO nova.virt.
2011-08-05 13:49:15,312 DEBUG nova.utils [-] Running cmd (subprocess): sudo qemu-nbd -c /dev/nbd15 /var/lib/
2011-08-05 13:49:16,346 DEBUG nova.utils [-] Running cmd (subprocess): sudo tune2fs -c 0 -i 0 /dev/nbd15 from (pid=1303) execute /usr/lib/
2011-08-05 13:49:18,096 DEBUG nova.utils [-] Running cmd (subprocess): sudo mount /dev/nbd15 /tmp/tmpsNhDZB from (pid=1303) execute /usr/lib/
2011-08-05 13:49:18,123 DEBUG nova.utils [-] Running cmd (subprocess): sudo mkdir -p /tmp/tmpsNhDZB/
2011-08-05 13:49:18,133 DEBUG nova.utils [-] Running cmd (subprocess): sudo chown root /tmp/tmpsNhDZB/
2011-08-05 13:49:18,142 DEBUG nova.utils [-] Running cmd (subprocess): sudo chmod 700 /tmp/tmpsNhDZB/
2011-08-05 13:49:18,151 DEBUG nova.utils [-] Running cmd (subprocess): sudo tee -a /tmp/tmpsNhDZB/
2011-08-05 13:49:18,188 DEBUG nova.utils [-] Running cmd (subprocess): sudo umount /dev/nbd15 from (pid=1303) execute /usr/lib/
2011-08-05 13:49:19,675 DEBUG nova.utils [-] Running cmd (subprocess): rmdir /tmp/tmpsNhDZB from (pid=1303) execute /usr/lib/
2011-08-05 13:49:19,696 DEBUG nova.utils [-] Running cmd (subprocess): sudo qemu-nbd -d /dev/nbd15 from (pid=1303) execute /usr/lib/
2011-08-05 13:49:21,929 DEBUG nova.virt.
2011-08-05 13:49:22,052 DEBUG nova.compute.
2011-08-05 13:49:22,157 INFO nova.virt.
IPtables on conpute only node
Chain PREROUTING (policy ACCEPT 358 packets, 22820 bytes)
pkts bytes target prot opt in out source destination
358 22820 nova-compute-
Chain INPUT (policy ACCEPT 1 packets, 328 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 5 packets, 372 bytes)
pkts bytes target prot opt in out source destination
5 372 nova-compute-OUTPUT all -- * * 0.0.0.0/0 0.0.0.0/0
Chain POSTROUTING (policy ACCEPT 9 packets, 1684 bytes)
pkts bytes target prot opt in out source destination
9 1684 nova-compute-
9 1684 nova-postroutin
0 0 MASQUERADE tcp -- * * 192.168.122.0/24 !192.168.122.0/24 masq ports: 1024-65535
0 0 MASQUERADE udp -- * * 192.168.122.0/24 !192.168.122.0/24 masq ports: 1024-65535
0 0 MASQUERADE all -- * * 192.168.122.0/24 !192.168.122.0/24
Chain nova-compute-OUTPUT (1 references)
pkts bytes target prot opt in out source destination
Chain nova-compute-
pkts bytes target prot opt in out source destination
Chain nova-compute-
pkts bytes target prot opt in out source destination
Chain nova-compute-
pkts bytes target prot opt in out source destination
Chain nova-compute-snat (1 references)
pkts bytes target prot opt in out source destination
9 1684 nova-compute-
Chain nova-postroutin
pkts bytes target prot opt in out source destination
9 1684 nova-compute-snat all -- * * 0.0.0.0/0 0.0.0.0/0
Question information
- Language:
- English Edit question
- Status:
- Solved
- Assignee:
- No assignee Edit question
- Solved by:
- Vish Ishaya
- Solved:
- Last query:
- Last reply: