OpenStack Compute (nova)

OpenStack Compute & Glance Host Information

Asked by Arun Thulasi

Hello All,

I am bringing up Nova Compute as part of a three-node setup (Cloud controller, Network controller and Compute host) and I find that Nova Compute picks up the localhost's IP as the Glance Host despite glance_api_servers clearly specifying the Cloud Controller's IP in nova.conf.

The cloud controller has keystone, cinder, glance and quantum-server running. When I run nova image-list on the compute host with keystone credentials (username, tenant name, password and auth-url), I am able to get a list of all registered images as expected.

Where does openstack-nova-compute pick up the Glance Host information from? I was under the assumption that it picks it up from the nova.conf file, but it does not appear to be so based on this behavior. I enabled 'debug' and I see that the glance host is being set to the IP address of the local host where compute runs on. I did notice that the glance_api_servers is set to 'localhost:9292' on the Cloud Controller. Thinking that this might be an issue (RPC sends 'localhost:9292' to compute and compute uses that), I modified it, restarted my services and I still run into the same problem.

I did _not_ run into this problem in my earlier attempts and wanted to check if this was a known issue. I could not find a similar thread, but if there is one, it would be very helpful if you could point me to it.

Environment
---------------
Fedora 18 on all three nodes
Grizzly (sourced from RDO) - openstack-nova-compute-2013.1.2-4.fc19.noarch

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Solved by:
Arun Thulasi
Solved:
Last query:
Last reply:
Revision history for this message
Arun Thulasi (arunuke) said : 		#1

Here is some more information illustrating the problem

Cloud Controller
===============

--> Cloud Controller IP : 10.10.10.175

[root@holmes utils]# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:35:98:4d brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.175/24 brd 10.10.10.255 scope global eth0
    inet6 fe80::5054:ff:fe35:984d/64 scope link
       valid_lft forever preferred_lft forever

--> Cloud Controller's nova.conf has the same information

[root@holmes utils]# cat /etc/nova/nova.conf | grep -i glance
#Glance
glance_api_servers = 10.10.10.175:9292
image_service = nova.image.glance.GlanceImageService

--> Cloud Controller's glance endpoints are looking good

[root@holmes utils]# keystone --os-username admin --os-password admin --os-tenant-name admin --os-auth-url http://10.10.10.175:35357/v2.0 endpoint-list | grep 9292
| d4496273c7fa48d0bdf1715799ce127d | RegionOne | http://10.10.10.175:9292 | http://10.10.10.175:9292 | http://10.10.10.175:9292 | 3582ac906e894484933f3ce9e914d79a |

--> Glance works fine when using appropriate endpoints on Cloud Controller

[root@holmes utils]# glance --os-username admin --os-password admin --os-tenant-name admin --os-auth-url http://10.10.10.175:35357/v2.0 image-list
+--------------------------------------+--------+-------------+------------------+-----------+--------+
| ID | Name | Disk Format | Container Format | Size | Status |
 +--------------------------------------+--------+-------------+------------------+-----------+--------+
| c765f5dd-22a3-4300-b248-22963a4b808c | Cirros | qcow2 | bare | 13147648 | active |
| 2985ea70-43f0-4159-ad3c-313800aa9470 | Ubuntu | qcow2 | bare | 252510208 | active |
 +--------------------------------------+--------+-------------+------------------+-----------+--------+
[root@holmes utils]#

Compute Host
==============

--> Compute host IP : 10.10.10.152

[root@adler ~]# ip addr
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:c4:25:ca brd ff:ff:ff:ff:ff:ff
    inet 10.10.10.152/24 brd 10.10.10.255 scope global eth0
    inet6 fe80::5054:ff:fec4:25ca/64 scope link
       valid_lft forever preferred_lft forever

--> Compute host's nova.conf file has the Cloud Controller's IP for glance_image_servers

[root@adler ~]# cat /etc/nova/nova.conf | grep -i glance
# Glance
glance_api_servers = 10.10.10.175:9292
image_service = nova.image.glance.GlanceImageService

--> Compute host is able to get image information by using keystone endpoints

[root@adler ~]# nova --os-username admin --os-password admin --os-tenant-name admin --os-auth-url http://10.10.10.175:35357/v2.0 image-list
+--------------------------------------+--------+--------+--------+
| ID | Name | Status | Server |
 +--------------------------------------+--------+--------+--------+
| c765f5dd-22a3-4300-b248-22963a4b808c | Cirros | ACTIVE | |
| 2985ea70-43f0-4159-ad3c-313800aa9470 | Ubuntu | ACTIVE | |
 +--------------------------------------+--------+--------+--------+
[root@adler ~]# nova --os-username admin --os-password admin --os-tenant-name admin --os-auth-url http://10.10.10.175:5000/v2.0 image-list
+--------------------------------------+--------+--------+--------+
| ID | Name | Status | Server |
 +--------------------------------------+--------+--------+--------+
| c765f5dd-22a3-4300-b248-22963a4b808c | Cirros | ACTIVE | |
| 2985ea70-43f0-4159-ad3c-313800aa9470 | Ubuntu | ACTIVE | |
 +--------------------------------------+--------+--------+--------+
[root@adler ~]#

--> Compute host's start up information contains localhost's IP address for glance (from compute.log with Debug and Verbose enabled)

 DEBUG nova.service [-] glance_api_servers : ['10.10.10.152:9292'] wait /usr/lib/python2.7/site-packages/nova/service.py:205

--> Compute fails to create a new instance since it is not able to find a glance host at 10.10.10.152 (from compute.log)

| fault | {u'message': u'GlanceConnectionFailed', u'code': 500, u'details': u'Connection to glance host 10.10.10.152:9292 failed: Error communicating with http://10.10.10.152:9292 [Errno 111] ECONNREFUSED |

I believe glance picks up the value from glance.py

glance_opts = [
    cfg.StrOpt('glance_host',
               default='$my_ip',
               help='default glance hostname or ip'),

However, I am trying to determine how to over-ride this with the values in nova.conf. Any inputs are greatly appreciated.

Revision history for this message
Arun Thulasi (arunuke) said : 		#2

Problem solved.

the glance_api_servers information was not inside the [Default] section, but underneath the [keystone_authtoken] section. I had my qpid_hostname information within [Default] and that was picked up as expected. Moved the glance_api_servers information into [Default] and things are working fine now.