OpenStack Compute (nova)

Multi-Node Compute Installation

Asked by Shounak Acharya

Hi,

I was in the process of setting up a production Openstack Cluster. I have some very major doubts in my mind due to which I am not able to proceed any further. Please clear some of the

1) I have a cloud controller noode which runs all nova services. But on this node I don't want nova-compute to run, though I have installed it as I was following a tutorial.
2) This node also runs the MySQL as the database and Keystone as the Identity server, RabbitMQ, nova-api etc and the openstack dashboard
3) I have two Xen Cloud Platform Hosts. I have installed a domU guest on each of them which runs the nova-compute only.
4) In addition one of these domUs also runs the glance imaging service with swift as the backend object store.
5) I was successfully able to integrate glance and keystone with swift and was able to view images in the openstack dashboard.

Now the questions start

1) If I run nova-network on all compute nodes and not on the cloud-controller node then would this setup work? To rephrase, is it necessary or suggested to run the netwok services from the compute node, considering i use FlatNetworkManager. I mean one network manager on every compute node and none on the cloud-controller if it does not run the nova-compute
2) If the compute node runs only nova-compute, then how does it talk to the rest of the cloud and conversely how does the cloud controller come to know of the compute nodes in its cluster.
3) Since I am using keystone, do I need to create projects exclusively using euca2ools or keystone profiles will work
4) The nova-compute service also refuses to start on the domU compute node.

I am attaching the configuration files here. Kindly suggest the changes to meet my requirements

(I) The cloud-controller node ----- All services except nova-compute

------------------------------------------nova.conf-------------------------------------------------------------------

# DATABASE
--sql_connection=mysql://nova:notnova@10.1.27.112:3306/nova

# LOGS/STATE
--verbose
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova

# RABBITMQ
--rabbit_password=guest
--rabbit_port=5672
--rabbit_host=10.1.27.112

# SCHEDULER
--scheduler_driver=nova.scheduler.simple.SimpleScheduler

# NETWORK
--network_manager=nova.network.manager.FlatDHCPManager
--fixed_range=192.168.100.0/24
--flat_network_dhcp_start=192.168.100.2
--public_interface=eth0
--flat_interface=eth0
--flat_network_bridge=br100

# GLANCE
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=10.1.27.113:9292

# COMPUTE
--compute_manager=nova.compute.manager.ComputeManager
--libvirt_type=qemu

# VNCPROXY
#--vncproxy_url=http://192.168.206.130:6080
#--vncproxy_wwwroot=/var/lib/nova/noVNC

# MISC
--use_deprecated_auth=false
--allow_admin_api=true
--enable_zone_routing=true

# KEYSTONE
--keystone_ec2_url=http://10.1.27.112:5000/v2.0/ec2tokens

-----------------------------------api-paste.ini--------------------------------------------------------------------------------

#######
# EC2 #
#######

[composite:ec2]
use = egg:Paste#urlmap
/: ec2versions
/services/Cloud: ec2cloud
/services/Admin: ec2admin
/latest: ec2metadata
/2007-01-19: ec2metadata
/2007-03-01: ec2metadata
/2007-08-29: ec2metadata
/2007-10-10: ec2metadata
/2007-12-15: ec2metadata
/2008-02-01: ec2metadata
/2008-09-01: ec2metadata
/2009-04-04: ec2metadata
/1.0: ec2metadata

[pipeline:ec2cloud]
#pipeline = logrequest ec2noauth cloudrequest authorizer ec2executor
#pipeline = logrequest authenticate cloudrequest authorizer ec2executor
#pipeline = logrequest ec2lockout authenticate cloudrequest authorizer ec2executor
pipeline = logrequest totoken authtoken keystonecontext cloudrequest authorizer ec2executor

[pipeline:ec2admin]
#pipeline = logrequest ec2noauth adminrequest authorizer ec2executor
#pipeline = logrequest authenticate adminrequest authorizer ec2executor
pipeline = logrequest totoken authtoken keystonecontext adminrequest authorizer ec2executor

[pipeline:ec2metadata]
pipeline = logrequest ec2md

[pipeline:ec2versions]
pipeline = logrequest ec2ver

[filter:logrequest]
paste.filter_factory = nova.api.ec2:RequestLogging.factory

[filter:ec2lockout]
paste.filter_factory = nova.api.ec2:Lockout.factory
[filter:authenticate]
paste.filter_factory = nova.api.ec2:Authenticate.factory

[filter:cloudrequest]
controller = nova.api.ec2.cloud.CloudController
paste.filter_factory = nova.api.ec2:Requestify.factory

[filter:adminrequest]
controller = nova.api.ec2.admin.AdminController
paste.filter_factory = nova.api.ec2:Requestify.factory

[filter:authorizer]
paste.filter_factory = nova.api.ec2:Authorizer.factory

[app:ec2executor]
paste.app_factory = nova.api.ec2:Executor.factory

[app:ec2ver]
paste.app_factory = nova.api.ec2:Versions.factory

[app:ec2md]
paste.app_factory = nova.api.ec2.metadatarequesthandler:MetadataRequestHandler.factory

#############
# Openstack #
#############

[composite:osapi]
use = egg:Paste#urlmap
/: osversions
/v1.0: openstackapi10
/v1.1: openstackapi11

[pipeline:openstackapi10]
pipeline = faultwrap auth ratelimit osapiapp10

[pipeline:openstackapi11]
pipeline = faultwrap auth ratelimit extensions osapiapp11

[filter:faultwrap]
paste.filter_factory = nova.api.openstack:FaultWrapper.factory

[filter:auth]
paste.filter_factory = nova.api.openstack.auth:AuthMiddleware.factory

[filter:ratelimit]
paste.filter_factory = nova.api.openstack.limits:RateLimitingMiddleware.factory

[filter:extensions]
paste.filter_factory = nova.api.openstack.extensions:ExtensionMiddleware.factory

[app:osapiapp10]
paste.app_factory = nova.api.openstack:APIRouterV10.factory

[app:osapiapp11]
paste.app_factory = nova.api.openstack:APIRouterV11.factory

[pipeline:osversions]
pipeline = faultwrap osversionapp

[app:osversionapp]
paste.app_factory = nova.api.openstack.versions:Versions.factory

##########
# Shared #
##########

[filter:keystonecontext]
paste.filter_factory = keystone.middleware.nova_keystone_context:NovaKeystoneContext.factory

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 10.1.27.112
service_port = 5000
auth_host = 10.1.27.112
auth_port = 35357
auth_protocol = http
auth_uri = http://10.1.27.112:5000/v2.0/
admin_token = 11121314151617181920

(II) The cloud-compute node ------- Runs XenAPI

---------------------------nova.conf---------------------------------------------------------

--verbose
#DATABASE
--sql_connection=mysql://nova:notnova@10.1.27.112:3306/nova
--network_manager=nova.network.manager.FlatManager
--image_service=nova.image.glance.GlanceImageService
--glance_api_servers=10.1.27.113:9292
--flat_network_bridge=xenbr0
--fixed_range=10.1.27.0/24
--public-interface=eth0
--flat_interface=eth0
--cc_host=10.1.27.112
--connection_type=xenapi
--xenapi_connection_url=https://10.1.16.83
--xenapi_connection_username=root
--xenapi_connection_password=$fa90eb81dc
--reboot_timeout=600
--rescue_timeout=86400
--resize_confirm_window=86400
--allow_admin_api
--enabled_apis=osapi_compute
--allow_resize_to_same_host
--xenapi_generate_swap
--firewall_driver=nova.virt.xenapi.firewall.Dom0IptablesFirewallDriver
# RABBITMQ
--rabbit_password=guest
--rabbit_port=5672
--rabbit_host=10.1.27.112

# KEYSTONE
--keystone_ec2_url=http://10.1.27.112:5000/v2.0/ec2tokens

So please comment whether this is correct, though I am sure I am going wrong somewhere on the compute-node configuration and please suggest changes.

Thanks
Shounak

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Shounak Acharya (shounak-acharya) said : 		#1

Dear All,

Please help me..... Its a bit urgent.....

Revision history for this message
Launchpad Janitor (janitor) said : 		#2

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Revision history for this message
Shounak Acharya (shounak-acharya) said : 		#3

Guys...... Please help me...... Isn't there anyone free to answer my querries.....

Revision history for this message
John Garbutt (johngarbutt) said : 		#4

Hi,

1) If I run nova-network on all compute nodes and not on the cloud-controller node then would this setup work? To rephrase, is it necessary or suggested to run the netwok services from the compute node, considering i use FlatNetworkManager. I mean one network manager on every compute node and none on the cloud-controller if it does not run the nova-compute

if you use FlatNetworkManager (not FlatDHCPNetworkManager) it only does DB operations, so you will probably not need many of them. If you use FlatDHCP then you will need to configure it correctly:
https://blueprints.launchpad.net/nova/+spec/xenapi-ha-nova-network

2) If the compute node runs only nova-compute, then how does it talk to the rest of the cloud and conversely how does the cloud controller come to know of the compute nodes in its cluster.

nova-compute registers with rabbit mq and registers that queue with the database, and talks to the database for other things too. You will need to have that DomU sitting on the same management network as your other nodes

3) Since I am using keystone, do I need to create projects exclusively using euca2ools or keystone profiles will work

If you are using Essex, life with keystone should be fine, just follow the standard docs using keystone manage, horizon, etc.

4) The nova-compute service also refuses to start on the domU compute node.

What is the error? Make sure you have install the XenServer plugins installed into XenServer (and the vif isolation rules) and make sure they have the execute bit set.

You might find it easier to try using DevStack to get started. Although you will not want to do that for production. We have improved the docs a bit on the wiki now:
http://wiki.openstack.org/XenServer

Hope that helps.

Can you help with this problem?

Provide an answer of your own, or ask Shounak Acharya for more information if necessary.

To post a message you must log in.