RabbitMQ is down

You should only be running one rabbitmq server and you should specify that server as the rabbit_host on both machines.

Vish

On Apr 21, 2011, at 9:28 AM, Shweta P wrote:

> New question #153751 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/153751
>
> I have setup the installations on the controller and one compute node.
>
> When I run the this command after finishing the basic config on both these machinces. I get a connection error.
>
> euca-authorize -P icmp -t -1:-1 default
> [Errno 111] Connection refused
>
> I am not sure if it is linked, but I checked the status of the rabbitMq server. When I run the command
>
>
> administrator@ubuntu:~$ sudo rabbitmqctl list_connections
> Listing connections ...
> ...done.
>
>
> It does not list any connections. I restarted the rabbitMQ server on both controller and the compute node. It still does not List any connections.
>
>
> This is what the last log says
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> Stop-and-halt request received; halting in 1000 milliseconds
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> stopped TCP Listener on 0.0.0.0:5672
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> application: rabbit
> exited: stopped
> type: temporary
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> application: mnesia
> exited: stopped
> type: temporary
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> application: os_mon
> exited: stopped
> type: temporary
>
> Though when I run the status command it does not say it is stopped
> administrator@ubuntu:~$ sudo rabbitmqctl status
> Status of node rabbit@ubuntu ...
> [{running_applications,[{rabbit,"RabbitMQ","1.7.2"},
> {mnesia,"MNESIA CXC 138 12","4.4.12"},
> {os_mon,"CPO CXC 138 46","2.2.4"},
> {sasl,"SASL CXC 138 11","2.1.8"},
> {stdlib,"ERTS CXC 138 10","1.16.4"},
> {kernel,"ERTS CXC 138 10","2.13.4"}]},
> {nodes,[rabbit@ubuntu]},
> {running_nodes,[rabbit@ubuntu]}]
> ...done.
>
> What do I need to do?
>
>
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Thanks Vish.

So I specify that in the nova.conf file right?. I have already done that
Thats my nova.conf file in both the machines.

--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--logfile
--s3_host=172.18.117.22
--rabbit_host=172.18.117.22
--ec2_host=172.18.117.22
--ec2_url=http://172.18.117.22:8773/services/Cloud
--network_manager=nova.network.manager.VlanManager
--fixed_range=10.0.112.0/24
--network_size=8
--sql_connection=mysql://root:nova@172.18.117.22/nova

The error you see is not due to rabbit issues. Looks like it is euca having trouble talking to the api. You should make sure that the ip in the credentials you are using is the correct ip of your nova-api host and nova-api is running.

Vish

On Apr 21, 2011, at 9:28 AM, Shweta P wrote:

> New question #153751 on OpenStack Compute (nova):
> https://answers.launchpad.net/nova/+question/153751
>
> I have setup the installations on the controller and one compute node.
>
> When I run the this command after finishing the basic config on both these machinces. I get a connection error.
>
> euca-authorize -P icmp -t -1:-1 default
> [Errno 111] Connection refused
>
> I am not sure if it is linked, but I checked the status of the rabbitMq server. When I run the command
>
>
> administrator@ubuntu:~$ sudo rabbitmqctl list_connections
> Listing connections ...
> ...done.
>
>
> It does not list any connections. I restarted the rabbitMQ server on both controller and the compute node. It still does not List any connections.
>
>
> This is what the last log says
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> Stop-and-halt request received; halting in 1000 milliseconds
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> stopped TCP Listener on 0.0.0.0:5672
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> application: rabbit
> exited: stopped
> type: temporary
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> application: mnesia
> exited: stopped
> type: temporary
>
> =INFO REPORT==== 21-Apr-2011::11:21:21 ===
> application: os_mon
> exited: stopped
> type: temporary
>
> Though when I run the status command it does not say it is stopped
> administrator@ubuntu:~$ sudo rabbitmqctl status
> Status of node rabbit@ubuntu ...
> [{running_applications,[{rabbit,"RabbitMQ","1.7.2"},
> {mnesia,"MNESIA CXC 138 12","4.4.12"},
> {os_mon,"CPO CXC 138 46","2.2.4"},
> {sasl,"SASL CXC 138 11","2.1.8"},
> {stdlib,"ERTS CXC 138 10","1.16.4"},
> {kernel,"ERTS CXC 138 10","2.13.4"}]},
> {nodes,[rabbit@ubuntu]},
> {running_nodes,[rabbit@ubuntu]}]
> ...done.
>
> What do I need to do?
>
>
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Thanks Vish. As you said it was not a rabbit issue.

I still face the issue though and my ip address in the nova.conf file is right. The machines are able to ping each other.

Just to help me troubleshoot please correct me if my understanding is wrong.

When I run the euca-authorize -P icmp -t -1:-1 default on the compute node. It connects to nova-api instance running on my cloud controller right?

So should the nova-api.log have records of this connection?

I tried to restart the nova-compute,nova-api,nova-scheduler,nova-network,nova-objectstore and the libvirt on the cloud controller and restarted the nova-network and nova-compute on the compute node.

The nova-*.log files do not seem to be updated when I restart the services. It should be right?

Also I just have one /etc/nova/nova.conf. I do not have a conf file for each of the services above like nova-api.conf .. etc. Is that a problem?

Please let me know any other items i need to be verifying.

Thanks

On Apr 22, 2011, at 10:39 AM, Shweta P wrote:

> Question #153751 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153751
>
> Status: Answered => Open
>
> Shweta P is still having a problem:
> Thanks Vish. As you said it was not a rabbit issue.
>
> I still face the issue though and my ip address in the nova.conf file is
> right. The machines are able to ping each other.

can you check the ip in your novarc credentials? It should be the same for all euca-commands so it is strange that it isn't working. Are you running from the same place you did euca-run-instances etc.? Did you remember to source your novarc credentials before running the command?

>
> Just to help me troubleshoot please correct me if my understanding is
> wrong.
>
> When I run the euca-authorize -P icmp -t -1:-1 default on the compute
> node. It connects to nova-api instance running on my cloud controller
> right?

Yes

>
> So should the nova-api.log have records of this connection?

If you get connection refused then it won't show in the log because it is probably connecting to the wrong ip
>
> I tried to restart the nova-compute,nova-api,nova-scheduler,nova-network
> ,nova-objectstore and the libvirt on the cloud controller and restarted
> the nova-network and nova-compute on the compute node.
>
> The nova-*.log files do not seem to be updated when I restart the
> services. It should be right?

yes they should be updating assuming that log_path is set properly and you are starting them with nova-api --flagfile=/path/to/flagfile or via upstart scripts
>
> Also I just have one /etc/nova/nova.conf. I do not have a conf file for
> each of the services above like nova-api.conf .. etc. Is that a problem?

That is fine

>
> Please let me know any other items i need to be verifying.
>
> Thanks
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

Actually none of the euca-commands work. None that need to access the nova-api i guess.

Both the machines are in the same subnet and are able to ping each other. So I am not sure why the euca connection is unable to occur. Should I check for any specific ports where the controller needs to be listening?

have you checkd your credentials? You need to source the novarc credentials. If you do have your novarc credentials sourced and it still isn't working, open them up and check the ip listed under EC2_URL. Make sure that you can hit that ip from whichever machine you are running the the euca commands on and that it is the ip where you are running nova-api.

On Apr 22, 2011, at 1:57 PM, Shweta P wrote:

> Question #153751 on OpenStack Compute (nova) changed:
> https://answers.launchpad.net/nova/+question/153751
>
> Status: Answered => Open
>
> Shweta P is still having a problem:
> Actually none of the euca-commands work. None that need to access the
> nova-api i guess.
>
> Both the machines are in the same subnet and are able to ping each
> other. So I am not sure why the euca connection is unable to occur.
> Should I check for any specific ports where the controller needs to be
> listening?
>
> --
> You received this question notification because you are a member of Nova
> Core, which is an answer contact for OpenStack Compute (nova).

I didnt have the nova-api service running on the 8773 port. That was why I was getting the cnnection refused error, dont know why though.

I had another controller setup , with which I could move forward.

Thanks for the suggestions

I am having the same issue. I followed manual installation according to cactus document. "euca-authorize -P icmp -t -1:-1 default" step gives"[Errno 111] Connection refused" . Port 8773 is not listening. nova-api is running. If anyone had resolved thhis issue it will be helpful.

haynes.

I am having the same issue. I followed manual installation according to cactus document. "euca-authorize -P icmp -t -1:-1 default" step gives"[Errno 111] Connection refused" . Port 8773 is not listening. nova-api is running. If anyone had resolved thhis issue it will be helpful.

haynes.

sorry.. nova-api is not running. its is dying after sometime.