Synaps

RabittMQ error

Asked by Henry Wong

Getting this in the synaps-api logs:

2014-03-07 19:08:14 ERROR synaps.api.cloudwatch [req-ad40ae8a-2d36-4428-b8e8-ec92b49c01c0 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] Environment: {"SCRIPT_NAME": "/monitor", "REQUEST_METHOD": "POST", "HTTP_HOST": "162.150.4.117:8776", "PATH_INFO": "/", "SERVER_PROTOCOL": "HTTP/1.0", "HTTP_USER_AGENT": "aws-sdk-java/1.3.12 Mac_OS_X/10.8.5 Java_HotSpot(TM)_64-Bit_Server_VM/20.65-b04-462", "HTTP_CONNECTION": "Keep-Alive", "SERVER_NAME": "162.150.4.117", "REMOTE_ADDR": "10.172.52.3", "CONTENT_TYPE": "application/x-www-form-urlencoded; charset=utf-8", "wsgi.url_scheme": "http", "SERVER_PORT": "8776", "GATEWAY_INTERFACE": "CGI/1.1"}
2014-03-07 19:08:14 ERROR synaps.api.cloudwatch [req-ad40ae8a-2d36-4428-b8e8-ec92b49c01c0 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] 500: An unknown error has occurred. Please try your request again.
2014-03-07 19:08:14 INFO synaps.api.cloudwatch [req-ad40ae8a-2d36-4428-b8e8-ec92b49c01c0 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] 21.559977s 10.172.52.3 POST /monitor/ MonitorController:PutMetricData 500 [aws-sdk-java/1.3.12 Mac_OS_X/10.8.5 Java_HotSpot(TM)_64-Bit_Server_VM/20.65-b04-462] application/x-www-form-urlencoded text/xml
2014-03-07 19:08:17 INFO synaps.rpc [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] Connecting to rabbit_host synaps-rabbitmq 5672
2014-03-07 19:08:17 WARNING synaps.rpc [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] AMQP connection exception occurred 0 time(s)... retrying.
2014-03-07 19:08:17 WARNING synaps.rpc [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] AMQP connection exception occurred 1 time(s)... retrying.
2014-03-07 19:08:20 WARNING synaps.rpc [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] AMQP connection exception occurred 2 time(s)... retrying.
2014-03-07 19:08:24 WARNING synaps.rpc [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] AMQP connection exception occurred 3 time(s)... retrying.
2014-03-07 19:08:30 WARNING synaps.rpc [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] AMQP connection exception occurred 4 time(s)... retrying.
2014-03-07 19:08:38 ERROR synaps.api.cloudwatch [req-e2c4fdef-d3c2-4c20-a990-fed27fc5cfa1 a060861db3514b4289948c34543d5208 a060861db3514b4289948c34543d5208] Unexpected error raised: 2.0
(synaps.api.cloudwatch): TRACE: Traceback (most recent call last):
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/api/cloudwatch/__init__.py", line 387, in __call__
(synaps.api.cloudwatch): TRACE: result = api_request.invoke(context)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/api/cloudwatch/apirequest.py", line 83, in invoke
(synaps.api.cloudwatch): TRACE: result = method(context, **args)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/api/cloudwatch/monitor.py", line 458, in put_metric_data
(synaps.api.cloudwatch): TRACE: is_admin=context.is_admin)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/monitor/api.py", line 311, in put_metric_data
(synaps.api.cloudwatch): TRACE: self.rpc.send_msg(rpc.PUT_METRIC_DATA_MSG_ID, message)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/rpc/__init__.py", line 159, in send_msg
(synaps.api.cloudwatch): TRACE: publish(json.dumps(body), properties, use_metric_pool)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/rpc/__init__.py", line 135, in publish
(synaps.api.cloudwatch): TRACE: with mq_pool.item() as conn:
(synaps.api.cloudwatch): TRACE: File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
(synaps.api.cloudwatch): TRACE: return self.gen.next()
(synaps.api.cloudwatch): TRACE: File "<string>", line 15, in item_impl
(synaps.api.cloudwatch): TRACE: File "/usr/lib/python2.7/dist-packages/eventlet/pools.py", line 117, in get
(synaps.api.cloudwatch): TRACE: created = self.create()
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/rpc/__init__.py", line 52, in __init__
(synaps.api.cloudwatch): TRACE: self.connect()
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/synaps-2013.2.3-py2.7.egg/synaps/rpc/__init__.py", line 73, in connect
(synaps.api.cloudwatch): TRACE: self.conn = pika.BlockingConnection(con_param)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/pika-0.9.6-py2.7.egg/pika/adapters/base_connection.py", line 61, in __init__
(synaps.api.cloudwatch): TRACE: super(BaseConnection, self).__init__(parameters, on_open_callback)
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/pika-0.9.6-py2.7.egg/pika/connection.py", line 513, in __init__
(synaps.api.cloudwatch): TRACE: self._connect()
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/pika-0.9.6-py2.7.egg/pika/connection.py", line 802, in _connect
(synaps.api.cloudwatch): TRACE: self._adapter_connect()
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/pika-0.9.6-py2.7.egg/pika/adapters/blocking_connection.py", line 137, in _adapter_connect
(synaps.api.cloudwatch): TRACE: super(BlockingConnection, self)._adapter_connect()
(synaps.api.cloudwatch): TRACE: File "/usr/local/lib/python2.7/dist-packages/pika-0.9.6-py2.7.egg/pika/adapters/base_connection.py", line 120, in _adapter_connect
(synaps.api.cloudwatch): TRACE: self.params.retry_delay)
(synaps.api.cloudwatch): TRACE: AMQPConnectionError: 2.0
(synaps.api.cloudwatch): TRACE:

From the client:
 PutMetricData Req: {Namespace: SYNAPSDEMO/2013, MetricData: [{MetricName: DiskUsage, Dimensions: [{Name: instanceId, Value: instance-00000112, }], Value: 10.2, }, {MetricName: RequestsPerSecond, Dimensions: [{Name: swlbId, Value: swlb-12345, }], Value: 1000.0, }], }
log4j:WARN No appenders could be found for logger (com.amazonaws.request).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" Status Code: 500, AWS Service: AmazonCloudWatch, AWS Request ID: req-2da73bfa-eca4-414e-bcbc-ac12d5f02e94, AWS Error Code: 500, AWS Error Message: An unknown error has occurred. Please try your request again.
 at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:556)
 at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:289)
 at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170)
 at com.amazonaws.services.cloudwatch.AmazonCloudWatchClient.invoke(AmazonCloudWatchClient.java:739)
 at com.amazonaws.services.cloudwatch.AmazonCloudWatchClient.putMetricData(AmazonCloudWatchClient.java:308)
 at synaps.CWSample.examPutMetricData(CWSample.java:125)
 at synaps.CWSample.main(CWSample.java:37)

Any suggestions to getting around this error?

Question information

Language:
English Edit question
Status:
Answered
For:
Synaps Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
June Yi (gochist) said : 		#1

2014-03-07 19:08:17 INFO synaps.rpc [* * *] Connecting to rabbit_host synaps-rabbitmq 5672

Please check your rabbitmq service is running on the host synaps-rabbitmq. (eg. telnet synaps-rabbitmq 5672)

Can you help with this problem?

Provide an answer of your own, or ask Henry Wong for more information if necessary.

To post a message you must log in.