euca-describe-instances fails at random

Asked by gobexe on 2011-10-24

Issuing the command again usually succeeds. Is it a bug or am I missing something?
Log file for nova-api shows:

2011-10-24 15:57:07,653 AUDIT nova.api [280c21a7-11e3-4e46-bb1b-e25e28298c30 admin grmp] Authenticated Request For admin:grmp)
2011-10-24 15:57:07,654 DEBUG nova.api [-] action: DescribeInstances from (pid=2111) __call__ /usr/lib/pymodules/python2.6/nova/api/ec2/__init__.py:239
2011-10-24 15:57:07,654 DEBUG nova.compute.api [-] Searching by: {'deleted': False} from (pid=2111) get_all /usr/lib/pymodules/python2.6/nova/compute/api.py:862
2011-10-24 15:57:07,750 ERROR nova.api [280c21a7-11e3-4e46-bb1b-e25e28298c30 admin grmp] Unexpected error raised: Parent instance <FixedIp at 0x59acc10> is not bound to a Session; lazy load operation of attribute 'network' cannot proceed
(nova.api): TRACE: Traceback (most recent call last):
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/nova/api/ec2/__init__.py", line 347, in __call__
(nova.api): TRACE: result = api_request.invoke(context)
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/nova/api/ec2/apirequest.py", line 78, in invoke
(nova.api): TRACE: result = method(context, **args)
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/nova/api/ec2/cloud.py", line 1151, in describe_instances
(nova.api): TRACE: instance_id=instance_id)
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/nova/api/ec2/cloud.py", line 1160, in _format_describe_instances
(nova.api): TRACE: return {'reservationSet': self._format_instances(context, **kwargs)}
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/nova/api/ec2/cloud.py", line 1265, in _format_instances
(nova.api): TRACE: if fixed['network'] and use_v6:
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/nova/db/sqlalchemy/models.py", line 76, in __getitem__
(nova.api): TRACE: return getattr(self, key)
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/sqlalchemy/orm/attributes.py", line 163, in __get__
(nova.api): TRACE: instance_dict(instance))
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/sqlalchemy/orm/attributes.py", line 382, in get
(nova.api): TRACE: value = callable_(passive=passive)
(nova.api): TRACE: File "/usr/lib/pymodules/python2.6/sqlalchemy/orm/strategies.py", line 578, in __call__
(nova.api): TRACE: (mapperutil.state_str(state), self.key)
(nova.api): TRACE: DetachedInstanceError: Parent instance <FixedIp at 0x59acc10> is not bound to a Session; lazy load operation of attribute 'network' cannot proceed
(nova.api): TRACE:
2011-10-24 15:57:07,789 ERROR nova.api [280c21a7-11e3-4e46-bb1b-e25e28298c30 admin grmp] Environment: {"HTTP_AUTHORIZATION": "AWS 4f2d710f-8bf3-488d-af22-5eed57a74190:grmp:LDydSvdJ+LCZMFp2VYWfvCSKYvc=", "CONTENT_TYPE": "text/plain", "SCRIPT_NAME": "/services/Cloud", "HTTP_DATE": "Mon, 24 Oct 2011 13:57:07 GMT", "REQUEST_METHOD": "GET", "HTTP_HOST": "172.31.212.81:8773", "PATH_INFO": "/", "SERVER_PROTOCOL": "HTTP/1.0", "QUERY_STRING": "AWSAccessKeyId=4f2d710f-8bf3-488d-af22-5eed57a74190%3Agrmp&Action=DescribeInstances&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-10-24T13%3A57%3A07&Version=2009-11-30&Signature=V2EMeYb7QQfXmDw5k8KDtELcuNDkDcdC%2BTp8IeRQqBg%3D", "CONTENT_LENGTH": "0", "HTTP_USER_AGENT": "Boto/1.9b (linux2)", "SERVER_NAME": "172.31.212.81", "REMOTE_ADDR": "172.31.212.81", "wsgi.url_scheme": "http", "SERVER_PORT": "8773", "GATEWAY_INTERFACE": "CGI/1.1", "HTTP_ACCEPT_ENCODING": "identity"}
2011-10-24 15:57:07,789 ERROR nova.api [280c21a7-11e3-4e46-bb1b-e25e28298c30 admin grmp] UnknownError: An unknown error has occurred. Please try your request again.
2011-10-24 15:57:07,790 INFO nova.api [280c21a7-11e3-4e46-bb1b-e25e28298c30 admin grmp] 0.153793s 172.31.212.81 GET /services/Cloud/ CloudController:DescribeInstances 400 [Boto/1.9b (linux2)] text/plain text/xml

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Compute (nova) Edit question
Assignee:
No assignee Edit question
Last query:
2011-10-24
Last reply:
2012-04-29

I am seeing this too. If I spin up a number of instances and the API server is heavily loaded, I seem to see it more often.

Tom Fifield (fifieldt) said : #2

Hi,

It looks like the bug here (https://bugs.launchpad.net/nova/+bug/855660) was solved for the Essex release, so I propose closing this Question.

Regards,

Tom

Can you help with this problem?

Provide an answer of your own, or ask gobexe for more information if necessary.

To post a message you must log in.