problem verifying glance

Asked by Don Mize on 2012-04-16

I've been going round and round with this. Been all through the questions and tried on two different machines and ended up
at the same spot below. Any ideas?

root@xnode9:~/images# glance --os_username=adminUser --os_password=nova --os_tenant_name=openstackDemo --os_region_name=RegionOne --os_auth_url=http://172.17.86.9:5000/v2.0 add name="tty-linux-kernel" disk_format=aki container_format=aki < ttylinux-uec-amd64-12.1_2.6.35-22_1-vmlinuz
Uploading image 'tty-linux-kernel'
Failed to add image. Got error:
The request returned 500 Internal Server Error

The response body:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 336, in handle_one_response
    result = self.application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in __call__
    return app(environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 299, in __call__
    response = request.get_response(self.application)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1086, in get_response
    application, catch_exc_info=False)
  File "/usr/lib/python2.7/dist-packages/webob/request.py", line 1055, in call_application
    app_iter = application(self.environ, start_response)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 147, in __call__
    resp = self.call_func(req, *args, **self.kwargs)
  File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 210, in call_func
    return self.func(req, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/keystone/common/wsgi.py", line 318, in __call__
    for line in req.body_file:
TypeError: 'LimitedLengthFile' object is not iterable

Note: Your image metadata may still be in the registry, but the image's status will likely be 'killed'.
========================================================================================================================[100%] 41.7M/s, ETA 0h 0m 0s

Question information

Language:
English Edit question
Status:
Solved
For:
Glance Edit question
Assignee:
No assignee Edit question
Solved by:
Don Mize
Solved:
2012-04-18
Last query:
2012-04-18
Last reply:
2012-04-17
smallma (s-rain) said : #1

Can you paste the following information?

1. keystone and glance version
2. glance-registry-paste.ini, [paste_deploy] in glance-registry.conf glance-api-paste.ini and [paste_deploy] in glance-api.conf
3. export | grep OS in client node
4. endpoint-list in keystone

Regards,

Rain.

Don Mize (donald-mize) said : #2

keystone 2012.1-0ubuntu1

glance 2012.1-0ubuntu2

glance-registry-paste.ini:
------------------------------------

# Default minimal pipeline
[pipeline:glance-registry]
pipeline = context registryapp

# Use the following pipeline for keystone auth
# i.e. in glance-registry.conf:
# [paste_deploy]
# flavor = keystone
#
[pipeline:glance-registry-keystone]
pipeline = authtoken context registryapp

[app:registryapp]
paste.app_factory = glance.common.wsgi:app_factory
glance.app_factory = glance.registry.api.v1:API

[filter:context]
context_class = glance.registry.context.RequestContext
paste.filter_factory = glance.common.wsgi:filter_factory
glance.filter_factory = glance.common.context:ContextMiddleware

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_tenant_name = service
admin_user = glance
admin_password = glance
---------------------------------------------------

Glance-registry.conf:
---------------------
[paste_deploy]
flavor = keystone
---------------------

Glance-api.conf:
----------------
[paste_deploy]
flavor = keystone
-----------------

There was no paste deploy in glance-api-paste.ini, the install and deploy document didn't say anything about it.

root@xnode9:/etc/glance# export | grep OS
declare -x LESSCLOSE="/usr/bin/lesspipe %s %s"

root@xnode9:/etc/glance# keystone --token ADMIN --endpoint http://172.17.86.9:35357/v2.0 endpoint-list
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| id | region | publicurl | internalurl | adminurl |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| 5f780a73764f4c828b57b91366cb75b9 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 653a2bf1b3b04985b9d140b9626f8797 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 78186b626031484f8dab4de08753a7b5 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| a1c8ef113861443a8a5bac01ff44bb5e | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+

Thanks!
Don

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of smallma
Sent: Tuesday, April 17, 2012 4:11 AM
To: Mize, Donald
Subject: Re: [Question #193860]: problem verifying glance

Your question #193860 on Glance changed:
https://answers.launchpad.net/glance/+question/193860

    Status: Open => Needs information

smallma requested more information:
Can you paste the following information?

1. keystone and glance version
2. glance-registry-paste.ini, [paste_deploy] in glance-registry.conf glance-api-paste.ini and [paste_deploy] in glance-api.conf
3. export | grep OS in client node
4. endpoint-list in keystone

Regards,

Rain.

--
To answer this request for more information, you can either reply to
this email or enter your reply at the following page:
https://answers.launchpad.net/glance/+question/193860

You received this question notification because you asked the question.

Don Mize (donald-mize) said : #3

I've discovered this in the glance-registry.log

2012-04-17 14:12:23 3978 ERROR [glance.registry.db.api] (ProgrammingError) (1146, "Table 'glance.images' doesn't exist") 'SELECT images.created_at AS images_created_at, images.updated_at AS images_updated_at, images.deleted_at AS images_deleted_at, images.deleted AS images_deleted, images.id AS images_id, images.name AS images_name, images.disk_format AS images_disk_format, images.container_format AS images_container_format, images.size AS images_size, images.status AS images_status, images.is_public AS images_is_public, images.location AS images_location, images.checksum AS images_checksum, images.min_disk AS images_min_disk, images.min_ram AS images_min_ram, images.owner AS images_owner, images.protected AS images_protected \nFROM images \n LIMIT %s' (1,)
2012-04-17 14:12:23 3978 ERROR [glance.registry.db.api] Could not ensure database connection and consistency. Ensure database configuration and permissions are correct and database has been migrated since last upgrade by running 'glance-manage db_sync'

I use the glance-manage db_sync command and I still get the error. The database does have something in it, although I don't see the glance.images table. I'm missing a configuration step I think.

-----Original Message-----
From: Mize, Donald
Sent: Tuesday, April 17, 2012 11:35 AM
To: '<email address hidden>'
Subject: RE: [Question #193860]: problem verifying glance

keystone 2012.1-0ubuntu1

glance 2012.1-0ubuntu2

glance-registry-paste.ini:
------------------------------------

# Default minimal pipeline
[pipeline:glance-registry]
pipeline = context registryapp

# Use the following pipeline for keystone auth
# i.e. in glance-registry.conf:
# [paste_deploy]
# flavor = keystone
#
[pipeline:glance-registry-keystone]
pipeline = authtoken context registryapp

[app:registryapp]
paste.app_factory = glance.common.wsgi:app_factory
glance.app_factory = glance.registry.api.v1:API

[filter:context]
context_class = glance.registry.context.RequestContext
paste.filter_factory = glance.common.wsgi:filter_factory
glance.filter_factory = glance.common.context:ContextMiddleware

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_tenant_name = service
admin_user = glance
admin_password = glance
---------------------------------------------------

Glance-registry.conf:
---------------------
[paste_deploy]
flavor = keystone
---------------------

Glance-api.conf:
----------------
[paste_deploy]
flavor = keystone
-----------------

There was no paste deploy in glance-api-paste.ini, the install and deploy document didn't say anything about it.

root@xnode9:/etc/glance# export | grep OS
declare -x LESSCLOSE="/usr/bin/lesspipe %s %s"

root@xnode9:/etc/glance# keystone --token ADMIN --endpoint http://172.17.86.9:35357/v2.0 endpoint-list
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| id | region | publicurl | internalurl | adminurl |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+
| 5f780a73764f4c828b57b91366cb75b9 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 653a2bf1b3b04985b9d140b9626f8797 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| 78186b626031484f8dab4de08753a7b5 | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
| a1c8ef113861443a8a5bac01ff44bb5e | RegionOne | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 | http://172.17.86.9:5000/v2.0 |
+----------------------------------+-----------+------------------------------+------------------------------+------------------------------+

Thanks!
Don

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of smallma
Sent: Tuesday, April 17, 2012 4:11 AM
To: Mize, Donald
Subject: Re: [Question #193860]: problem verifying glance

Your question #193860 on Glance changed:
https://answers.launchpad.net/glance/+question/193860

    Status: Open => Needs information

smallma requested more information:
Can you paste the following information?

1. keystone and glance version
2. glance-registry-paste.ini, [paste_deploy] in glance-registry.conf glance-api-paste.ini and [paste_deploy] in glance-api.conf
3. export | grep OS in client node
4. endpoint-list in keystone

Regards,

Rain.

--
To answer this request for more information, you can either reply to
this email or enter your reply at the following page:
https://answers.launchpad.net/glance/+question/193860

You received this question notification because you asked the question.

smallma (s-rain) said : #4

Hello Don,
First, you should stop glance and remove DB. And then execute "glance-manage db_sync". at last, you set up glance-api and glance-registry again.

BTW, you endpoint looks like incorrect. Please refer to the following keystone command. and I don't know why there was no paste deploy in glance-api-paste.ini, it's stange. you should check installation steps again. glance-api-paste.ini is very important.

keystone endpoint-create --region RegionOne --service_id XXXXXXXXXXX --publicurl http://172.17.86.9:9292/v1 --adminurl http://172.17.86.9:9292/v1 --internalurl http://172.17.86.9:9292/v1

Regards,

Rain

Don Mize (donald-mize) said : #5

Changing the endpoint fixed the problem. However, I used glance in default DB mode, which means using sqlite. I did try dropping the DB and doing the glance-manage db_sync. It did put markers in the DB for glance, but none of the tables were there.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of smallma
Sent: Tuesday, April 17, 2012 10:01 PM
To: Mize, Donald
Subject: Re: [Question #193860]: problem verifying glance

Your question #193860 on Glance changed:
https://answers.launchpad.net/glance/+question/193860

smallma posted a new comment:
Hello Don,
First, you should stop glance and remove DB. And then execute "glance-manage db_sync". at last, you set up glance-api and glance-registry again.

BTW, you endpoint looks like incorrect. Please refer to the following keystone command. and I don't know why there was no paste deploy in glance-api-paste.ini, it's stange. you should check installation steps again. glance-api-paste.ini is very important.

keystone endpoint-create --region RegionOne --service_id XXXXXXXXXXX --publicurl http://172.17.86.9:9292/v1 --adminurl http://172.17.86.9:9292/v1 --internalurl http://172.17.86.9:9292/v1

Regards,

Rain

--
You received this question notification because you asked the question.

Don Mize (donald-mize) said : #6

The endpoints were wrong.