glance add failed

Asked by chnzzw on 2011-11-22

help:
 keystone+glance, glance add failed, who can help me?

linux-rskn:/usr/lib64/python2.6/site-packages/glance-tools # glance add name=chnzzw is_public=true < /opt/ubuntu-11.10-server-cloudimg-amd64-disk1.img
glance_endpoint = http://127.0.0.1:9292/v1.1/2
action=/v1.1/2/images
headers={'content-length': 225443840, 'x-image-meta-container_format': u'ovf', 'x-image-meta-min_disk': u'0', 'x-auth-token': u'4750b0c1-8279-4b24-8af5-e4e65aae35d3', 'x-image-meta-size': 225443840, 'x-image-meta-is_public': u'True', 'x-image-meta-min_ram': u'0', 'content-type': 'application/octet-stream', 'x-image-meta-disk_format': u'raw', 'x-image-meta-name': u'chnzzw'}
Failed to add image. Got error:
image
Note: Your image metadata may still be in the registry, but the image's status will likely be 'killed'.
linux-rskn:/usr/lib64/python2.6/site-packages/glance-tools # glance --debug index
params={'marker': None, 'sort_key': None, 'limit': 10, 'sort_dir': None}
glance_endpoint = http://127.0.0.1:9292/v1.1/2
action=/v1.1/2/images?limit=10
headers={'x-auth-token': u'4750b0c1-8279-4b24-8af5-e4e65aae35d3'}
res=<httplib.HTTPResponse instance at 0x155e050>
Traceback (most recent call last):
  File "/usr/bin/glance", line 1207, in <module>
    result = command(options, args)
  File "/usr/bin/glance", line 60, in wrapper
    ret = func(*args, **kwargs)
  File "/usr/bin/glance", line 498, in images_index
    print_header=True)
  File "/usr/bin/glance", line 444, in _images_index
    images = client.get_images(**parameters)
  File "/usr/lib64/python2.6/site-packages/glance/client.py", line 56, in get_images
    data = json.loads(res.read())['images']
KeyError: 'images'

Question information

Language:
English Edit question
Status:
Solved
For:
Glance Edit question
Assignee:
No assignee Edit question
Solved by:
chnzzw
Solved:
2011-11-30
Last query:
2011-11-30
Last reply:
2011-11-29
Jay Pipes (jaypipes) said : #1

Would you mind paste-binning your glance-api log contents. Please remove any security credentials from the paste!

Thanks!
-jay

chnzzw (chnzzw) said : #2

1.OKļ¼Œthe glance-api log and glance --debug index result as follow.
2.I fix the bugs of the glance but still have the error.
3.please help me solve the error,thanks all.

2011-11-23 12:24:03 DEBUG [glance-api] ********************************************************************************
2011-11-23 12:24:03 DEBUG [glance-api] Configuration options gathered from config file:
2011-11-23 12:24:03 DEBUG [glance-api] /etc/glance/glance-api.conf
2011-11-23 12:24:03 DEBUG [glance-api] ================================================
2011-11-23 12:24:03 DEBUG [glance-api] bind_host 0.0.0.0
2011-11-23 12:24:03 DEBUG [glance-api] bind_port 9292
2011-11-23 12:24:03 DEBUG [glance-api] debug True
2011-11-23 12:24:03 DEBUG [glance-api] default_store file
2011-11-23 12:24:03 DEBUG [glance-api] delayed_delete False
2011-11-23 12:24:03 DEBUG [glance-api] filesystem_store_datadir /var/lib/glance/images/
2011-11-23 12:24:03 DEBUG [glance-api] image_cache_datadir /var/lib/glance/image-cache/
2011-11-23 12:24:03 DEBUG [glance-api] image_cache_enabled False
2011-11-23 12:24:03 DEBUG [glance-api] image_cache_stall_timeout 86400
2011-11-23 12:24:03 DEBUG [glance-api] log_file /var/log/glance/api.log
2011-11-23 12:24:03 DEBUG [glance-api] notifier_strategy noop
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_host localhost
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_notification_topic glance_notifications
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_password guest
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_port 5672
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_use_ssl false
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_userid guest
2011-11-23 12:24:03 DEBUG [glance-api] rabbit_virtual_host /
2011-11-23 12:24:03 DEBUG [glance-api] registry_host 0.0.0.0
2011-11-23 12:24:03 DEBUG [glance-api] registry_port 9191
2011-11-23 12:24:03 DEBUG [glance-api] s3_store_access_key <20-char AWS access key>
2011-11-23 12:24:03 DEBUG [glance-api] s3_store_bucket <lowercased 20-char aws access key>glance
2011-11-23 12:24:03 DEBUG [glance-api] s3_store_create_bucket_on_put False
2011-11-23 12:24:03 DEBUG [glance-api] s3_store_host 127.0.0.1:8080/v1.0/
2011-11-23 12:24:03 DEBUG [glance-api] s3_store_secret_key <40-char AWS secret key>
2011-11-23 12:24:03 DEBUG [glance-api] swift_enable_snet False
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_auth_address 127.0.0.1:8080/v1.0/
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_container glance
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_create_container_on_put False
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_key a86850deb2742ec3cb41518e26aa2d89
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_large_object_chunk_size 200
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_large_object_size 5120
2011-11-23 12:24:03 DEBUG [glance-api] swift_store_user jdoe
2011-11-23 12:24:03 DEBUG [glance-api] use_syslog False
2011-11-23 12:24:03 DEBUG [glance-api] verbose True
2011-11-23 12:24:03 DEBUG [glance-api] ********************************************************************************
2011-11-23 12:24:03 DEBUG [routes.middleware] Initialized with method overriding = True, and path info altering = True
2011-11-23 12:24:03 DEBUG [eventlet.wsgi.server] (15053) wsgi starting up on http://0.0.0.0:9292/
2011-11-23 12:24:26 DEBUG [glance.api.middleware.version_negotiation] Processing request: POST /v1.1/2/images Accept:
2011-11-23 12:24:26 DEBUG [glance.api.middleware.version_negotiation] Unknown version in versioned URI: 1.1. Returning version choices.
2011-11-23 12:24:27 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [23/Nov/2011 12:24:27] "POST /v1.1/2/images HTTP/1.1" 300 339 0.883178

linux-rskn:/usr/lib64/python2.6/site-packages/glance/common # glance --debug index
Traceback (most recent call last):
  File "/usr/bin/glance", line 1207, in <module>
    result = command(options, args)
  File "/usr/bin/glance", line 60, in wrapper
    ret = func(*args, **kwargs)
  File "/usr/bin/glance", line 498, in images_index
    print_header=True)
  File "/usr/bin/glance", line 444, in _images_index
    images = client.get_images(**parameters)
  File "/usr/lib64/python2.6/site-packages/glance/client.py", line 53, in get_images
    res = self.do_request("GET", "/images", params=params)
  File "/usr/lib64/python2.6/site-packages/glance/common/client.py", line 140, in do_request
    method, action, body=body, headers=headers, params=params)
  File "/usr/lib64/python2.6/site-packages/glance/common/client.py", line 230, in _do_request
    raise exception.MultipleChoices(body=res.read())
glance.common.exception.MultipleChoices: The request returned a 302 Multiple Choices. This generally means that you have not included a version indicator in a request URI.

The body of response returned:
{"versions": [{"status": "CURRENT", "id": "v1.1", "links": [{"href": "http://0.0.0.0:9292/v1/", "rel": "self"}]}, {"status": "SUPPORTED", "id": "v1.0", "links": [{"href": "http://0.0.0.0:9292/v1/", "rel": "self"}]}]}

Jay Pipes (jaypipes) said : #3

Hi! You need to update your glance client Python library I believe... which version do you have installed?

chnzzw (chnzzw) said : #4

1. glance version: glance-2011.3.tar.gz GLANCE_VERSION = ['2011', '3']

2.keystone: keystone-1.0-py2.6.egg-info

I use the diablo version of the openstack including the nova,glance and the keystone.
.
I think that the glance is the newest version. what is the reason?

please give the way to solve the error.

Thanks.

Jay Pipes (jaypipes) said : #5

The URI that your client is attempting to hit is this:

/v1.1/2/images

That is not a valid Glance images URL. That looks like a Compute API URL (it includes the tenant "2" after a /v1.1/ version identifier.

I have a suspicion that your application pipeline is not set correctly. Could you please pastebin your glance-api.conf and glance-registry.conf please?

Thanks!
-jay

chnzzw (chnzzw) said : #6

1. I use the keystone, so I modify the glance-api.conf and glance-registry.conf as follows:

glance-api.conf :
[DEFAULT]
# Show more verbose log output (sets INFO log level output)
verbose = True

# Show debugging output in logs (sets DEBUG log level output)
debug = True

# Which backend store should Glance use by default is not specified
# in a request to add a new image to Glance? Default: 'file'
# Available choices are 'file', 'swift', and 's3'
default_store = file

# Address to bind the API server
bind_host = 0.0.0.0

# Port the bind the API server to
bind_port = 9292

# Address to find the registry server
registry_host = 0.0.0.0

# Port the registry server is listening on
registry_port = 9191

# Log to this file. Make sure you do not set the same log
# file for both the API and registry servers!
log_file = /var/log/glance/api.log

# Send logs to syslog (/dev/log) instead of to file specified by `log_file`
use_syslog = False

# ============ Notification System Options =====================

# Notifications can be sent when images are create, updated or deleted.
# There are three methods of sending notifications, logging (via the
# log_file directive), rabbit (via a rabbitmq queue) or noop (no
# notifications sent, the default)
notifier_strategy = noop

# Configuration options if sending notifications via rabbitmq (these are
# the defaults)
rabbit_host = localhost
rabbit_port = 5672
rabbit_use_ssl = false
rabbit_userid = guest
rabbit_password = guest
rabbit_virtual_host = /
rabbit_notification_topic = glance_notifications

# ============ Filesystem Store Options ========================

# Directory that the Filesystem backend store
# writes image data to
filesystem_store_datadir = /var/lib/glance/images/

# ============ Swift Store Options =============================

# Address where the Swift authentication service lives
swift_store_auth_address = 127.0.0.1:8080/v1.0/

# User to authenticate against the Swift authentication service
swift_store_user = jdoe

# Auth key for the user authenticating against the
# Swift authentication service
swift_store_key = a86850deb2742ec3cb41518e26aa2d89

# Container within the account that the account should use
# for storing images in Swift
swift_store_container = glance

# Do we create the container if it does not exist?
swift_store_create_container_on_put = False

# What size, in MB, should Glance start chunking image files
# and do a large object manifest in Swift? By default, this is
# the maximum object size in Swift, which is 5GB
swift_store_large_object_size = 5120

# When doing a large object manifest, what size, in MB, should
# Glance write chunks to Swift? This amount of data is written
# to a temporary disk buffer during the process of chunking
# the image file, and the default is 200MB
swift_store_large_object_chunk_size = 200

# Whether to use ServiceNET to communicate with the Swift storage servers.
# (If you aren't RACKSPACE, leave this False!)
#
# To use ServiceNET for authentication, prefix hostname of
# `swift_store_auth_address` with 'snet-'.
# Ex. https://example.com/v1.0/ -> https://snet-example.com/v1.0/
swift_enable_snet = False

# ============ S3 Store Options =============================

# Address where the S3 authentication service lives
s3_store_host = 127.0.0.1:8080/v1.0/

# User to authenticate against the S3 authentication service
s3_store_access_key = <20-char AWS access key>

# Auth key for the user authenticating against the
# S3 authentication service
s3_store_secret_key = <40-char AWS secret key>

# Container within the account that the account should use
# for storing images in S3. Note that S3 has a flat namespace,
# so you need a unique bucket name for your glance images. An
# easy way to do this is append your AWS access key to "glance".
# S3 buckets in AWS *must* be lowercased, so remember to lowercase
# your AWS access key if you use it in your bucket name below!
s3_store_bucket = <lowercased 20-char aws access key>glance

# Do we create the bucket if it does not exist?
s3_store_create_bucket_on_put = False

# ============ Image Cache Options ========================

image_cache_enabled = False

# Directory that the Image Cache writes data to
# Make sure this is also set in glance-pruner.conf
image_cache_datadir = /var/lib/glance/image-cache/

# Number of seconds after which we should consider an incomplete image to be
# stalled and eligible for reaping
image_cache_stall_timeout = 86400

# ============ Delayed Delete Options =============================

# Turn on/off delayed delete
delayed_delete = False

[pipeline:glance-api]
pipeline = versionnegotiation authtoken auth-context apiv1app

# NOTE: use the following pipeline for keystone
# pipeline = versionnegotiation authtoken auth-context apiv1app

# To enable Image Cache Management API replace pipeline with below:
# pipeline = versionnegotiation context imagecache apiv1app
# NOTE: use the following pipeline for keystone auth (with caching)
# pipeline = versionnegotiation authtoken auth-context imagecache apiv1app

[pipeline:versions]
pipeline = versionsapp

[app:versionsapp]
paste.app_factory = glance.api.versions:app_factory

[app:apiv1app]
paste.app_factory = glance.api.v1:app_factory

[filter:versionnegotiation]
paste.filter_factory = glance.api.middleware.version_negotiation:filter_factory

[filter:imagecache]
paste.filter_factory = glance.api.middleware.image_cache:filter_factory

[filter:context]
paste.filter_factory = glance.common.context:filter_factory

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_token = 999888777666

[filter:auth-context]
paste.filter_factory = keystone.middleware.glance_auth_token:filter_factory

glance-registry.conf :
[DEFAULT]
# Show more verbose log output (sets INFO log level output)
verbose = True

# Show debugging output in logs (sets DEBUG log level output)
debug = False

# Address to bind the registry server
bind_host = 0.0.0.0

# Port the bind the registry server to
bind_port = 9191

# Log to this file. Make sure you do not set the same log
# file for both the API and registry servers!
log_file = /var/log/glance/registry.log

# Send logs to syslog (/dev/log) instead of to file specified by `log_file`
use_syslog = False

# SQLAlchemy connection string for the reference implementation
# registry server. Any valid SQLAlchemy connection string is fine.
# See: http://www.sqlalchemy.org/docs/05/reference/sqlalchemy/connections.html#sqlalchemy.create_engine
sql_connection = sqlite:////var/lib/glance/glance.sqlite

# Period in seconds after which SQLAlchemy should reestablish its connection
# to the database.
#
# MySQL uses a default `wait_timeout` of 8 hours, after which it will drop
# idle connections. This can result in 'MySQL Gone Away' exceptions. If you
# notice this, you can lower this value to ensure that SQLAlchemy reconnects
# before MySQL can drop the connection.
sql_idle_timeout = 3600

# Limit the api to return `param_limit_max` items in a call to a container. If
# a larger `limit` query param is provided, it will be reduced to this value.
api_limit_max = 1000

# If a `limit` query param is not provided in an api request, it will
# default to `limit_param_default`
limit_param_default = 25

[pipeline:glance-registry]
pipeline = authtoken auth-context registryapp
# NOTE: use the following pipeline for keystone
# pipeline = authtoken auth-context registryapp

[app:registryapp]
paste.app_factory = glance.registry.server:app_factory

[filter:context]
context_class = glance.registry.context.RequestContext
paste.filter_factory = glance.common.context:filter_factory

[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
service_protocol = http
service_host = 127.0.0.1
service_port = 5000
auth_host = 127.0.0.1
auth_port = 35357
auth_protocol = http
auth_uri = http://127.0.0.1:5000/
admin_token = 999888777666

[filter:auth-context]
context_class = glance.registry.context.RequestContext
paste.filter_factory = keystone.middleware.glance_auth_token:filter_factory

2.when I don't use the keystone, I can glance add successed. the glane-api.conf and glance-registry.conf as follows:

in the glane-api.conf :
modify :
[pipeline:glance-api]
pipeline = versionnegotiation authtoken auth-context apiv1app
to:
[pipeline:glance-api]
pipeline = versionnegotiation context apiv1app

in the glance-registry.conf :
modify :
[pipeline:glance-registry]
pipeline = authtoken auth-context registryapp
to:
[pipeline:glance-registry]
pipeline = context registryapp

now I glance add success not using the keystone with the glance-api.conf and glance-registry.conf

I can browse and lanch the instance using the keystone with the glance-api.conf and glance-registry.conf

env |grep OS:

OS_AUTH_USER=admin
OS_AUTH_STRATEGY=keystone
OS_AUTH_URL=http://127.0.0.1:5000/v2.0/
OS_AUTH_TENANT=demo
OS_AUTH_KEY=openstack

please help me find the error,thanks.

chnzzw (chnzzw) said : #7

linux-iop5:~ # keystone-manage user list
id enabled tenant
-------------------------------------------------------------------------------
1 1 None
2 1 None
linux-iop5:~ # keystone-manage role list
id name
-------------------------------------------------------------------------------
1 Admin
2 Member
3 KeystoneAdmin
4 KeystoneServiceAdmin
linux-iop5:~ # keystone-manage tenant list
id name enabled
-------------------------------------------------------------------------------
1 admin 1
2 demo 1
linux-iop5:~ # keystone-manage service list
id name type
-------------------------------------------------------------------------------
1 nova compute
2 glance image
3 keystone identity
linux-iop5:~ # keystone-manage endpointTemplates list
All EndpointTemplates
service region Public URL
-------------------------------------------------------------------------------
nova RegionOne http://127.0.0.1:8774/v1.1/%tenant_id%
glance RegionOne http://127.0.0.1:9292/v1.1/%tenant_id%
keystone RegionOne http://127.0.0.1:5000/v2.0
linux-iop5:~ #

chnzzw (chnzzw) said : #9

sqlite> .table
credentials roles token
endpoint_templates services user_roles
endpoints tenants users
sqlite> select * from credentials;
1|1|1|EC2|admin|openstack
2|2|2|EC2|demo|openstack
sqlite> select * from roles;
1|Admin||
2|Member||
3|KeystoneAdmin||
4|KeystoneServiceAdmin||
sqlite> select * from token;
999888777666|1|1|2015-02-05 00:00:00.000000
8b623dc5-343d-4414-9b77-5e89e63bb4c4|1||2011-11-27 10:08:24.166254
7758bee8-5354-4f1d-877f-28d784d2dfb5|1|2|2011-11-27 12:29:46.543959
sqlite> select * from endpoint_templates;
1|RegionOne|1|http://127.0.0.1:8774/v1.1/%tenant_id%|http://127.0.0.1:8774/v1.1/ %tenant_id%|http://127.0.0.1:8774/v1.1/%tenant_id%|1|1
2|RegionOne|2|http://127.0.0.1:9292/v1.1/%tenant_id%|http://127.0.0.1:9292/v1.1/ %tenant_id%|http://127.0.0.1:9292/v1.1/%tenant_id%|1|1
3|RegionOne|3|http://127.0.0.1:5000/v2.0|http://127.0.0.1:35357/v2.0|http://127. 0.0.1:5000/v2.0|1|1
sqlite> select * from services;
1|nova|compute|Nova Compute Service
2|glance|image|Glance Image Service
3|keystone|identity|Keystone Identity Service
sqlite> select * from user_roles;
1|1|1|1
2|2|2|2
3|1|1|2
4|1|1|
5|1|3|
6|1|4|
sqlite> select * from endpoints;
sqlite> select * from tenants;
1|admin||1
2|demo||1
sqlite> select * from users;
1|admin|$6$rounds=40000$6YHUpERc8O1PcOb1$CrWoGumpt9P6JDiJs12SRkaldxWFBuldqu4rss2 /25YQfG0qDG2yX3rFTDEJVjWX3ap6aXPLlDHDDMZG4meFj.||1|
2|demo|$6$rounds=40000$jaI4J/RNEVZvPtyW$vAaT4sUp418JJXd.dhhAJeG/dQqdQHKcYETY3OO6 E.UAAmR.IaCVySRaIwpUerMasqT/p7tk5xHtjYNqBHDpe.||1|
sqlite>

Kevin L. Mitchell (klmitch) said : #10

I suspect your problem is the %tenant_id% in your glance endpoint template. Try updating the template to remove that. (It should be there for nova, but not for glance.)

chnzzw (chnzzw) said : #11

I delete the date from the table endpoint_templates.

excute this command:
keystone-manage $* endpointTemplates add RegionOne glance http://127.0.0.1:9292/v1.1/%tenant_id% http://127.0.0.1:9292/v1.1/%tenant_id% http://127.0.0.1:9292/v1.1/%tenant_id% 1 1

But can resolve the problem.

please tell me how to update or removed. which files or tables must be updated or removed.

thanks a lot.

Kevin L. Mitchell (klmitch) said : #12

Again, the problem is likely that you have "%tenant_id%" in the URLs you're adding to the endpointTemplate for glance. You should omit "%tenant_id%" from the URLs when adding the endpointTemplate for glance.

chnzzw (chnzzw) said : #13

I find the problem:
Add -A $TOKEN to the glance add command, excute success.

glance -A $TOKEN add name=haha is_public=true < /opt/natty-server-cloudimg-amd64-disk1.img

Mike Perez (thingee) said : #14

I'm also having the same problem mentioned above:

$ glance add -A $TOKEN name="image test" is_public=true < ~/image_test.iso --host=192.168.106.191
Failed to add image. Got error:
The request returned a 302 Multiple Choices. This generally means that you have not included a version indicator in a request URI.

I've tried both removing %tenant_id% from the endpointTemplates for glance and passing the token...still no luck.

Jay Pipes (jaypipes) said : #15

Hi! You need to have --host=192.168.106.191 before your redirect....

Mike Perez (thingee) said : #16

Oh doi...yay late nights going into the morning. :)

Thanks for your reply Jay! Well unfortunately the problem is still present.

thingee@nibbler:~$ /opt/stack/glance/bin/glance -A $token add name="image test" is_public=true --host=192.168.106.191 < ~/image_test.iso
Failed to add image. Got error:
The request returned a 302 Multiple Choices. This generally means that you have not included a version indicator in a request URI.

The body of response returned:
{"versions": [{"status": "CURRENT", "id": "v1.1", "links": [{"href": "http://0.0.0.0:9292/v1/", "rel": "self"}]}, {"status": "SUPPORTED", "id": "v1.0", "links": [{"href": "http://0.0.0.0:9292/v1/", "rel": "self"}]}]}
Note: Your image metadata may still be in the registry, but the image's status will likely be 'killed'.
thingee@nibbler:~$ env | grep OS_AUTH
OS_AUTH_USER=demo
OS_AUTH_STRATEGY=keystone
OS_AUTH_URL=http://127.0.0.1:5000/v2.0/
OS_AUTH_TENANT=demo
OS_AUTH_KEY=secrete

Jay Pipes (jaypipes) said : #17

Heya, what versions of the Glance client package are you using?

Mike Perez (thingee) said : #18

I've deployed with devstack so it appears to just being pull from master:

2012.1-dev

Jay Pipes (jaypipes) said : #19

Very odd... I'm a little stumped on this one. Usually this happens when there is an old Glance client lib installed. Let me do some investigating and I will get back to you.

Mikyung Kang (mkkang) said : #20

Hi, is this resolved? I got same error.

I'm running keystone and glance servers on different nodes. Also I'm using 2012.1-dev version.
On keystone server, OS_AUTH_* is set.
On glance server, OS_AUTH_* is not set. Only glance-api/registry.conf includes keystone configuration.
"glance index/add" on glance server
or "euca-run-instances/euca-describe-images" on keystone server...
They don't have any problems. All images are open to all.

But, I want to add member to each glance image so that image can be managed by specific member.
To do this, I run same openrc (OS_AUTH_*) with keystone server on glance server.
Then "glance index/add" provide the following error.

------------------------------
Failed to show index. Got error:
The request returned a 302 Multiple Choices. This generally means that you have not included a version indicator in a request URI.

The body of response returned:
{"versions": [{"status": "CURRENT", "id": "v1.1", "links": [{"href": "http://0.0.0.0:9292/v1/", "rel": "self"}]}, {"status": "SUPPORTED", "id": "v1.0", "links": [{"href": "http://0.0.0.0:9292/v1/", "rel": "self"}]}]}
------------------------------

Jay Pipes (jaypipes) said : #21

Please paste the output of the glance-api log file. I need to see the request that has been sent to the API server.

Thanks!
-jay

Mikyung Kang (mkkang) said : #22

Hello Jay,
This is my glance-api log file. Thanks.

2011-12-13 18:25:57 DEBUG [glance-api] ********************************************************************************
2011-12-13 18:25:57 DEBUG [glance-api] Configuration options gathered from config file:
2011-12-13 18:25:57 DEBUG [glance-api] /etc/glance/glance-api.conf
2011-12-13 18:25:57 DEBUG [glance-api] ================================================
2011-12-13 18:25:57 DEBUG [glance-api] backlog 4096
2011-12-13 18:25:57 DEBUG [glance-api] bind_host 0.0.0.0
2011-12-13 18:25:57 DEBUG [glance-api] bind_port 9292
2011-12-13 18:25:57 DEBUG [glance-api] debug True
2011-12-13 18:25:57 DEBUG [glance-api] default_store file
2011-12-13 18:25:57 DEBUG [glance-api] delayed_delete False
2011-12-13 18:25:57 DEBUG [glance-api] filesystem_store_datadir /var/lib/glance/images/
2011-12-13 18:25:57 DEBUG [glance-api] image_cache_dir /var/lib/glance/image-cache/
2011-12-13 18:25:57 DEBUG [glance-api] log_file /var/log/glance/api.log
2011-12-13 18:25:57 DEBUG [glance-api] notifier_strategy noop
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_host localhost
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_notification_topic glance_notifications
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_password guest
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_port 5672
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_use_ssl false
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_userid guest
2011-12-13 18:25:57 DEBUG [glance-api] rabbit_virtual_host /
2011-12-13 18:25:57 DEBUG [glance-api] rbd_store_ceph_conf /etc/ceph/ceph.conf
2011-12-13 18:25:57 DEBUG [glance-api] rbd_store_chunk_size 8
2011-12-13 18:25:57 DEBUG [glance-api] rbd_store_pool images
2011-12-13 18:25:57 DEBUG [glance-api] rbd_store_user glance
2011-12-13 18:25:57 DEBUG [glance-api] registry_client_protocol http
2011-12-13 18:25:57 DEBUG [glance-api] registry_host 0.0.0.0
2011-12-13 18:25:57 DEBUG [glance-api] registry_port 9191
2011-12-13 18:25:57 DEBUG [glance-api] s3_store_access_key <20-char AWS access key>
2011-12-13 18:25:57 DEBUG [glance-api] s3_store_bucket <lowercased 20-char aws access key>glance
2011-12-13 18:25:57 DEBUG [glance-api] s3_store_create_bucket_on_put False
2011-12-13 18:25:57 DEBUG [glance-api] s3_store_host 127.0.0.1:8080/v1.0/
2011-12-13 18:25:57 DEBUG [glance-api] s3_store_secret_key <40-char AWS secret key>
2011-12-13 18:25:57 DEBUG [glance-api] scrub_time 43200
2011-12-13 18:25:57 DEBUG [glance-api] scrubber_datadir /var/lib/glance/scrubber
2011-12-13 18:25:57 DEBUG [glance-api] swift_enable_snet False
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_auth_address 127.0.0.1:8080/v1.0/
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_container glance
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_create_container_on_put False
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_key a86850deb2742ec3cb41518e26aa2d89
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_large_object_chunk_size 200
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_large_object_size 5120
2011-12-13 18:25:57 DEBUG [glance-api] swift_store_user jdoe
2011-12-13 18:25:57 DEBUG [glance-api] use_syslog False
2011-12-13 18:25:57 DEBUG [glance-api] verbose True
2011-12-13 18:25:57 DEBUG [glance-api] ********************************************************************************
2011-12-13 18:25:57 DEBUG [routes.middleware] Initialized with method overriding = True, and path info altering = True
2011-12-13 18:25:57 DEBUG [eventlet.wsgi.server] (16971) wsgi starting up on http://0.0.0.0:9292/
2011-12-13 18:26:11 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:26:11 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:26:11 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:26:11 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:26:11 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:26:11 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:26:11] "GET /v1/images?limit=10 HTTP/1.1" 200 2018 0.038183
2011-12-13 18:26:13 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:26:13 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:26:13 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:26:13 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:26:13 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:26:13 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:26:13] "GET /v1/images?marker=1a7c9443-e69e-47f3-aa06-d1781dbb0f9c&limit=10 HTTP/1.1" 200 1103 0.023296
2011-12-13 18:26:13 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:26:13 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:26:13 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:26:13 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:26:13 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:26:13 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:26:13] "GET /v1/images?marker=ad35b9c2-8fb8-422e-a8a3-7f201a29a224&limit=10 HTTP/1.1" 200 162 0.018879
2011-12-13 18:28:32 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:28:32 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:28:32 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:28:32 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:28:32 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:28:32 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:28:32] "GET /v1/images?limit=10 HTTP/1.1" 200 2018 0.020744
2011-12-13 18:28:33 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:28:33 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:28:33 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:28:33 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:28:33 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:28:33 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:28:33] "GET /v1/images?marker=1a7c9443-e69e-47f3-aa06-d1781dbb0f9c&limit=10 HTTP/1.1" 200 1103 0.023446
2011-12-13 18:28:33 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:28:33 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:28:33 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:28:33 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:28:33 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:28:33 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:28:33] "GET /v1/images?marker=ad35b9c2-8fb8-422e-a8a3-7f201a29a224&limit=10 HTTP/1.1" 200 162 0.023170
2011-12-13 18:28:46 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1.1//images Accept:
2011-12-13 18:28:46 DEBUG [glance.api.middleware.version_negotiation] Unknown version in versioned URI: 1.1. Returning version choices.
2011-12-13 18:28:46 DEBUG [eventlet.wsgi.server] 10.0.11.1 - - [13/Dec/2011 18:28:46] "GET /v1.1//images?limit=10 HTTP/1.1" 300 339 0.000883
2011-12-13 18:29:36 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:29:36 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:29:36 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:29:36 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:29:36 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:29:36 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:29:36] "GET /v1/images?limit=10 HTTP/1.1" 200 2018 0.044182
2011-12-13 18:29:36 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:29:36 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:29:36 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:29:36 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:29:36 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:29:36 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:29:36] "GET /v1/images?marker=1a7c9443-e69e-47f3-aa06-d1781dbb0f9c&limit=10 HTTP/1.1" 200 1103 0.023401
2011-12-13 18:29:36 DEBUG [glance.api.middleware.version_negotiation] Processing request: GET /v1/images Accept:
2011-12-13 18:29:36 DEBUG [glance.api.middleware.version_negotiation] Matched versioned URI. Version: 1.0
2011-12-13 18:29:36 DEBUG [routes.middleware] Matched GET /images
2011-12-13 18:29:36 DEBUG [routes.middleware] Route path: '/images', defaults: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:29:36 DEBUG [routes.middleware] Match dict: {'action': u'index', 'controller': <glance.common.wsgi.Resource object at 0x1821490>}
2011-12-13 18:29:36 DEBUG [eventlet.wsgi.server] 127.0.0.1 - - [13/Dec/2011 18:29:36] "GET /v1/images?marker=ad35b9c2-8fb8-422e-a8a3-7f201a29a224&limit=10 HTTP/1.1" 200 162 0.019342

Jay Pipes (jaypipes) said : #23

Notice in the API log that *some* requests are being received with a request of:

GET /v1/images?limit=10

whereas some requests are being received as:

GET /v1.1//images?limit=10

The former is working just fine, the latter is returning a 302 since it does not understand v1.1.// apparently.

Can you isolate which client is actually issuing the incorrect call vs which clients are issuing the correct (v1/images/) call?

Thanks,
-jay

Mikyung Kang (mkkang) said : #24

Hi Jay,

I'm working on same glance server.

A terminal: just run "glance index" --> GET /v1/images?limit=10 (OK)
B terminal: after setting OS_AUTH_*, run "glance index" --> GET /v1.1//images?limit=10 (ERROR: 302 Multiple Choices)

sh-4.1# cat openrc

HOST_IP=**** // our keystone server IP
export NOVA_PROJECT_ID=demo
export NOVA_USERNAME=demo
export NOVA_PASSWORD=guest
export NOVA_API_KEY=$NOVA_PASSWORD
#export NOVA_URL="http://65.114.169.38:8774/v1.1/"
export NOVA_AUTH_URL=http://$HOST_IP:5000/v2.0/
export NOVA_VERSION=1.1
export NOVA_REGION_NAME=RegionOne

export EC2_URL=http://$HOST_IP:8773/services/Cloud
export EC2_ACCESS_KEY=demo
#export EC2_ACCESS_KEY="demo:guest"
export EC2_SECRET_KEY=secretpassword
#export EC2_SECRET_KEY="secretpassword"

export OS_AUTH_USER=$NOVA_USERNAME
export OS_AUTH_KEY=$NOVA_API_KEY
export OS_AUTH_TENANT=$NOVA_PROJECT_ID
export OS_AUTH_URL=http://$HOST_IP:5000/v2.0/
export OS_AUTH_STRATEGY=keystone

If I change NOVA_VERSION to 1,
sh-4.1# source ./openrc
sh-4.1# glance index
Not authorized to make this request. Check your credentials (OS_AUTH_USER, OS_AUTH_KEY, ...).

Thanks,
Mikyung

Jay Pipes (jaypipes) said : #25

Hi Mikyung!

I'm going to add this as a bug report and try to get a repeatable test case together. I have a suspicion that it is the endpoint template for the image service that Keystone is returning that is the culprit here.

I'll update you when I have more information.

Thanks!
-jay

Eric Dodemont (dodeeric) said : #26

To solve the problem:

Change the glance endpointemplate in keystone:

From:

   http://Glance-API-IP:9292/v1.1/%tenant_id%

To:

   http://Glance-API-IP:9292/v1 (without the trailing slash!!!)