OpenStack Object Storage (swift)

Swift and Keystone Integration problems

Asked by crayon_z on 2011-12-07

My environment:
    Ubuntu 10.04
    Swift 1.4.3
    Keystone v2012.1
    Swift and Keystone are on the same machine

I've succeed in configuring Swift 1.4.3 and the old version Keystone. However, when I try to upgrade Keystone to the v2012.1 version, it turns out some errors as follows:

root@Swift-Proxy1:~# swift -A https://127.0.0.1:5000/v1.0 -U front2009 -K front2009 stat -v
Auth GET failed: https://127.0.0.1:5000/v1.0 401 Unauthorized

The proxy log is:
Dec 7 20:21:41 Swift-Proxy1 proxy-server 127.0.0.1 127.0.0.1 07/Dec/2011/12/21/41 HEAD /v1/AUTH_2 HTTP/1.0 401 - - 1c817fbd-cc96-4f9d-a3e4-2f17dca73a99 - - - txccd387984b8246c9b1ed06b3d42688ac - 0.0005
Dec 7 20:21:42 Swift-Proxy1 proxy-server 127.0.0.1 127.0.0.1 07/Dec/2011/12/21/42 HEAD /v1/AUTH_2 HTTP/1.0 401 - - 1c817fbd-cc96-4f9d-a3e4-2f17dca73a99 - - - txb37c45451c6649e8aa19364f06b2dd20 - 0.0005

The keystone log is:
2011-12-07 20:21:41 WARNING [eventlet.wsgi.server] 127.0.0.1 - - [07/Dec/2011 20:21:41] "GET /v1.0 HTTP/1.1" 204 278 0.073750
2011-12-07 20:21:41 WARNING [eventlet.wsgi.server] 127.0.0.1 - - [07/Dec/2011 20:21:41] "GET /v1.0 HTTP/1.1" 204 278 0.073750
2011-12-07 20:21:42 WARNING [eventlet.wsgi.server] 127.0.0.1 - - [07/Dec/2011 20:21:42] "GET /v1.0 HTTP/1.1" 204 278 0.066493
2011-12-07 20:21:42 WARNING [eventlet.wsgi.server] 127.0.0.1 - - [07/Dec/2011 20:21:42] "GET /v1.0 HTTP/1.1" 204 278 0.066493

Here is the steps I take to upgrade:

First, I fetch the newest version of Keystone from github and setup successfully. Then, I configure the keystone.conf and proxy.conf. The content of this two files are as follows:

keystone.conf
-------------------------------------------------------------------------------------------
[DEFAULT]
verbose = False
debug = False
default_store = sqlite
log_file = keystone.log
log_dir = /var/log
backends = keystone.backends.sqlalchemy
service-header-mappings = {
    'nova' : 'X-Server-Management-Url',
    'swift' : 'X-Storage-Url',
    'cdn' : 'X-CDN-Management-Url'}
service_host = 0.0.0.0
service_port = 5000
service_ssl = True
admin_host = 0.0.0.0
admin_port = 35357
admin_ssl = True
certfile = /etc/keystone/ssl/certs/keystone.pem
keyfile = /etc/keystone/ssl/private/keystonekey.pem
ca_certs = /etc/keystone/ssl/certs/ca.pem
cert_required = True
auth_protocol = https
auth_uri = https://localhost:5000/
certfile = /etc/keystone/ssl/certs/middleware-key.pem
keyfile = /etc/keystone/ssl/certs/middleware-key.pem
keystone-admin-role = Admin
keystone-service-admin-role = KeystoneServiceAdmin
hash-password = True

[keystone.backends.sqlalchemy]
sql_connection = mysql://keystone_root:xxxx@10.50.2.101/keystone
backend_entities = ['UserRoleAssociation', 'Endpoints', 'Role', 'Tenant',
'User', 'Credentials', 'EndpointTemplates', 'Token',
'Service']
sql_idle_timeout = 30

[pipeline:admin]
pipeline =
urlrewritefilter
admin_api

[pipeline:keystone-legacy-auth]
pipeline =
    urlrewritefilter
    legacy_auth
    service_api

[app:service_api]
paste.app_factory = keystone.server:service_app_factory

[app:admin_api]
paste.app_factory = keystone.server:admin_app_factory

[filter:urlrewritefilter]
paste.filter_factory = keystone.middleware.url:filter_factory

[filter:legacy_auth]
paste.filter_factory = keystone.frontends.legacy_token_auth:filter_factory

[filter:debug]
paste.filter_factory = keystone.common.wsgi:debug_filter_factory

proxy-server.conf
-------------------------------------------------------------------------------------------------------------------
[DEFAULT]
cert_file = /etc/swift/cert.crt
key_file = /etc/swift/cert.key
bind_port = 8080
workers = 32
user = swift

[pipeline:main]
pipeline = catch_errors healthcheck cache keystone proxy-server

[app:proxy-server]
use = egg:swift#proxy
set log_name = proxy-server
set log_facility = LOG_LOCAL0
set log_level = INFO
set access_log_name = proxy-server
set access_log_facility = LOG_LOCAL0
set access_log_level = INFO
account_autocreate = true
allow_account_management = true

[filter:keystone]
use = egg:keystone#swiftauth
auth_protocol = https
auth_host = 127.0.0.0
auth_port = 35357
admin_token = 999888777666
delay_auth_decision = 0
service_protocol = https
service_host = 127.0.0.0
service_port = 5000
service_pass = dTpw
cache = swift.cache

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
set log_name = cache
memcache_servers = 192.168.0.101:11211,192.168.0.102:11211

[filter:catch_errors]
use = egg:swift#catch_errors

Then I delete the old DB of Keystone and create some new data and run the command mentioned above. Then I got that error. When I change the "use = egg:keystone#swiftauth" in [filter:keystone] to "use = egg:keystone#tokenauth". Then run the same command, it turns out another error as follows:

root@Swift-Proxy1:~# swift -A https://127.0.0.1:5000/v1.0 -U demo -K front2009 stat -v
Account HEAD failed: https://127.0.0.1:8080/v1/AUTH_2 500 Internal Server Error

The proxy doesn't print any log and the keystone log is:
2011-12-07 20:29:46 WARNING [eventlet.wsgi.server] 127.0.0.1 - - [07/Dec/2011 20:29:46] "GET /v1.0 HTTP/1.1" 204 278 0.069633
2011-12-07 20:29:46 WARNING [eventlet.wsgi.server] 127.0.0.1 - - [07/Dec/2011 20:29:46] "GET /v1.0 HTTP/1.1" 204 278 0.069633

Is there anyone who has met this problem?

Thanks~

Question information

Language:: English Edit question

Status:: Solved

For:: OpenStack Object Storage (swift) Edit question

Assignee:: No assignee Edit question

Solved by:: crayon_z

Solved:: 2011-12-08

Last query:: 2011-12-08

Last reply:

Link existing bug

Revision history for this message

crayon_z (crayon-z) said on 2011-12-08:

Solved.

Revision history for this message

Tushar Patil (tpatil) said on 2011-12-08:

How did you solved the problem?

Revision history for this message

Sean Zhang (xyzhanggz) said on 2012-01-12:

Hi crayon_z,
How did you solve the problem? I met the same problem but I can't find way out.

Revision history for this message

crayon_z (crayon-z) said on 2012-01-12:

It's the config file problem. The new version of Keystone has changed the way to communicate with swift. so you should change swift's config file as follows:

[pipeline:main]
pipeline = catch_errors healthcheck cache tokenauth swiftauth proxy-server

[filter:swiftauth]
use = egg:keystone#swiftauth
keystone_url = http://127.0.0.1:5000/v2.0
keystone_admin_token = 999888777666
keystone_swift_operator_roles = Admin, SwiftOperator
keystone_tenant_user_admin = true

[filter:tokenauth]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
auth_protocol = http
auth_host = 127.0.0.1
auth_port = 35357
auth_uri = http://127.0.0.1:5000/
admin_token = 999888777666
delay_auth_decision = 0
memecache_host = 192.168.0.101:11211

Tokenauth is an authentication middleware and swiftauth is an authorization middleware. They are all located in /keystone/middleware directory. You should let swift know where you can find these two middleware.

Revision history for this message

Sean Zhang (xyzhanggz) said on 2012-01-14:

Hi crayon_z,

I got the message "KeyError: 'server'" when I ran the swift proxy server
(swift-init prxoy start) with the configure you gave me. So, could you
please help me to find out what the problems?

I've got stuck for serveral days and can find a way out. The environments
and configuration are listed below. Any help will be highly appreciated!

host operating system: Ubuntu-server 11.10
swift version: 1.4.3-0ubuntu2 (installed by apt-get install command)
keystone version: keystone 2012.1-dev (installation directory:
/home/chang/keystone)

the data in the database are generated by sample data (by running
keystone/bin/sampledata)

/etc/swift/proxy-server.conf:
[DEFAULT]
# Enter these next two values if using SSL certifications
cert_file = /etc/swift/cert.crt
key_file = /etc/swift/cert.key
bind_port= 8080
workers = 4
user = swift

[pipeline:main]
# keep swauth in the line below if you plan to use swauth for authentication
#pipeline = healthcheck cache swauth proxy-server
pipeline = healthcheck cache tokenauth swiftauth proxy-server

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true

[filter:tokenauth]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
auth_protocol = http
auth_host = 127.0.0.1
auth_port = 5001
auth_uri = http://127.0.0.1:5000/
admin_token = 999888777666
delay_auth_decision = 0

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
memcache_servers = <my host IP>:11211

part of /etc/keystone/keystone.conf:

service_host = 127.0.0.1
service_port = 5000
admin_host = 127.0.0.1
admin_port = 5001

Regards,
Sean

On 12 January 2012 19:25, crayon_z <email address hidden>wrote:

> Question #181167 on OpenStack Object Storage (swift) changed:
> https://answers.launchpad.net/swift/+question/181167
>
> crayon_z posted a new comment:
> It's the config file problem. The new version of Keystone has changed
> the way to communicate with swift. so you should change swift's config
> file as follows:
>
> [pipeline:main]
> pipeline = catch_errors healthcheck cache tokenauth swiftauth proxy-server
>
> [filter:swiftauth]
> use = egg:keystone#swiftauth
> keystone_url = http://127.0.0.1:5000/v2.0
> keystone_admin_token = 999888777666
> keystone_swift_operator_roles = Admin, SwiftOperator
> keystone_tenant_user_admin = true
>
> [filter:tokenauth]
> paste.filter_factory = keystone.middleware.auth_token:filter_factory
> auth_protocol = http
> auth_host = 127.0.0.1
> auth_port = 35357
> auth_uri = http://127.0.0.1:5000/
> admin_token = 999888777666
> delay_auth_decision = 0
> memecache_host = 192.168.0.101:11211
>
> Tokenauth is an authentication middleware and swiftauth is an
> authorization middleware. They are all located in /keystone/middleware
> directory. You should let swift know where you can find these two
> middleware.
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
>

Hi crayon_z,

I got the message "KeyError: 'server'" when I ran the swift proxy server
(swift-init prxoy start) with the configure you gave me. So, could you
please help me to find out what the problems?

I've got stuck for serveral days and can find a way out. The environments
and configuration are listed below. Any help will be highly appreciated!

host operating system: Ubuntu-server 11.10
swift version: 1.4.3-0ubuntu2 (installed by apt-get install command)
keystone version: keystone 2012.1-dev (installation directory:
/home/chang/keystone)

the data in the database are generated by sample data (by running
keystone/bin/sampledata)

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
memcache_servers = <my host IP>:11211

part of /etc/keystone/keystone.conf:

service_host = 127.0.0.1
service_port = 5000
admin_host = 127.0.0.1
admin_port = 5001

Regards,
Sean

On 12 January 2012 19:25, crayon_z <question181167@answers.launchpad.net>wrote:

Revision history for this message

crayon_z (crayon-z) said on 2012-01-14:

Hi Sean, could you paste the complete error log of proxy?

Revision history for this message

Fatih Güçlü Akkaya (gucluakkaya) said on 2012-02-07:

I am also facing the same problem and here is the error log while starting swift-proxy server:

File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/pymodules/python2.6/swift/common/wsgi.py", line 123, in run_wsgi
    loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 204, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 225, in loadobj
    return context.create()
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 625, in create
    return self.object_type.invoke(self)
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 168, in invoke
    app = filter(app)
  File "/usr/local/lib/python2.6/dist-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py", line 661, in auth_filter
    return AuthProtocol(filteredapp, conf)
  File "/usr/local/lib/python2.6/dist-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py", line 244, in __init__
    self._init_protocol_common(app, conf) # Applies to all protocols
  File "/usr/local/lib/python2.6/dist-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py", line 148, in _init_protocol_common
    logger.info("Starting the %s component", PROTOCOL_NAME)
  File "/usr/lib/python2.6/logging/__init__.py", line 1048, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python2.6/logging/__init__.py", line 1165, in _log
    self.handle(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 1175, in handle
    self.callHandlers(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 1212, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 673, in handle
    self.emit(record)
  File "/usr/lib/python2.6/logging/handlers.py", line 771, in emit
    msg = self.format(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 648, in format
    return fmt.format(record)
  File "/usr/lib/pymodules/python2.6/swift/common/utils.py", line 391, in format
    msg = logging.Formatter.format(self, record)
  File "/usr/lib/python2.6/logging/__init__.py", line 439, in format
    s = self._fmt % record.__dict__
KeyError: 'server'

I am also facing the same problem and here is the error log while starting swift-proxy server:

File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/pymodules/python2.6/swift/common/wsgi.py", line 123, in run_wsgi
    loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 204, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 225, in loadobj
    return context.create()
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 625, in create
    return self.object_type.invoke(self)
  File "/usr/lib/pymodules/python2.6/paste/deploy/loadwsgi.py", line 168, in invoke
    app = filter(app)
  File "/usr/local/lib/python2.6/dist-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py", line 661, in auth_filter
    return AuthProtocol(filteredapp, conf)
  File "/usr/local/lib/python2.6/dist-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py", line 244, in __init__
    self._init_protocol_common(app, conf)  # Applies to all protocols
  File "/usr/local/lib/python2.6/dist-packages/keystone-2012.1-py2.6.egg/keystone/middleware/auth_token.py", line 148, in _init_protocol_common
    logger.info("Starting the %s component", PROTOCOL_NAME)
  File "/usr/lib/python2.6/logging/__init__.py", line 1048, in info
    self._log(INFO, msg, args, **kwargs)
  File "/usr/lib/python2.6/logging/__init__.py", line 1165, in _log
    self.handle(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 1175, in handle
    self.callHandlers(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 1212, in callHandlers
    hdlr.handle(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 673, in handle
    self.emit(record)
  File "/usr/lib/python2.6/logging/handlers.py", line 771, in emit
    msg = self.format(record)
  File "/usr/lib/python2.6/logging/__init__.py", line 648, in format
    return fmt.format(record)
  File "/usr/lib/pymodules/python2.6/swift/common/utils.py", line 391, in format
    msg = logging.Formatter.format(self, record)
  File "/usr/lib/python2.6/logging/__init__.py", line 439, in format
    s = self._fmt % record.__dict__
KeyError: 'server'

Revision history for this message

crayon_z (crayon-z) said on 2012-02-13:

What's the command do you use to start your proxy server? Is it "swift-init proxy restart"?

Revision history for this message

sarita (sarita18narwal) said on 2012-10-10:

on Swift-init proxy start its giving error

Starting proxy-server...(/etc/swift/proxy-server.conf)
Traceback (most recent call last):
  File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 122, in run _wsgi
    loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 450, in get_context
    global_additions=global_additions)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 562, in _pipeline_app_context
    for name in pipeline[:-1]]
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 458, in get_context
    section)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 517, in _context_from_explicit
    value = import_string(found_expr)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 22, in import_string
    return pkg_resources.EntryPoint.parse("x=" + s).load(False)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1989, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named keystone.middleware.auth_token

on Swift-init proxy start its giving error

Starting proxy-server...(/etc/swift/proxy-server.conf)
Traceback (most recent call last):
  File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/python2.7/dist-packages/swift/common/wsgi.py", line 122, in run                                                                             _wsgi
    loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in                                                                              loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 271, in                                                                              loadobj
    global_conf=global_conf)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 296, in                                                                              loadcontext
    global_conf=global_conf)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 320, in                                                                              _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 450, in                                                                              get_context
    global_additions=global_additions)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 562, in                                                                              _pipeline_app_context
    for name in pipeline[:-1]]
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 458, in                                                                              get_context
    section)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 517, in                                                                              _context_from_explicit
    value = import_string(found_expr)
  File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 22, in                                                                              import_string
    return pkg_resources.EntryPoint.parse("x=" + s).load(False)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1989, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named keystone.middleware.auth_token

To post a message you must log in.

Ask a question

Edit question

OpenStack Object Storage (swift)

Swift and Keystone Integration problems

Question information

Related bugs

Related FAQ:

Subscribers