OpenStack Identity (keystone)

unit tests heavy on disk IO

Bug #843062 reported by klmitch
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Fix Released
High
Dolph Mathews

Bug Description

When I run unit tests, I see a very large amount of disk IO. I moved the test databases to a tmpfs directory, which eliminated the disk IO and gave a 2-3x speedup in the unit tests.

Unit tests should run against an in-memory database. This will probably be incompatible with the sampledata method of populating a test database.

Tags: bug
Revision history for this message
klmitch (q-noreply) wrote :

You're right, an in memory database will be created and destroyed on every line of sampledata.sh - rendering it useless.

I'd like to replace server start/stop and bootstrap.sh with an approach that can be utilized by setUp & tearDown for keystone.test.system, which would make an in-memory database (e.g. sqlite://:memory:) VERY desirable.

Revision history for this message
klmitch (q-noreply) wrote :

We now have sql tests running with an in-memory DB (thanks dolph and yuriy). LDAP is still disk-based.

Dolph Mathews (dolph)
Changed in keystone:
assignee: nobody → Dolph Mathews (dolph)
status: Confirmed → In Progress
Revision history for this message
Dolph Mathews (dolph) wrote :

A fix was committed for the LDAP backend which markedy improved performance, however LDAP still runs an order of magnitude slower than pure SQL.

For example, on a dual core CPU with 4GB of ram and SSD...
SQL-only: 20.093s
LDAP: 293.231s

According to my system load, neither test suite is limited by my hard drive anymore... both are CPU-bottlenecked as they should be. However, there must be some other performance issue with fakeldap. I would suggest closing this issue and opening a new one to track this new bottleneck.

Revision history for this message
Dolph Mathews (dolph) wrote :

For reference, here's the in-memory fake-LDAP fix: https://review.openstack.org/#change,324

Revision history for this message
Dolph Mathews (dolph) wrote :

My mistake, I posted my LDAP benchmark from PRIOR to the above mentioned patch... to clarify:

SQL-only: 20.093s <- Latest
LDAP before in-memory patch: 293.231s
LDAP after in-memory patch: 104.033s <- Latest

Dolph Mathews (dolph)
Changed in keystone:
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.