Ceilometer issues with image tags used in Savanna

Asked by Bill Stokes

Savanna [Glance] image tags require a Hadoop version for determing the correct base image to suggest for launching a cluster. These tags if they contain special charaters (most noteably '.') cause issue with the MongoDB that Ceilometer uses. Currently, I have hacked the image tag information to sub all special charaters to '_', and will submit this as a bug if that is the next step for this question.

The details are: once an image is inserted in Glance, Ceilometer starts collecting meters about the image. In the details that it collects about the image are all its properties. Ceilometer stores the meters in MongoDB. If a property name (not value) has a dot and/or dollar symbol in it, as you saw in case of Savanna, the insert of the meter into MongoDB fails.

Below is a document (called meter) that Ceilometer tries to insert into MongoDB for an image. Notice that it is a recursive set of key, value pairs, and in the hierarchy path resource_metadata.properties, there is a key (_savanna_...) that has a dot in it. MongoDB refuses to accept the insert when Ceilometer tries to save the data. It never will.

Actually any meter (not just for Glance images) that have a key name containing a dot or dollar will not get in to MongoDB. But, OS services themselves (like Glance) may accept key names with dots and dollars. That is the crux of the issue. The only way to solve it to not have property names with the forbidden MongoDB characters.

{
     'counter_name': 'image.size',
     'user_id': None,
     'message_signature': 'ff2881317978023f21b28710d6dbed18fd1ddb83af6d75cd3a601e2ec6719136',
     'timestamp': datetime.datetime(2013,10,29,6,50,15),
     'resource_id': '28cf4ad9-75ad-42e9-b230-cf98bfa64c70',
     'message_id': '5e32cef6-4066-11e3-b649-78e3b50a8129',
     'source': 'openstack',
     'counter_unit': 'B',
     'counter_volume': 729067520,
     'project_id': 'f1e1f2f638e54627aa0c86a3acd81d4a',
     'resource_metadata': {
          'status': 'active',
          'name': 'XXX.image',
          'deleted': False,
          'container_format': 'bare',
          'created_at': '2013-10-28T23: 57: 56',
          'disk_format': 'iso',
          'updated_at': '2013-10-29T00: 13: 14',
          'properties': {
               '_savanna_tag_1.2.1': 'false',
               'metadata_savanna_tag_1.2.1': 'false'
          },
          'protected': False,
          'checksum': 'e235b63c02644e219b7bf3668f479c9e',
          'min_disk': 8,
          'is_public': True,
          'deleted_at': None,
          'min_ram': 512,
          'size': 729067520
     },
     'counter_type': 'gauge'
}

Question information

Language:
English Edit question
Status:
Solved
For:
Sahara Edit question
Assignee:
No assignee Edit question
Solved by:
Bill Stokes
Solved:
Last query:
Last reply:
Revision history for this message
Matthew Farrellee (mattf) said :
#1

This is a very cool find.

It looks to me like either a bug in the Ceilometer implementation or the Glance API.

Either Glace has to state that . and $ are invalid in tags or Ceilometer has to accept . and $ and encode them to work with MongoDB. Savanna, and others, should not have to know the specifics of the Ceilometer implementation to properly interact with Glance.

IMHO, this is something for Ceilometer to fix. The fact that MongoDB is currently backing Ceilometer should not leak beyond Ceilometer. So I'm going to move this over to Ceilometer for comment.

Revision history for this message
Bill Stokes (stokesb1) said :
#2

Thanks for the response, I'll move this over to the Ceilometer team.