registered images is queued, not showing up

Actually, if I can guess its ID, it is shown as available like

seki@OSCC:/tmp$ euca-describe-images ami-00000008
IMAGE ami-00000008 win8/win08-64bit.qcow2.manifest.xml available public x86_64 machine

And I can run instance out of it
seki@OSCC:/tmp$ euca-run-instances ami-00000008
RESERVATION r-nr8zt1pa test default
INSTANCE i-00000026 ami-00000008 scheduling None (test, None) 0 m1.small 2011-04-27T20:13:04Z unknown zone

It is just not showing up automatically under the pure euca-describe-images call.

After further investigation, this is the exact same issue I am having (see https://answers.launchpad.net/glance/+question/154442). Please consider these duplicate questions.

Cheers,
   Graham

I can duplicate this. I am still investigating why a new version of glance may have caused this to manifest itself, but from what I can tell, the issue is that the s3 proxy code sets image['status'] to queued, but when it updates the status, it is actually updating image['properties']['image_state'].

From nova/image/s3.py (lines 153-157):

metadata.update({'disk_format': image_format,
                         'container_format': image_format,
                         'status': 'queued',
                         'is_public': True,
                         'properties': properties})

This updates the images table and sets the status to queued. Then later (in various places, but line 200 is where it is set to available):

 with open(unz_filename) as image_file:
                self.service.update(context, image_id, metadata, image_file)
            metadata['properties']['image_state'] = 'available'
            self.service.update(context, image_id, metadata)

Then finally the image is returned on line 207.

Checking the DB confirms these findings:

mysql> select * from image_properties where image_id = 4;
+----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
| id | image_id | name | value | created_at | updated_at | deleted_at | deleted |
+----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
| 21 | 4 | architecture | x86_64 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
| 18 | 4 | image_location | uec-publish-test/maverick-server-uec-amd64.img.manifest.xml | 2011-04-28 16:10:20 | NULL | NULL | 0 |
| 19 | 4 | image_state | available | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 |
| 17 | 4 | kernel_id | 3 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
| 20 | 4 | project_id | sandbox | 2011-04-28 16:10:20 | NULL | NULL | 0 |
+----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
5 rows in set (0.00 sec)

mysql> select * from images where id = 4;
+----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
| id | name | size | status | is_public | location | created_at | updated_at | deleted_at | deleted | disk_format | container_format | checksum |
+----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
| 4 | NULL | 1476395008 | queued | 1 | file:///var/lib/glance/images/4 | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 | ami | ami | 0bf9f38f48b0140cf84d3f26afd5666f |
+----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
1 row in set (0.00 sec)

Running:

update images set status = 'active' where id = 4;

or alteratively

update images set status = 'active' where status = 'queued'

would do it as well, but make sure that you are not actively uploading any images when you run that or you may end up breaking something :-)

The above should be a work to fix it, and hopefully a patch will be released soon :-)

I can duplicate this. I am still investigating why a new version of glance may have caused this to manifest itself, but from what I can tell, the issue is that the s3 proxy code sets image['status'] to queued, but when it updates the status, it is actually updating image['properties']['image_state'].

From nova/image/s3.py (lines 153-157):

metadata.update({'disk_format': image_format,
                         'container_format': image_format,
                         'status': 'queued',
                         'is_public': True,
                         'properties': properties})

This updates the images table and sets the status to queued. Then later (in various places, but line 200 is where it is set to available):

 with open(unz_filename) as image_file:
                self.service.update(context, image_id, metadata, image_file)
            metadata['properties']['image_state'] = 'available'
            self.service.update(context, image_id, metadata)

Then finally the image is returned on line 207.

Checking the DB confirms these findings:

mysql> select * from image_properties where image_id = 4;
+----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
| id | image_id | name | value | created_at | updated_at | deleted_at | deleted |
+----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
| 21 | 4 | architecture | x86_64 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
| 18 | 4 | image_location | uec-publish-test/maverick-server-uec-amd64.img.manifest.xml | 2011-04-28 16:10:20 | NULL | NULL | 0 |
| 19 | 4 | image_state | available | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 |
| 17 | 4 | kernel_id | 3 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
| 20 | 4 | project_id | sandbox | 2011-04-28 16:10:20 | NULL | NULL | 0 |
+----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
5 rows in set (0.00 sec)

mysql> select * from images where id = 4;
+----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
| id | name | size | status | is_public | location | created_at | updated_at | deleted_at | deleted | disk_format | container_format | checksum |
+----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
| 4 | NULL | 1476395008 | queued | 1 | file:///var/lib/glance/images/4 | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 | ami | ami | 0bf9f38f48b0140cf84d3f26afd5666f |
+----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
1 row in set (0.00 sec)

Running:

update images set status = 'active' where id = 4;

or alteratively

update images set status = 'active' where status = 'queued'

would do it as well, but make sure that you are not actively uploading any images when you run that or you may end up breaking something :-)

The above should be a work to fix it, and hopefully a patch will be released soon :-)

In glance change #125 in glance/registry/db/api.py an additional filter on status='active' was added. This would do it.

Thanks for the workaround, the fix should be pretty easy I imagine

Cheers,
   Graham

On Apr 28, 2011, at 12:10 PM, Kevin Bringard <email address hidden> wrote:

> Question #154460 on Glance changed:
> https://answers.launchpad.net/glance/+question/154460
>
> Status: Open => Answered
>
> Kevin Bringard proposed the following answer:
> I can duplicate this. I am still investigating why a new version of
> glance may have caused this to manifest itself, but from what I can
> tell, the issue is that the s3 proxy code sets image['status'] to
> queued, but when it updates the status, it is actually updating
> image['properties']['image_state'].
>
>> From nova/image/s3.py (lines 153-157):
>
>
> metadata.update({'disk_format': image_format,
> 'container_format': image_format,
> 'status': 'queued',
> 'is_public': True,
> 'properties': properties})
>
> This updates the images table and sets the status to queued. Then later
> (in various places, but line 200 is where it is set to available):
>
> with open(unz_filename) as image_file:
> self.service.update(context, image_id, metadata, image_file)
> metadata['properties']['image_state'] = 'available'
> self.service.update(context, image_id, metadata)
>
>
> Then finally the image is returned on line 207.
>
> Checking the DB confirms these findings:
>
>
> mysql> select * from image_properties where image_id = 4;
> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
> | id | image_id | name | value | created_at | updated_at | deleted_at | deleted |
> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
> | 21 | 4 | architecture | x86_64 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
> | 18 | 4 | image_location | uec-publish-test/maverick-server-uec-amd64.img.manifest.xml | 2011-04-28 16:10:20 | NULL | NULL | 0 |
> | 19 | 4 | image_state | available | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 |
> | 17 | 4 | kernel_id | 3 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
> | 20 | 4 | project_id | sandbox | 2011-04-28 16:10:20 | NULL | NULL | 0 |
> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
> 5 rows in set (0.00 sec)
>
>
> mysql> select * from images where id = 4;
> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
> | id | name | size | status | is_public | location | created_at | updated_at | deleted_at | deleted | disk_format | container_format | checksum |
> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
> | 4 | NULL | 1476395008 | queued | 1 | file:///var/lib/glance/images/4 | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 | ami | ami | 0bf9f38f48b0140cf84d3f26afd5666f |
> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
> 1 row in set (0.00 sec)
>
> Running:
>
> update images set status = 'active' where id = 4;
>
> or alteratively
>
> update images set status = 'active' where status = 'queued'
>
> would do it as well, but make sure that you are not actively uploading
> any images when you run that or you may end up breaking something :-)
>
> The above should be a work to fix it, and hopefully a patch will be
> released soon :-)
>
> --
> You received this question notification because you are a direct
> subscriber of the question.

Interesting. The filter might stop non-available images from showing up, but images that successfully register should be changed to active. This makes me think that some part of the decrypting process is actually failing. Unless there is some reason that the filter would stop the update code in s3.py from working.

Vish

On Apr 28, 2011, at 10:39 AM, Graham Hemingway wrote:

> Question #154460 on Glance changed:
> https://answers.launchpad.net/glance/+question/154460
>
> Graham Hemingway proposed the following answer:
> In glance change #125 in glance/registry/db/api.py an additional filter
> on status='active' was added. This would do it.
>
> Thanks for the workaround, the fix should be pretty easy I imagine
>
> Cheers,
> Graham
>
>
> On Apr 28, 2011, at 12:10 PM, Kevin Bringard
> <email address hidden> wrote:
>
>> Question #154460 on Glance changed:
>> https://answers.launchpad.net/glance/+question/154460
>>
>> Status: Open => Answered
>>
>> Kevin Bringard proposed the following answer:
>> I can duplicate this. I am still investigating why a new version of
>> glance may have caused this to manifest itself, but from what I can
>> tell, the issue is that the s3 proxy code sets image['status'] to
>> queued, but when it updates the status, it is actually updating
>> image['properties']['image_state'].
>>
>>> From nova/image/s3.py (lines 153-157):
>>
>>
>> metadata.update({'disk_format': image_format,
>> 'container_format': image_format,
>> 'status': 'queued',
>> 'is_public': True,
>> 'properties': properties})
>>
>> This updates the images table and sets the status to queued. Then later
>> (in various places, but line 200 is where it is set to available):
>>
>> with open(unz_filename) as image_file:
>> self.service.update(context, image_id, metadata, image_file)
>> metadata['properties']['image_state'] = 'available'
>> self.service.update(context, image_id, metadata)
>>
>>
>> Then finally the image is returned on line 207.
>>
>> Checking the DB confirms these findings:
>>
>>
>> mysql> select * from image_properties where image_id = 4;
>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>> | id | image_id | name | value | created_at | updated_at | deleted_at | deleted |
>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>> | 21 | 4 | architecture | x86_64 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> | 18 | 4 | image_location | uec-publish-test/maverick-server-uec-amd64.img.manifest.xml | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> | 19 | 4 | image_state | available | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 |
>> | 17 | 4 | kernel_id | 3 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> | 20 | 4 | project_id | sandbox | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>> 5 rows in set (0.00 sec)
>>
>>
>> mysql> select * from images where id = 4;
>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>> | id | name | size | status | is_public | location | created_at | updated_at | deleted_at | deleted | disk_format | container_format | checksum |
>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>> | 4 | NULL | 1476395008 | queued | 1 | file:///var/lib/glance/images/4 | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 | ami | ami | 0bf9f38f48b0140cf84d3f26afd5666f |
>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>> 1 row in set (0.00 sec)
>>
>> Running:
>>
>> update images set status = 'active' where id = 4;
>>
>> or alteratively
>>
>> update images set status = 'active' where status = 'queued'
>>
>> would do it as well, but make sure that you are not actively uploading
>> any images when you run that or you may end up breaking something :-)
>>
>> The above should be a work to fix it, and hopefully a patch will be
>> released soon :-)
>>
>> --
>> You received this question notification because you are a direct
>> subscriber of the question.
>
> --
> You received this question notification because you are a member of
> Glance Core, which is an answer contact for Glance.

I just tried setting the Glance DB entries to 'active' for my two images, but both images fail to create good instances. From Vish's comment it makes me think they might still be encrypted. I am continuing to explore.

Cheers,
   Graham

On Apr 28, 2011, at 1:45 PM, Vish Ishaya <email address hidden> wrote:

> Question #154460 on Glance changed:
> https://answers.launchpad.net/glance/+question/154460
>
> Vish Ishaya proposed the following answer:
> Interesting. The filter might stop non-available images from showing
> up, but images that successfully register should be changed to active.
> This makes me think that some part of the decrypting process is actually
> failing. Unless there is some reason that the filter would stop the
> update code in s3.py from working.
>
> Vish
>
> On Apr 28, 2011, at 10:39 AM, Graham Hemingway wrote:
>
>> Question #154460 on Glance changed:
>> https://answers.launchpad.net/glance/+question/154460
>>
>> Graham Hemingway proposed the following answer:
>> In glance change #125 in glance/registry/db/api.py an additional filter
>> on status='active' was added. This would do it.
>>
>> Thanks for the workaround, the fix should be pretty easy I imagine
>>
>> Cheers,
>> Graham
>>
>>
>> On Apr 28, 2011, at 12:10 PM, Kevin Bringard
>> <email address hidden> wrote:
>>
>>> Question #154460 on Glance changed:
>>> https://answers.launchpad.net/glance/+question/154460
>>>
>>> Status: Open => Answered
>>>
>>> Kevin Bringard proposed the following answer:
>>> I can duplicate this. I am still investigating why a new version of
>>> glance may have caused this to manifest itself, but from what I can
>>> tell, the issue is that the s3 proxy code sets image['status'] to
>>> queued, but when it updates the status, it is actually updating
>>> image['properties']['image_state'].
>>>
>>>> From nova/image/s3.py (lines 153-157):
>>>
>>>
>>> metadata.update({'disk_format': image_format,
>>> 'container_format': image_format,
>>> 'status': 'queued',
>>> 'is_public': True,
>>> 'properties': properties})
>>>
>>> This updates the images table and sets the status to queued. Then later
>>> (in various places, but line 200 is where it is set to available):
>>>
>>> with open(unz_filename) as image_file:
>>> self.service.update(context, image_id, metadata, image_file)
>>> metadata['properties']['image_state'] = 'available'
>>> self.service.update(context, image_id, metadata)
>>>
>>>
>>> Then finally the image is returned on line 207.
>>>
>>> Checking the DB confirms these findings:
>>>
>>>
>>> mysql> select * from image_properties where image_id = 4;
>>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>>> | id | image_id | name | value | created_at | updated_at | deleted_at | deleted |
>>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>>> | 21 | 4 | architecture | x86_64 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>>> | 18 | 4 | image_location | uec-publish-test/maverick-server-uec-amd64.img.manifest.xml | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>>> | 19 | 4 | image_state | available | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 |
>>> | 17 | 4 | kernel_id | 3 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>>> | 20 | 4 | project_id | sandbox | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>>> 5 rows in set (0.00 sec)
>>>
>>>
>>> mysql> select * from images where id = 4;
>>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>>> | id | name | size | status | is_public | location | created_at | updated_at | deleted_at | deleted | disk_format | container_format | checksum |
>>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>>> | 4 | NULL | 1476395008 | queued | 1 | file:///var/lib/glance/images/4 | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 | ami | ami | 0bf9f38f48b0140cf84d3f26afd5666f |
>>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>>> 1 row in set (0.00 sec)
>>>
>>> Running:
>>>
>>> update images set status = 'active' where id = 4;
>>>
>>> or alteratively
>>>
>>> update images set status = 'active' where status = 'queued'
>>>
>>> would do it as well, but make sure that you are not actively uploading
>>> any images when you run that or you may end up breaking something :-)
>>>
>>> The above should be a work to fix it, and hopefully a patch will be
>>> released soon :-)
>>>
>>> --
>>> You received this question notification because you are a direct
>>> subscriber of the question.
>>
>> --
>> You received this question notification because you are a member of
>> Glance Core, which is an answer contact for Glance.
>
> --
> You received this question notification because you are a direct
> subscriber of the question.

I set the db entry to active and it worked for me.

After checking the code, it appears that the code doesn't switch the state to active like i thought it did. I suppose we could just create the image (in s3.py) in the active state instead of the queued state.

Vish

On Apr 28, 2011, at 10:39 AM, Graham Hemingway wrote:

> Question #154460 on Glance changed:
> https://answers.launchpad.net/glance/+question/154460
>
> Graham Hemingway proposed the following answer:
> In glance change #125 in glance/registry/db/api.py an additional filter
> on status='active' was added. This would do it.
>
> Thanks for the workaround, the fix should be pretty easy I imagine
>
> Cheers,
> Graham
>
>
> On Apr 28, 2011, at 12:10 PM, Kevin Bringard
> <email address hidden> wrote:
>
>> Question #154460 on Glance changed:
>> https://answers.launchpad.net/glance/+question/154460
>>
>> Status: Open => Answered
>>
>> Kevin Bringard proposed the following answer:
>> I can duplicate this. I am still investigating why a new version of
>> glance may have caused this to manifest itself, but from what I can
>> tell, the issue is that the s3 proxy code sets image['status'] to
>> queued, but when it updates the status, it is actually updating
>> image['properties']['image_state'].
>>
>>> From nova/image/s3.py (lines 153-157):
>>
>>
>> metadata.update({'disk_format': image_format,
>> 'container_format': image_format,
>> 'status': 'queued',
>> 'is_public': True,
>> 'properties': properties})
>>
>> This updates the images table and sets the status to queued. Then later
>> (in various places, but line 200 is where it is set to available):
>>
>> with open(unz_filename) as image_file:
>> self.service.update(context, image_id, metadata, image_file)
>> metadata['properties']['image_state'] = 'available'
>> self.service.update(context, image_id, metadata)
>>
>>
>> Then finally the image is returned on line 207.
>>
>> Checking the DB confirms these findings:
>>
>>
>> mysql> select * from image_properties where image_id = 4;
>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>> | id | image_id | name | value | created_at | updated_at | deleted_at | deleted |
>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>> | 21 | 4 | architecture | x86_64 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> | 18 | 4 | image_location | uec-publish-test/maverick-server-uec-amd64.img.manifest.xml | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> | 19 | 4 | image_state | available | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 |
>> | 17 | 4 | kernel_id | 3 | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> | 20 | 4 | project_id | sandbox | 2011-04-28 16:10:20 | NULL | NULL | 0 |
>> +----+----------+----------------+-------------------------------------------------------------+---------------------+---------------------+------------+---------+
>> 5 rows in set (0.00 sec)
>>
>>
>> mysql> select * from images where id = 4;
>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>> | id | name | size | status | is_public | location | created_at | updated_at | deleted_at | deleted | disk_format | container_format | checksum |
>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>> | 4 | NULL | 1476395008 | queued | 1 | file:///var/lib/glance/images/4 | 2011-04-28 16:10:20 | 2011-04-28 16:10:44 | NULL | 0 | ami | ami | 0bf9f38f48b0140cf84d3f26afd5666f |
>> +----+------+------------+--------+-----------+---------------------------------+---------------------+---------------------+------------+---------+-------------+------------------+----------------------------------+
>> 1 row in set (0.00 sec)
>>
>> Running:
>>
>> update images set status = 'active' where id = 4;
>>
>> or alteratively
>>
>> update images set status = 'active' where status = 'queued'
>>
>> would do it as well, but make sure that you are not actively uploading
>> any images when you run that or you may end up breaking something :-)
>>
>> The above should be a work to fix it, and hopefully a patch will be
>> released soon :-)
>>
>> --
>> You received this question notification because you are a direct
>> subscriber of the question.
>
> --
> You received this question notification because you are a member of
> Glance Core, which is an answer contact for Glance.

Should we file a bug to get this fix in?
Thanks. G

Done: https://bugs.launchpad.net/nova/+bug/773388