Comment 23 for bug 1929597

Revision history for this message
Grant Orndorff (orndorffgrant) wrote :

First of all, sorry for the radio silence on our end. We decided to rerun the tests and forgot to reply here that we were doing so.

As discussed out-of-band: for future SRU verification, we plan on adding specific output to our integration tests that say which version is being tested.

For this release, this is what we've done:

We tested and used this feature to run all of the integration tests with proposed enabled: https://github.com/canonical/ubuntu-advantage-client/pull/1673

You can verify that this was used in the new output attached because "enable_proposed = True" is logged in the "Config options" section at the top of each file.

To further ensure that not only was proposed enabled, but also that the correct version was tested, we ran additional ad-hoc tests using the enable_proposed feature. These additional tests have extra logging that does output the version tested by running `ua version` on the machine. These are named "behave-lxd-YY.MM-ua-version.txt" in the attached tar. The extra logging looks like this:

2021-06-17 10:36:07,624 - ua-ci - INFO - Validating ubuntu-advantage-tools v.27.1~20.04.1

We believe that this is sufficient evidence to say that our "enable_proposed" feature is using the correct version of ubuntu-advantage-tools for all of our integration tests. As a result, I'm changing the tags back to verification-done.

We recognize that this is not a great solution and plan on improving it for our next release.