[SRU] ubuntu-advantage-tools (27.0.2 -> 27.1) Xenial, Bionic, Focal, Groovy, Hirsute

Bug #1929597 reported by Grant Orndorff
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ubuntu-advantage-tools (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Focal
Fix Released
Undecided
Unassigned
Groovy
Fix Released
Undecided
Unassigned
Hirsute
Fix Released
Undecided
Unassigned
Impish
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
This release provides fixes to numerous issues encountered and reported by users against the recent uatools 27.0.2 release. In general, there were situations or phrasing of messages were causing users confusion. For instance, improve error messaging if trying to enable livepatch with an old snapd installed (LP: #1927329) and noise in logs from (non-critical) segfault messages (LP: #1929123). In addition, this includes some refinements to how we communicate product availability based on what the user already has installed.

Below are the notable high-level changes in this SRU:

   * avoid segfault when comparing null Apt file origin to esm (LP: #1929123)
   * make cis service non-beta
   * livepatch: alert when snapd does not have wait cmd (LP: #1927329)
   * Do not show beta services in autocomplete (GH: #1594)
   * Block enabling fips if fips-updates once enabled (GH: #1600)
   * pro: obtain AWS IMDSv2 API token before trying to grab pkcs7 doc (GH: #1608)
   * Several tweaks to output and messages

Finally, these are the links for the notable github issues we are solving in this release:

https://github.com/canonical/ubuntu-advantage-client/issues/1600
https://github.com/canonical/ubuntu-advantage-client/issues/1594
https://github.com/canonical/ubuntu-advantage-client/issues/1608

See the [Changelog] section below for a full list of fixes and other changes.

[Test Case]
The following development and SRU process was followed:
https://wiki.ubuntu.com/UbuntuAdvantageToolsUpdates

The ubuntu-advantage-tools team will be in charge of attaching the
artifacts and console output of the appropriate run to the bug.
ubuntu-advantage-tools team members will not mark ‘verification-done’
until this has happened.

* Automated Test Results
Attached in a comment below.

* Manual Test Results
<TODO: attach manual upgrade path test from previous LTS to current -proposed release>

[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned integration tests are attached to this bug.

This release mostly touches various messages output by ua-tools. If anyone is relying on particular output of ua-tools (which they shouldn't), then these changes will affect them.

We overrode some lintian errors in this release. It is possible that if we overrode them too liberally, it could prevent us from getting legitimate lintian errors in the future.

[Discussion]
This is a follow up release to 27 that addresses the issues discovered and reported after that release. It does not introduce any major features. The closest thing to a "new feature" is removing beta status from the "cis" service, but that is not so much adding a new feature as it is making an existing feature more visible.

[Changelog]
ubuntu-advantage-tools (27.1) impish; urgency=medium

  * d/control:
    - specify debianutils min version
  * d/changelog:
    - fix lintian typos amend and redact incorrect 27.0 entry (GH: #1624)
  * lintian:
    - override ubuntu-advantage-pro wanted-by-target cloud-init
    - override xenial specific errors
    - rename package-specific overrides for pro vs tools
  * New upstream release 27.1:
    - apt-hook:
      + avoid segfault when comparing null Apt file origin to esm
        (LP: #1929123)
      + avoid wrapping static message formats at 80 chars
      + update go build flags based on lintian warnings (GH: #1626)
      + only add newlines for MOTD if message file length is non-zero
    - attach: do not print contract name if empty
    - autocomplete: Do not show beta services in autocomplete (GH: #1594)
    - cis:
      + make service non-beta
      + post enable message pointing to docs
      + update cis help url
    - docs: update releases.md per SRU review feedback on branch structuring
    - enable: correct messaging for beta service (GH: #1588)
    - errors: print a more helpful message when ssl fails (GH: #1618)
    - fips:
      + Block enabling fips if fips-updates once enabled (GH: #1600)
      + Update output of fips commands (GH: #1631)
    - livepatch: alert when snapd does not have wait cmd (LP: #1927329)
    - logging: remove tracebacks for UserFacingErrors (GH: #1586)
    - messaging:
      + Infra and Apps messaging is mutually exclusive (GH: #1573)
      + point to u.com/16-04 instead of u.com/advantage on ESM (GH: #1584)
      + separate _remove_msg_template. emit no warranty on infra disabled
    - pro: obtain AWS IMDSv2 API token before trying to grab pkcs7 doc
      (GH: #1608)
    - status: do not show info if not on contract (GH: #1592)
    - tests:
      + drop trusty specific tests
      + fix mock for handle_message_operations
      + fix motd message for bionic (GH: #1615)
      + integration tests for hirsute and groovy
      + manual test for trusty upgrade to xenial
      + reboot after dist-upgrade for upgrade test
      + test enabling CIS on focal (GH: #1582)
      + update messages in integration tests (GH: #1635)
      + use proposed pocket on xenial upgrade test
    - jenkins:
      + add pytest runs for xenial and bionic
      + run focal lxd integration tests

description: updated
description: updated
description: updated
Revision history for this message
Grant Orndorff (orndorffgrant) wrote :

Integration test output attached.

description: updated
description: updated
Chad Smith (chad.smith)
description: updated
description: updated
description: updated
Changed in ubuntu-advantage-tools (Ubuntu Impish):
status: New → Fix Committed
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Grant, or anyone else affected,

Accepted ubuntu-advantage-tools into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.1~21.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-advantage-tools (Ubuntu Hirsute):
status: New → Fix Committed
tags: added: verification-needed verification-needed-hirsute
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Grant, or anyone else affected,

Accepted ubuntu-advantage-tools into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.1~20.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-advantage-tools (Ubuntu Groovy):
status: New → Fix Committed
tags: added: verification-needed-groovy
Changed in ubuntu-advantage-tools (Ubuntu Focal):
status: New → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Grant, or anyone else affected,

Accepted ubuntu-advantage-tools into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.1~20.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-advantage-tools (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Grant, or anyone else affected,

Accepted ubuntu-advantage-tools into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.1~18.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Grant, or anyone else affected,

Accepted ubuntu-advantage-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ubuntu-advantage-tools/27.1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in ubuntu-advantage-tools (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (ubuntu-advantage-tools/27.1~20.04.1)

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.1~20.04.1) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.1~20.04.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (ubuntu-advantage-tools/27.1~20.10.1)

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.1~20.10.1) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.1~20.10.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (ubuntu-advantage-tools/27.1~21.04.1)

All autopkgtests for the newly accepted ubuntu-advantage-tools (27.1~21.04.1) for hirsute have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-advantage-tools/27.1~21.04.1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/hirsute/update_excuses.html#ubuntu-advantage-tools

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :

We are now adding all of the tests that were performed using the package in proposed.
There is only one test that is not passing:

Attached FIPS upgrade across LTS releases

On this test we enable FIPS on xenial and then upgrade the machine to bionic. The reason that it is failing is because we enable the propose pocket before doing the do-release-upgrade operation.
We can see on LP: #21551181 that shim-signed never built properly in -proposed.

dist-upgrade removes the shim-signed package and any subsequent reboot got us an UEFI "Access Denied" message on the default boot option because we didn't have the proper secureboot signed bits in place.

Now, if we run that same test with the package on staging ppa (which have the exact same ubuntu-advantage-tools package as the one proposed):
https://launchpad.net/~ua-client/+archive/ubuntu/staging/

We can see that the test pass without an issue. Because of that, we have also added this test output into the other test results.

Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :
tags: added: verification-done verification-done-bionic verification-done-focal verification-done-groovy verification-done-hirsute verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-groovy verification-needed-hirsute verification-needed-xenial
Revision history for this message
Chris Halse Rogers (raof) wrote :

Has anyone looked at the autopkgtest failures? They're all failure to install dependencies on i386. It's possible that this is expected, but it's unclear, and the tests should either be explicitly ignored (with justification) or fixed. Or both, if they're expected to fail - ignored for this upload, and then removed in a subsequent upload.

Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :

Hi Chris, the i386 build is expected to fail on focal.
Is there a way for us to officially ignore those tests for i386 on focal ?

Revision history for this message
Grant Orndorff (orndorffgrant) wrote :

Yep we actually aren't supporting i386 on focal or anything later than focal. I'm also not sure how to officially ignore those tests or have them removed for future uploads though.

Revision history for this message
Lucas Albuquerque Medeiros de Moura (lamoura) wrote :

This test failure is expected on all releases after Focal, since we dropped 32-bit i386 support on those releases. Therefore, we are not producing the binary package for i386 on these releases.
Because of that, we believe that we can ignore those tests and treat them not as a regression.

Please add hint on Focal, Groovy and Hirsute

Revision history for this message
Robie Basak (racb) wrote :

Do you have any way to verify that your automated testing did actually test the correct version of the package that is in proposed for each series please? This has bitten us in the past when verification "passed" but with the wrong version, and a regression hit the updates pocket.

The usual way to ensure this is to have the automation output the package version it is testing against, as required by the "mentioning the version of the package you tested" part of the automatic call for testing added to the bug when the package is accepted into proposed. But I don't see this in your test output anywhere.

Apart from that, this looks good to go. I've added hints for i386 in Focal, Groovy and Hirsute.

Revision history for this message
Brian Murray (brian-murray) wrote :

I'm removing the verification tags per Robie's question in comment #21 so that this will not accidentally get released.

tags: added: verification-needed verification-needed-bionic verification-needed-focal verification-needed-groovy verification-needed-hirsute verification-needed-xenial
removed: verification-done verification-done-bionic verification-done-focal verification-done-groovy verification-done-hirsute verification-done-xenial
Revision history for this message
Grant Orndorff (orndorffgrant) wrote :

First of all, sorry for the radio silence on our end. We decided to rerun the tests and forgot to reply here that we were doing so.

As discussed out-of-band: for future SRU verification, we plan on adding specific output to our integration tests that say which version is being tested.

For this release, this is what we've done:

We tested and used this feature to run all of the integration tests with proposed enabled: https://github.com/canonical/ubuntu-advantage-client/pull/1673

You can verify that this was used in the new output attached because "enable_proposed = True" is logged in the "Config options" section at the top of each file.

To further ensure that not only was proposed enabled, but also that the correct version was tested, we ran additional ad-hoc tests using the enable_proposed feature. These additional tests have extra logging that does output the version tested by running `ua version` on the machine. These are named "behave-lxd-YY.MM-ua-version.txt" in the attached tar. The extra logging looks like this:

2021-06-17 10:36:07,624 - ua-ci - INFO - Validating ubuntu-advantage-tools v.27.1~20.04.1

We believe that this is sufficient evidence to say that our "enable_proposed" feature is using the correct version of ubuntu-advantage-tools for all of our integration tests. As a result, I'm changing the tags back to verification-done.

We recognize that this is not a great solution and plan on improving it for our next release.

tags: added: verification-done verification-done-bionic verification-done-focal verification-done-groovy verification-done-hirsute verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-groovy verification-needed-hirsute verification-needed-xenial
Revision history for this message
Robie Basak (racb) wrote :

Thank you for working on this.

> # KNOWN FAILURE: staging commands failed due to livepatch.staging.canonical.com being outage with 503s during latest test run, waiting on IS for firewall fixes.
> # this resulted in the known 5 test failures

Noted. Given that this is caused elsewhere from this update and that you've already got passes previously albeit without version confirmation, I think this is good enough.

Changed in ubuntu-advantage-tools (Ubuntu Impish):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 27.1~21.04.1

---------------
ubuntu-advantage-tools (27.1~21.04.1) hirsute; urgency=medium

  * Backport new upstream release: (LP: #1929597) to hirsute

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 16:12:08 -0300

Changed in ubuntu-advantage-tools (Ubuntu Hirsute):
status: Fix Committed → Fix Released
Revision history for this message
Robie Basak (racb) wrote : Update Released

The verification of the Stable Release Update for ubuntu-advantage-tools has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 27.1~20.10.1

---------------
ubuntu-advantage-tools (27.1~20.10.1) groovy; urgency=medium

  * Backport new upstream release: (LP: #1929597) to groovy

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 16:05:18 -0300

Changed in ubuntu-advantage-tools (Ubuntu Groovy):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 27.1~20.04.1

---------------
ubuntu-advantage-tools (27.1~20.04.1) focal; urgency=medium

  * Backport new upstream release: (LP: #1929597) to focal

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 15:58:37 -0300

Changed in ubuntu-advantage-tools (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 27.1~18.04.1

---------------
ubuntu-advantage-tools (27.1~18.04.1) bionic; urgency=medium

  * Backport new upstream release: (LP: #1929597) to bionic

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 15:50:28 -0300

Changed in ubuntu-advantage-tools (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubuntu-advantage-tools - 27.1~16.04.1

---------------
ubuntu-advantage-tools (27.1~16.04.1) xenial; urgency=medium

  * Backport new upstream release: (LP: #1929597) to xenial

 -- Lucas Moura <email address hidden> Thu, 27 May 2021 15:40:20 -0300

Changed in ubuntu-advantage-tools (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.