apt does not accept globs and regexes in some cases

Bug #1872200 reported by Torsten Bronger
34
This bug affects 5 people
Affects Status Importance Assigned to Milestone
apt (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
Fix Released
Undecided
Unassigned
Groovy
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
Users can't use * wildcards anymore in focal, except by accident in apt list.

For apt list, we now start restricting wildcard syntax to the same syntax install now accepts, and at the same time we remove the restrictions on which patterns are accepted (which only accepted patterns starting in ~ or ?), so !~napt now works, and does not accidentally match as a wildcard.

[Test case]

Test that * wildcards work for both install and list, and test that a ? wildcard does not.

We included autopkgtests for those:

#### BEGIN TESTS #####

# * wildcards should still work
testsuccessequal "Listing...
automatic1/now 1.0 i386 [installed,local]
automatic2/now 1.0 i386 [installed,local]" apt list 'automatic*'

testfailureequal "Reading package lists...
Building dependency tree...
Reading state information...
Note, selecting 'automatic1' for glob 'automatic*'
Note, selecting 'automatic2' for glob 'automatic*'
automatic1 is already the newest version (1.0).
automatic1 set to manually installed.
automatic2 is already the newest version (1.0).
automatic2 set to manually installed.
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 broken : Depends: does-not-exist but it is not installable
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution)." apt install -s 'automatic*'

# other wildcards should fail

testfailureequal "Listing...
E: input:0-10: error: Expected pattern
   automatic?
   ^^^^^^^^^^" apt list 'automatic?'

testfailureequal "Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package automatic?" apt install -s 'automatic?'

#### END TESTS #####

Also it might be worth checking that apt list !~napt works. This used to produce an empty list, as it was accidentally matched as a wildcard and produced no result - now it produces every package whose name does not contain "apt".

[Regression potential]
The changes only affect interactive users, as apt(8) is not stable for in-script use, hence they can fix up their command-line if it stops working for them (though, really, more should work now).

No scripts will be broken :)

[Squashed in changes]
The SRU also fixes potential build failures by correctly prefxing nullptr_t with the std:: namespace, and includes updated Dutch documentation.

[Original bug report]
Observed with Ubuntu 20.04 Beta.

apt remove 'mypackage*' does not remove all installed packages starting with “mypackage”. Instead:

$ sudo apt remove 'mypackage*'
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package mypackage*

However:

$ sudo apt list --installed 'mypackage*'
Listing... Done
mypackage-data-v1/focal,focal,now 0.3.2-5build1 all [installed,automatic]
mypackage1/focal,now 0.3.2-5build1 amd64 [installed]

Revision history for this message
Julian Andres Klode (juliank) wrote :

Hmm seems I missed disabling that in apt list, good catch.

Revision history for this message
Torsten Bronger (bronger) wrote :

You mean, this feature is removed from apt?

Revision history for this message
Julian Andres Klode (juliank) wrote :

glob actually needs to come back to some extend (* only), but otherwise yes, read the NEWS.

Revision history for this message
Torsten Bronger (bronger) wrote :

Thank you. FWIW, the man page also needs to be updated then.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apt (Ubuntu):
status: New → Confirmed
Revision history for this message
Gunnar Hjalmarsson (gunnarhj) wrote :

I stumbled upon this when trying to install gnome-user-docs with all the available languages:

$ sudo apt install gnome-user-docs*
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package gnome-user-docs*

So far I can do what I want by using apt-get:

sudo apt-get install gnome-user-docs*

But I read in NEWS that the change is planned to propagate to apt-* ... Is there no less intrusive way to deal with package names such as g++?

Revision history for this message
Doug Smythies (dsmythies) wrote :

@Gunnar : Thank you for the workaround. It solved my problem.

Revision history for this message
Jakub Vaněk (vanek-jakub4) wrote :

Luckily, regular expression continue to work - they just have to start with ^ or end with $. Therefore, the command above could be rewritten as:

sudo apt install ^gnome-user-docs.*

While using the ^ character stops bash-completion from working, it is still much better than no pattern matching at all.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Yes, wildcard matching went away, I just re-added wildcard matching for "*" wildcards, as promised; and also changed apt list to reject unsupported wildcards (any other special character, we do not support ? or [] stuff).

@Jakub The "no pattern matching at all" I had to LOL a bit about. Because apt 2/1.9 introduces patterns, see apt-patterns(8). For example,

apt remove ?name(mypackage)?installed

apt remove ?obsolete

apt list ?essential

There's quite a few of them, and it's much more powerful than just matching on names.

Changed in apt (Ubuntu):
status: Confirmed → Fix Committed
description: updated
Changed in apt (Ubuntu Focal):
status: New → Triaged
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 2.1.1

---------------
apt (2.1.1) unstable; urgency=medium

  [ David Kalnischkies ]
  * Allow aptitude to MarkInstall broken packages via FromUser
  * Drop nowrap from po4a --porefs as it is no longer supported
  * Use "po4a --porefs file" instead of undocumented compat noline

  [ Artur Grącki ]
  * Fix typo in Polish translation of --help messages

 -- Julian Andres Klode <email address hidden> Fri, 08 May 2020 18:02:53 +0200

Changed in apt (Ubuntu Groovy):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Torsten, or anyone else affected,

Accepted apt into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.0.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in apt (Ubuntu Focal):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Doug Smythies (dsmythies) wrote :

I tested apt version 2.0.3. This:

sudo apt install gnome-user-docs* gnome-getting-started-docs*

whereas it did not before, and I had to use Gunnar's workaround from comment #6 above.

tags: added: verification-done-focal
removed: verification-needed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (apt/2.0.3)

All autopkgtests for the newly accepted apt (2.0.3) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.11-0ubuntu27 (amd64)
reprotest/0.7.14 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Chris Johnston (cjohnston) wrote :

Tested similar to Doug with successful results.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Julian Andres Klode (juliank) wrote :

FWIW, The upload has been superceded by a security update, and will need revalidation when 2.0.4 is uploaded.

tags: added: block-proposed-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote :

All autopkgtests for the newly accepted apt (2.0.3) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

apport/2.20.11-0ubuntu27 (amd64)
reprotest/0.7.14 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Proposed package removed from archive

The version of apt in the proposed pocket of Focal that was purported to fix this bug report has been removed because the bugs that were to be fixed by the upload were not verified in a timely (105 days) fashion.

Changed in apt (Ubuntu Focal):
status: Fix Committed → Won't Fix
Changed in apt (Ubuntu Focal):
status: Won't Fix → Triaged
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Torsten, or anyone else affected,

Accepted apt into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/2.0.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in apt (Ubuntu Focal):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-focal
removed: verification-done verification-done-focal
Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (apt/2.0.4)

All autopkgtests for the newly accepted apt (2.0.4) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

gcc-10/10.2.0-5ubuntu1~20.04 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#apt

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Julian Andres Klode (juliank) wrote :

autopkgtest has passed for 2.0.4, apt list '!~napt' also works with it. GCC failure was unrelated cloud problem on amd64.

tags: added: verification-done verification-done-focal
removed: block-proposed-focal verification-needed verification-needed-focal
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for apt has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 2.0.4

---------------
apt (2.0.4) focal; urgency=medium

  [ Julian Andres Klode ]
  * Merge 2.0.2ubuntu0.1 and 2.0.2ubuntu0.2 security updates with 2.0.3
    release.
  * pkgnames: Correctly set the default for AllNames to false, and do not
    exclude virtual packages if --all-names is specified (LP: #1876495)
  * Remove expired domain that became nsfw from debian/changelog
  * patterns: Terminate short pattern by ~ and ! (LP: #1911676)
  * Improve immediate configuration handling (LP: #1871268)
    - Do not immediately configure m-a: same packages in lockstep
    - Ignore failures from immediate configuration. This does not change the
      actual installation ordering - we never passed the return code to the
      caller and installation went underway anyway if it could be ordered at a
      later stage, this just removes spurious after-the-fact errors.

  [ JCGoran ]
  * Fix "extended_states" typo in apt-mark(8) (Closes: #969086)

apt (2.0.3) focal; urgency=medium

  [ Frans Spiesschaert ]
  * Dutch manpages translation update (Closes: #956313)

  [ David Kalnischkies ]
  * Add correct std namespace to nullptr_t

  [ Julian Andres Klode ]
  * Reinstate * wildcards (Closes: #953531) (LP: #1872200)
  * apt list: Fix behavior of regex vs fnmatch vs wildcards
  * debian/gbp.conf: We're in the 2.0.y branch
  * CI: Test on ubuntu:focal, not debian:unstable

 -- Julian Andres Klode <email address hidden> Thu, 14 Jan 2021 12:49:00 +0100

Changed in apt (Ubuntu Focal):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Related questions

Remote bug watches

Bug watches keep track of this bug in other bug trackers.