apt-get update hangs when apt-transport-https is not installed

Bug #1762766 reported by Graham Leggett
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apt (Debian)
Fix Released
Unknown
apt (Ubuntu)
Fix Released
Undecided
Unassigned
Trusty
Triaged
Undecided
Unassigned
Xenial
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
Two or more sources with an unknown protocol cause apt update to hang. For example, when using the https method on a system without integrated https support and without apt-transport-https installed.

[Test case]
Add two or more sources with an unknown protocol and run update

[Regression potential]
This changes the code to only register a method that actually exists. I don't see any potential for a regression here, but of course, if there were any, it would be related to not finding other methods.

[Original bug report]
When "apt-get update" is run on a docker container running Ubuntu v16.04 and containing an additional apt source repository hosted on an https webserver, the "apt-get update" command hangs.

The hang happens after connections to http ubuntu hosts are complete, and apt-get remains stuck on "Working" at 0%. Removing the sources file for the https repo causes apt-get to complete normally.

The source file contains 4 separate entries to 4 different repos on the same https server. When the source file is modified so that just *one* entry exists to one repo on the https server, we suddenly get a sensible error message that tells us that apt-transport-https needs to be installed.

Installing apt-transport-https into the docker container before adding the sources list to the https servers works around the problem and sanity returns.

Key notes:

- The use of docker isn't related to the bug, except that the docker image doesn't contain the apt-transport-https package whereas our cloud images do contain this package by default. This can give the impression that this is a docker bug when it's not.

- The hang in "apt-get update" seems to occur when the sources file contains more than one entry in the file. When just one entry exists in the file (and all other entries are commented out) a sensible error messages appears.

- We encountered this on a host that didn't support cut and paste, sorry :(

summary: - apt-get update hangs when apt-transport-https is no installed
+ apt-get update hangs when apt-transport-https is not installed
Changed in apt (Ubuntu):
status: New → Fix Released
Revision history for this message
Julian Andres Klode (juliank) wrote :

I'm not sure when we fixed that, but we did. So bionic works fine. Not sure about artful.

Changed in apt (Ubuntu Xenial):
status: New → Triaged
Revision history for this message
Julian Andres Klode (juliank) wrote :

Queued the fix for xenial in my repo for CI.

Changed in apt (Debian):
status: Unknown → Fix Released
Revision history for this message
Graham Leggett (minfrin-y) wrote :

Is it possible to backport this to trusty too? This bit us hard, and there are a lot of people out there posting this problem but with no solution.

Revision history for this message
Julian Andres Klode (juliank) wrote :

I think so, but I don't think it's worth pushing out an update just for that change. It's been broken since forever, so it's not like it's fixing a regression or something; so I'd probably wait for something else to attach it to.

Changed in apt (Ubuntu Trusty):
status: New → Triaged
Revision history for this message
Julian Andres Klode (juliank) wrote :

Or rather, I'm not sure it's worth it. I plan to look at other fixes in 1.6 soon, and see which of these we need for stable series.

Revision history for this message
Graham Leggett (minfrin-y) wrote :

In our case it burned a number of days of dev time, so this is definitely causing pain.

We've never seen this before because until docker, we have not encountered a system where apt-transport-https wasn't installed by default.

Changed in apt (Ubuntu Trusty):
status: Triaged → In Progress
status: In Progress → Confirmed
status: Confirmed → Triaged
Changed in apt (Ubuntu Xenial):
status: Triaged → In Progress
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Graham, or anyone else affected,

Accepted apt into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.2.27 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apt (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
description: updated
description: updated
Revision history for this message
Julian Andres Klode (juliank) wrote :

No package in proposed yet.

curl http://archive.ubuntu.com/ubuntu/dists/xenial-backports/main/binary-amd64/Packages.xz | xzcat | grep ^Package

does not show apt. odd.

Revision history for this message
Julian Andres Klode (juliank) wrote :

huh, why did it say backports? Me looks again, and sees the package.

Revision history for this message
Julian Andres Klode (juliank) wrote :

Verified now, despite me not changing my verification script. Odd.

with 1.2.26, it times out:

+ lxc exec lp1762766-xenial -- sh -c 'apt-cache policy apt libapt-pkg5.0 | grep Inst ; timeout 5s apt update'
  Installed: 1.2.26
  Installed: 1.2.26
0% [Working]+ lxc stop lp1762766-xenial

with 1.2.27, it works fine:

+ lxc exec lp1762766-xenial -- sh -c 'apt-cache policy apt libapt-pkg5.0 | grep Inst ; timeout 5s apt update'
  Installed: 1.2.27
  Installed: 1.2.27
Reading package lists... Done
E: The method driver /usr/lib/apt/methods/httpx could not be found.
E: The method driver /usr/lib/apt/methods/httpx could not be found.
E: Failed to fetch httpx://invalid.mirror.example.com/dists/bionic/InRelease
E: Failed to fetch httpx://invalid.mirror.example.com/dists/bionic-updates/InRelease
E: Some index files failed to download. They have been ignored, or old ones used instead.

tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for apt has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apt - 1.2.27

---------------
apt (1.2.27) xenial; urgency=medium

  [ David Kalnischkies ]
  * don't hang if multiple sources use unavailable method (Closes: 870675)
    (LP: #1762766)

  [ Julian Andres Klode ]
  * Fix lock counting in debSystem (LP: #1778547)
  * apt.conf.autoremove: Catch some new Ubuntu module packages (LP: #1778551)

 -- Julian Andres Klode <email address hidden> Tue, 26 Jun 2018 12:55:03 +0200

Changed in apt (Ubuntu Xenial):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.