apt-get source the pinned versions, not the latest available ones

Ok this may actually be a testing issue. We are aledgedly testing the "current" version of lxc against a specific kernel version:

adt-run [11:23:50]: host juju-prod-ues-proposed-migration-machine-5; command line: /home/ubuntu/autopkgtest/runner/adt-run --output-dir /tmp/adt-run.out.yaBjME --timeout-copy=6000 --setup-commands 'if grep -q trusty /etc/lsb-release; then apt-get install -y build-essential; fi' --apt-pocket=proposed=src:linux-meta --apt-upgrade lxc --env=ADT_TEST_TRIGGERS=linux-meta/3.13.0.70.76 --setup-commands 'apt-get install -y linux-image linux-headers || apt-get install -y linux-image-generic linux-headers-generic' --setup-commands /home/ubuntu/autopkgtest/setup-commands/cloud-vm-setup --- ssh -s /home/ubuntu/autopkgtest/ssh-setup/nova -- --flavor autopkgtest --image ubuntu/ubuntu-trusty-14.04-amd64-server-20151117-disk1.img --name adt-trusty-amd64-lxc-20151118-112350 --keyname=testbed-juju-prod-ues-proposed-migration-machine-5 --net-id=net_ues_proposed_migration -e ''"'"'http_proxy=http://squid.internal:3128'"'"'' -e ''"'"'https_proxy=http://squid.internal:3128'"'"'' -e ''"'"'no_proxy=127.0.0.1,127.0.1.1,localhost,localdomain,novalocal,internal,archive.ubuntu.com,security.ubuntu.com,ddebs.ubuntu.com'"'"'' --mirror=http://ftpmaster.internal/ubuntu

And yet we have found, downloaded, and are testing the backports pocket version of lxc:

dpkg-source: warning: failed to verify signature on ./lxc_1.1.5-0ubuntu2~ubuntu14.04.1.dsc
Get:1 http://ftpmaster.internal/ubuntu/ trusty-backports/main lxc 1.1.5-0ubuntu2~ubuntu14.04.1 (dsc) [2,458 B]
Get:2 http://ftpmaster.internal/ubuntu/ trusty-backports/main lxc 1.1.5-0ubuntu2~ubuntu14.04.1 (tar) [849 kB]
Get:3 http://ftpmaster.internal/ubuntu/ trusty-backports/main lxc 1.1.5-0ubuntu2~ubuntu14.04.1 (diff) [109 kB]
adt-run [11:27:42]: testing package lxc version 1.1.5-0ubuntu2~ubuntu14.04.1

But we test without backports enabled. I suspect we should not be finding this backports version for this test.

This appears to be an infrastructure issue, handing off to testing.

This appears to be related to the changes applied to the cloud images under bug: #1177432.

Indeed this is fallout from bug 1177432, where cloud-init changed to add -backports (even on stables).

I mitigated this by adding this beauty to the worker.conf's setup commands:

    sed -i '/^deb.*-backports/d' /etc/apt/sources.list `ls /etc/apt/sources.list.d/*2>/dev/null || true`

A more elegant fix would be to tell apt-get source to somehow respect the apt pinning, i. e. get the source from the series/pocket that it would install the binaries for. But this doesn't seem to be easy. We can't explicitly say "apt-get source foo/trusty" as we sometimes (but not always) have -updates, -proposed, PPAs, etc. We also shouldn't say "apt-get source foo=<version_of_trigger>" as we would then fail if there was another upload of foo in between britney ran the last time and the test started.

trusty-backports has "NotAutomatic: yes"

  http://archive.ubuntu.com/ubuntu/dists/trusty-backports/Release

Which means that, for binary packages at least, you have to explicitly ask for them from -backports (or have that forced on you by package relations). Does this not apply to source packages too?

If not, this might be an apt bug / feature request too.

(Just checked...

ubuntu@test:~$ apt-cache policy lxc
lxc:
  Installed: (none)
  Candidate: 1.0.7-0ubuntu0.10 # correctly respects NotAutomatic
  Version table:
     1.1.5-0ubuntu3~ubuntu14.04.1 0
        100 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-backports/main amd64 Packages
     1.0.7-0ubuntu0.10 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
     1.0.7-0ubuntu0.7 0
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
     1.0.3-0ubuntu3 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
ubuntu@test:~$ apt-get source lxc
[…]
Get:1 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-backports/main lxc 1.1.5-0ubuntu3~ubuntu14.04.1 (dsc) [2458 B]

So no, it doesn't.)

Generalizing bug title. This also applies to testing normal packages where trigger != test -- in that case the test will be taken from -proposed, but will run against the -release binary packages.

This is quite a nuisance to do with current apt, but this should do it: http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/commit/?id=7429f79124

This bug was fixed in the package autopkgtest - 3.19.1

---------------
autopkgtest (3.19.1) unstable; urgency=medium

  * adt-virt-qemu: If the executed command exits with 255, translate that to
    253 in the auxverb wrapper, as 255 is the exit code for failures of the
    auxverb itself. This avoids considering ordinary test failures as
    temporary testbed failures. This makes the QEMU runner behave the same way
    as the container ones.
  * tests/adt-run: Factorize common tests into superclasses.
  * doc/README.package-tests.rst: Point out that "Test-Command:" is also
    useful for running the same test script under different interpreters or
    different dependencies. (Closes: #809442)
  * Set summary_stream to None after closing it, to avoid trying to write
    late error messages from cleanup into it.
  * Gracefully handle failure to kill a timed out process.
  * ssh-setup/nova: Use image UUID instead of its name for "nova boot". This
    avoids failures if multiple images have the same name. (LP: #1524216)
  * adt-build-{lxc,lxd}: Run testbed setup scripts also if they are not
    executable. Thanks Antonio Terceiro! (Closes: #809917)
  * setup-commands/setup-testbed: Run apt-get autoremove when builing images.
  * setup-commands/setup-testbed: Handle options in sources.list. Thanks
    Antonio Terceiro!
  * tools/adt-build-lxd: Fix error message if apt does not have a proxy set.
  * setup-commands/setup-testbed: Install linux-headers-generic if there is no
    kernel (for containers), to make DKMS tests work when not being triggered
    by a kernel. (LP: #1531488)
  * tools/adt-build-lxc: Fix removal of LXC caches on btrfs.
  * tools/adt-build-lxc: Use "-B best" for lxc-create, for better performance
    on btrfs or LVM.
  * tools/adt-build-lxc: Fix hostname for updating containers to not end in
    ".new".
  * adt-virt-lxc: Opportunistically try lxc-clone with "-B btrfs", and fall
    back to default backing store if it fails. lxc-clone should do this by
    itself, but currently doesn't (LP #1532125).
  * adt-virt-lxc: Don't fail on stderr of lxc-start, it sometimes prints
    warnings.
  * When using --apt-source with --apt-pocket= with an explicit package list,
    let apt-get source download the source version that matches the pinning.
    This avoids running tests from -proposed against packages in -release.
    (LP: #1517426)
  * setup-commands/setup-testbed: Don't purge perl and python2.7. This works
    in sid and xenial, but is too greedy for older releases.

 -- Martin Pitt <email address hidden> Tue, 12 Jan 2016 09:23:16 +0100