adt-run fulfills dependencies with --built-tree contrary to the manpage

failure-case.log contains the output of "adt-run --built-tree facter-1.6.10 --- adt-virt-schroot saucy-amd64". Note that apt-get downloaded facter from the archive. From the manpage, I expect this not to happen.

In case it's relevant, my facter-1.6.10 is actually an unbuilt tree. But my test doesn't need the tree to be built, so I expected it to be passed to the test unchanged.

In the first case, "then it fulfils the facter test dependency from the archive. ", there is no bug. --built-tree merely means it will not take the locally built *.debs to fulfill dependencies, not that it won't install the test dependencies at all; the latter would not make any sense. This is indeed badly written in the manpage, so I'll adjust the documentation.

In the second case, I tried to reproduce that here and it did use the local .deb, not the deb from the archive. I'll clean up the documentation first and write some tests to cover all this.

I'm trying to understand how these arguments actually work, keeping my notes here. Probably not interesting to anyone else, but maybe for me for future reference.

unbuilt-tree
------------
--unbuilt-tree . --binary ../testpkg_1_all.deb → "build needed for binaries", builds package, installs adt built deb, ignores provided deb
-B .// ../testpkg_1_all.deb → no package build/build deps, fails to find testpkg

--binary ../testpkg_1_all.deb --unbuilt-tree . → build deps, builds package, installs built pkg, NOT given --binary
-B --binary ../testpkg_1_all.deb --unbuilt-tree . → no build, installs local testpkg

→ uses binaries built from the source by default (change with -B)
→ binaries for tests must come before source arg

source
------
../testpkg_1_all.deb ../testpkg_1.dsc → builds package, installs adt built deb
-B ../testpkg_1_all.deb ../testpkg_1.dsc → no build, installs provided deb
-B ../testpkg_1.dsc ../testpkg_1_all.deb → does not find testpkg.deb

→ equivalent to --unbuilt-tree, just with a .dsc

built-tree
----------
../testpkg_1_all.deb ./ → no build, installs local testpkg, pass

→ no binaries built from the source by default
→ binaries for tests must come before source arg

--source/--unbuilt-tree that get built, and --binary arguments cumulate debs via binaries.register(), and binaries.publish() then enables those. Thus --binary has to come before the source argument that has the tests to work, and specifying --binary with a built source makes no sense unless -B is being used.

Martin Pitt writes ("[Bug 1175557] Re: adt-run fulfills dependencies with --built-tree contrary to the manpage"):
> I'm trying to understand how these arguments actually work, keeping my
> notes here. Probably not interesting to anyone else, but maybe for me
> for future reference.

Do you want me to try to explain how I think they're supposed to
work ?

Ian.

I now added a selftest for --binary with --built-tree to capture this behaviour: http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=a456b

This accompanies the already existing one for --binary with --unbuilt-tree.

On Tue, Jan 28, 2014 at 04:22:06PM -0000, Ian Jackson wrote:
> Do you want me to try to explain how I think they're supposed to
> work ?

Please! I said this to pitti on IRC earlier:

11:10 <rbasak> pitti: incidentally, I find the manpage very confusing
and ambiguous whenever I'm trying to understand what the different
--built-tree, --unbuilt-tree, --source and --binary options will
actually do in terms of build or no build, where the build dependencies
will come from, and what test dependencies will be fulfilled and where
they will come from.

11:10 <rbasak> I'm not sure how to make that better, since I don't
really understand what the options do.

I clarified (hopefully) the manpage: http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=0dcce04

Please let me know if something is still unclear and you have further ideas how to improve it (or questions about the behaviour).

Hey Ian,

oh, you are listening here, nice! I'd appreciate if you could review http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=0dcce04 ; I'm fairly sure that this reflects how adt-run currently works, and that it mostly makes sense that way. If you think the behaviour is actually wrong, and the previous manpage was right, please let me know.

Robie Basak writes ("Re: [Bug 1175557] Re: adt-run fulfills dependencies with --built-tree contrary to the manpage"):
> On Tue, Jan 28, 2014 at 04:22:06PM -0000, Ian Jackson wrote:
> > Do you want me to try to explain how I think they're supposed to
> > work ?
>
> Please! I said this to pitti on IRC earlier:
>
> 11:10 <rbasak> pitti: incidentally, I find the manpage very confusing
> and ambiguous whenever I'm trying to understand what the different
> --built-tree, --unbuilt-tree, --source and --binary options will
> actually do in terms of build or no build, where the build dependencies
> will come from, and what test dependencies will be fulfilled and where
> they will come from.

So:

--unbuilt-tree and --dsc do the same thing: they, by default:
  * build the tree
  * use the binaries produced to satisfy any build-dependencies for
    sources further on the command line
  * use the binaries produced to satisfy any dependencies for tests
    further on the command line
  * run any applicable tests in that source package

It is possible to individually control these uses to which the tree is
put, though:

If --no-built-binaries, or --built-binaries-filter, or --binaries (or
--binaries-forbuilds or --binaries-fortests) is specified, then the
binaries aren't used for some of those purposes.

If --sources-no-tests is used, that source package's tests are not
run.

The relevant --*binaries* options are all the ones which precede a
particular --unbuilt-tree or --dsc, with later options of the same
kind overriding earlier ones.

I think, but am not sure, that there is an optimisation: if you
specify that the binaries from the source should not be used, and the
package's tests don't require the built source tree, then perhaps
adt-run won't build the source at all - since nothing needs it (and
you presumably weren't using adt-run for an archive rebuild test).

--binary specifies just a binary package to be used for tests which
appear subsequently on the adt-run command line. It will be used (by
default) to satisfy dependencies. --binary is subject to the effects
of preceding --binaries= and --binaries-forbuilds= and
--binaries-fortests= options. So if you say
   --binaries=install --binary foo.deb --binaries=auto [more stuff]
then foo.deb will be unconditionally installed while the
tests specified by [more stuff] take place.

--built-tree specifies some tests to be run, but does not provide any
binaries.

Another way to look at it is for me to explain the implementation of
the option/argument processor:

 * --built-binaries-filter sets a set of patterns. The set of
   patterns is applied to subsequent --unbuilt-tree and --source
   options. Any packages produced by the build matching the filter
   are treated as if they had been specified explicitly with --binary
   at the same place in the command line as the source package dide.

 * --binaries[-forbuilds|-fortests]=ignore|auto|install: There are two
   tristate variables, one for --binaries-forbuilds and one for
   --binaries-fortests. Saying just --binaries= sets both.
   --binaries-forbuilds applies when we are doing builds, and
   --binaries-fortests when we are doing tests. The trista...

Martin Pitt writes ("[Bug 1175557] Re: adt-run fulfills dependencies with --built-tree contrary to the manpage"):
> oh, you are listening here, nice!

Sure. To be honest I'm not doing very much but I can try to clear up
any misunderstandings and help explain the tangles in my code...

> I'd appreciate if you could review
> http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=commitdiff;h=0dcce04
> ; I'm fairly sure that this reflects how adt-run currently works, and
> that it mostly makes sense that way. If you think the behaviour is
> actually wrong, and the previous manpage was right, please let me know.

Your new text is generally right. The --apt-source option postdates
me so I don't know that your text for that is right. The old text for
--built-tree is wrong.

| .TP
| .BR -B " | " --no-built-binaries
|-Specifies that all built binaries should be ignored completely;
|-equivalent to
|+Specifies that all built binaries from source packages should be ignored
|+completely, and dependencies are satisfied with packages from the archive. Note
|+that packages still get built if a test requires
|+\fBbuild-needed\fR. Equivalent to

Of course it isn't quite right to say "from the archive"; they might
come from *.deb command line arguments too.

Regards,
Ian.

This bug was fixed in the package autopkgtest - 2.6.1

---------------
autopkgtest (2.6.1) unstable; urgency=medium

  * adt-virt-lxc: Call lxc-config through sudo as well like all the other LXC
    commands. This fixes running adt-run as user with the LXC runner.
  * adt-run: For copydown(), always chown the testbed files to the non-root
    user (if available). This fixes a "Permission denied" error with the
    schroot or LXC runners and --unbuilt-tree if the source tree has tight
    permissions. (LP: #1266811)
  * Adjust LxcRunner.test_tree_built_binaries to also work with current apt
    (similar to commit 5798b7f for the schroot runner).
  * adt-virt-lxc: Call "sudo test" and "sudo readlink" instead of os.lexists()
    and os.readlink() as recent LXC now makes the default container dir
    /var/lib/lxc not readably by non-users. Fixes running adt-virt-lxc/adt-run
    as user. (LP: #1266809)
  * Add test for --binary with --built-tree.
  * adt-run.1: Clarify and fix some wrong and misleading documentation what
    happens with debs produced by source packages and specified with --binary.
    (LP: #1175557)

 -- Martin Pitt <email address hidden> Wed, 29 Jan 2014 06:53:16 +0100