LP git Recipe based builds remove .mailmap file breaking build

Asked by Christian Ehrhardt  on 2020-10-21

TL;DR
- git contains file .mailmap, but the archive and build based on recipe is missing it
- .mailmap is then missing and breaks the build
- I fail to see why/where it is removed and how to overcome that

Details:

# Issue
direct upload builds and works, e.g. manual backport
https://launchpad.net/ubuntu/+source/openvswitch/2.13.1-0ubuntu0.20.04.1
But "the same" built by LP-recipe fails
https://code.launchpad.net/~paelzer/+recipe/server-backports-openvswitch-focal

I happened to find that there is a file missing as built by the LP-recipe

# .mailmap is in the upload that works
$ pull-lp-source openvswitch focal-proposed
$ ll openvswitch-2.13.1/.mailmap
-rw-rw-r-- 1 paelzer paelzer 4552 Aug 4 23:41 openvswitch-2.13.1/.mailmap

Recipe:
# git-build-recipe format 0.4 deb-version {debversion}~backport20.04-{time}
https://git.launchpad.net/~usd-import-team/ubuntu/+source/openvswitch ubuntu/devel
merge backport https://git.launchpad.net/~paelzer/+git/server-backports backport-openvswitch-focal

# .mailmap is also available in git
pkg https://git.launchpad.net/ubuntu/+source/openvswitch (fetch)

$ git checkout pkg/ubuntu/devel
$ ll .mailmap
-rw-rw-r-- 1 paelzer paelzer 4552 Sep 9 09:31 .mailmap

FYI: The git that the recipe uses has no .gitignore

# mailmap is also in the fixup branch in git
server-backports git+ssh://git.launchpad.net/~paelzer/+git/server-backports (fetch)

$ git checkout server-backports/backport-openvswitch-focal
$ ll .mailmap
-rw-rw-r-- 1 paelzer paelzer 4552 Sep 9 09:31 .mailmap

# But .mailmap is missing in the build via recipe:

Download tgz and dsc as created by recipe:
wget https://launchpad.net/~paelzer/+archive/ubuntu/backport-tests/+sourcefiles/openvswitch/2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1/openvswitch_2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1.dsc
wget https://launchpad.net/~paelzer/+archive/ubuntu/backport-tests/+sourcefiles/openvswitch/2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1/openvswitch_2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1.tar.xz

Extract
$ dpkg-source -x openvswitch_2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1.dsc created-by-recipe
dpkg-source: warning: extracting unsigned source package (openvswitch_2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1.dsc)
dpkg-source: info: extracting openvswitch in created-by-recipe
dpkg-source: info: unpacking openvswitch_2.13.1-0ubuntu1~backport20.04-202010210831~ubuntu20.04.1.tar.xz

Missing
$ ll created-by-recipe/.mailmap
ls: cannot access 'created-by-recipe/.mailmap': No such file or directory

# Not all .* files are removed

The git repo also has .cirrus.yml, .travis.yml .travis/ - those stay and are
still in the recipe created tarball as well, just .mailmap is missing.

# FTFBS due to that

I'd not mind about the file, but in this case it is breaking the build :-/
make[3]: *** No rule to make target '.mailmap', needed by 'distdir'. Stop.

# I fail to recreate this behavior for further debugging
on Focal:
ii git-build-recipe 0.3.6 all construct a Git branch from a recipe

$ rm -rf /tmp/testbuild1/*; git-build-recipe --no-build --allow-fallback-to-native --distribution focal openvswitch-focal.recipe /tmp/testbuild1/
$ ll /tmp/testbuild1/openvswitch-focal/.mailmap
-rw-rw-r-- 1 paelzer paelzer 4552 Okt 21 11:16 /tmp/testbuild1/openvswitch-focal/.mailmap

I'm puzzled and hence reach out to you for help.
1. what might be happening?
2. how to avoid that so I can build again?

Question information

Language:
English Edit question
Status:
Solved
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Solved by:
Christian Ehrhardt 
Solved:
2020-10-21
Last query:
2020-10-21
Last reply:
2020-10-21

This question was reopened

Colin Watson (cjwatson) said : #1

This is because .mailmap is listed in @tar_ignore_default_pattern in /usr/share/perl5/Dpkg/Source/Package.pm, and launchpad-buildd builds recipe-output source packages using dpkg-buildpackage with the -I option. This isn't currently configurable, I'm afraid.

Why does this package's override_dh_auto_build do a "make dist"? That's very strange - a package would normally assume that it's consuming the output of a "make dist", and not run it separately for itself. I think making that less strange might be the best way to work around this.

The make dist is related to the switch to OVN

commit d8dbc2dec4e7efdea5be452135a4d2c21419ad02
Author: James Page <email address hidden>
Date: Mon Jan 6 15:42:23 2020 +0000

    Misc updates for OVN switchover

diff --git a/debian/rules b/debian/rules
index 1eaf74226d..47d713ec4d 100755
--- a/debian/rules
+++ b/debian/rules
@@ -67,9 +67,9 @@ endif # i386/amd64/ppc64el
 endif # nocheck

 override_dh_auto_build:
- $(MAKE) $(PARALLEL) -C _debian distdir=openvswitch
+ $(MAKE) $(PARALLEL) -C _debian dist distdir=openvswitch
 ifneq (,$(filter i386 amd64 ppc64el arm64, $(DEB_HOST_ARCH)))
- $(MAKE) $(PARALLEL) -C _dpdk distdir=openvswitch_dpdk
+ $(MAKE) $(PARALLEL) -C _dpdk dist distdir=openvswitch_dpdk
 endif

 override_dh_auto_clean:

But now that I understand what happens I can patch my way around it.
Thank you!