PY_CFLAGS contains reference to -specs=/usr/share/dpkg/no-pie-compile.specs

Bug #1768644 reported by Anthony Sottile
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
python3.6 (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Cosmic
Fix Released
Undecided
Unassigned

Bug Description

SRU:

Impact]
 * Causing a build failure

[Test Case]
 * See below. Or just check that the flag doesn't show
   up when building extensions.

[Regression Potential]

 * I don't see any. These flags are injected anyway
   in the package builds for extensions. It's only about
   virtualenv installations.

Minimal reproduction (docker):

FROM ubuntu:bionic
RUN apt-get update && \
    apt-get install -y --no-install-recommends gcc python-dev virtualenv && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*
RUN virtualenv /venv -ppython3 && \
    /venv/bin/pip install ujson

Fails with:

Collecting ujson
  Downloading https://files.pythonhosted.org/packages/16/c4/79f3409bc710559015464e5f49b9879430d8f87498ecdc335899732e5377/ujson-1.35.tar.gz (192kB)
Building wheels for collected packages: ujson
  Running setup.py bdist_wheel for ujson: started
  Running setup.py bdist_wheel for ujson: finished with status 'error'
  Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-x7e5igby/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-de4xsrzr --python-tag cp36:
  running bdist_wheel
  running build
  running build_ext
  building 'ujson' extension
  creating build
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/python
  creating build/temp.linux-x86_64-3.6/lib
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/usr/include/python3.6m -I/venv/include/python3.6m -c ./python/ujson.c -o build/temp.linux-x86_64-3.6/./python/ujson.o -D_GNU_SOURCE
  x86_64-linux-gnu-gcc: error: /usr/share/dpkg/no-pie-compile.specs: No such file or directory
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for ujson
  Running setup.py clean for ujson
Failed to build ujson
Installing collected packages: ujson
  Running setup.py install for ujson: started
    Running setup.py install for ujson: finished with status 'error'
    Complete output from command /venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-x7e5igby/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-d0l7cnq5/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.6/ujson:
    running install
    running build
    running build_ext
    building 'ujson' extension
    creating build
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/python
    creating build/temp.linux-x86_64-3.6/lib
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I./python -I./lib -I/usr/include/python3.6m -I/venv/include/python3.6m -c ./python/ujson.c -o build/temp.linux-x86_64-3.6/./python/ujson.o -D_GNU_SOURCE
    x86_64-linux-gnu-gcc: error: /usr/share/dpkg/no-pie-compile.specs: No such file or directory
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-x7e5igby/ujson/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-d0l7cnq5/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python3.6/ujson" failed with error code 1 in /tmp/pip-install-x7e5igby/ujson/
The command '/bin/sh -c virtualenv /venv -ppython3 && /venv/bin/pip install ujson' returned a non-zero code: 1

The root cause of this seems to be the PY_CFLAGS sysconfig setting which contains flags that were used to build python itself, including a reference to a file which doesn't exist:

root@71b2faa3da66:/# python3 -m sysconfig | grep no-pie-compile
 CFLAGS = "-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security"
 CONFIGURE_CFLAGS = "-g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security"
 CONFIG_ARGS = "'--enable-shared' '--prefix=/usr' '--enable-ipv6' '--enable-loadable-sqlite-extensions' '--with-dbmliborder=bdb:gdbm' '--with-computed-gotos' '--without-ensurepip' '--with-system-expat' '--with-system-libmpdec' '--with-system-ffi' 'CC=x86_64-linux-gnu-gcc' 'CFLAGS=-g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security ' 'LDFLAGS=-Wl,-Bsymbolic-functions -specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'"
 PY_CFLAGS = "-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security"
 PY_CORE_CFLAGS = "-Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-EKG1lX/python3.6-3.6.5=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat -Werror=format-security -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -IObjects -IInclude -IPython -I. -I../Include -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPy_BUILD_CORE"

Revision history for this message
Matthias Klose (doko) wrote :

confirmed. we need to change that for 18.04. For now you can work around this by install the dpkg-dev package.

affects: python3-defaults (Ubuntu) → python3.6 (Ubuntu)
Changed in python3.6 (Ubuntu):
status: New → Triaged
Changed in python3.6 (Ubuntu Bionic):
status: New → Triaged
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python3.6 - 3.6.5-6ubuntu1

---------------
python3.6 (3.6.5-6ubuntu1) cosmic; urgency=medium

  * Disable the lto build for now.

 -- Matthias Klose <email address hidden> Thu, 03 May 2018 21:35:50 +0200

Changed in python3.6 (Ubuntu Cosmic):
status: Triaged → Fix Released
Revision history for this message
Anthony Sottile (asottile) wrote :

Any ETA on when this'll land for bionic?

Matthias Klose (doko)
description: updated
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Please test proposed package

Hello Anthony, or anyone else affected,

Accepted python3.6 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python3.6/3.6.6-1~18.04 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python3.6 (Ubuntu Bionic):
status: Triaged → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Anthony Sottile (asottile) wrote :

Confirmed fixed:

(venv) root@99e4d04174a9:/# pip install ujson
Collecting ujson
  Using cached https://files.pythonhosted.org/packages/16/c4/79f3409bc710559015464e5f49b9879430d8f87498ecdc335899732e5377/ujson-1.35.tar.gz
Building wheels for collected packages: ujson
  Running setup.py bdist_wheel for ujson ... done
  Stored in directory: /root/.cache/pip/wheels/28/77/e4/0311145b9c2e2f01470e744855131f9e34d6919687550f87d1
Successfully built ujson
Installing collected packages: ujson

Didn't have time to wait for proposed so I just downloaded the amd64 binary outputs and `gdebi -n` installed them.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python3.6 - 3.6.6-1~18.04

---------------
python3.6 (3.6.6-1~18.04) bionic-proposed; urgency=medium

  * SRU: LP: #1792143. Update Python 3.6 to the recent subminor release.

python3.6 (3.6.6-1) unstable; urgency=medium

  * Python 3.6.6 release.

python3.6 (3.6.6~rc1-4) unstable; urgency=medium

  * Update to 20180622 from the 3.6 branch.
    - Fix installation of idlelib testsuite.

python3.6 (3.6.6~rc1-3) unstable; urgency=medium

  * Update to 20180619 from the 3.6 branch.
  * Let ensurepip work with pip versions 9 or 10. Closes: #901797.
  * pdb(1): Fix pointer to HTML documentation. Closes: #901603.

python3.6 (3.6.6~rc1-1) unstable; urgency=medium

  * Python 3.6.6 release candidate 1.

python3.6 (3.6.5-10) unstable; urgency=medium

  * Update to 20180608 from the 3.6 branch.
  * Update the the sysconfig-debian-schemes patch, not yet applied.
    sysconfig: The 'posix_local' schema name is the default unless the env var
    DEB_PYTHON_INSTALL_LAYOUT is set to 'deb' or 'deb_system'. Use the latter
    for package buillds.
  * Build cmath as a builtin instead of an extension.
  * Run the test_code_module, test_zipfile tests in the autopkg tests again.
  * Build using GCC 8.

python3.6 (3.6.5-9) unstable; urgency=medium

  * python3.6: Add breaks on python3-all (<< 3.6.5~rc1-1),
    python3-dev (<< 3.6.5~rc1-1), python3-venv (<< 3.6.5-2). Closes: #898401.

python3.6 (3.6.5-8) unstable; urgency=medium

  * Fix the lto build for extensions, passing -fPIC to the linker.

python3.6 (3.6.5-6) unstable; urgency=medium

  * Don't inject dpkg's compiler specs into distutils. LP: #1768644.

python3.6 (3.6.5-5) unstable; urgency=medium

  * Fix the lto build on x86_64 architectures.
  * Make the build of the shared modules verbose again.

python3.6 (3.6.5-4) unstable; urgency=medium

  * Re-enable pgo/lto builds.

 -- Matthias Klose <email address hidden> Wed, 12 Sep 2018 20:26:19 +0200

Changed in python3.6 (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for python3.6 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.