Ubuntu
pyzmq package

python-libzmq: import zmq fails in python2 on Xenial

Asked by Damjan Miklic

On Ubuntu Xenial (upgraded from Trusty), in python 2.7.12, with python-libzmq installed, import zmq fails with the following error message:

ImportError: libzmq.so.3: cannot open shared object file: No such file or directory

Creating a symbolic link in /usr/lib/x86_64-linux-gnu

ln -s libzmq.so.5.0.0 libzmq.so.3

seems to fix the issue. Is this a bug in python-zmq or in libzmq3?

Question information

Language:
English Edit question
Status:
Solved
For:
Ubuntu pyzmq Edit question
Assignee:
No assignee Edit question
Solved by:
Damjan Miklic
Solved:
Last query:
Last reply:
Revision history for this message
actionparsnip (andrew-woodhead666) said : 		#1

What is the output of:

sudo updatedb; locate libzmq*

Thanks

Revision history for this message
Damjan Miklic (damjan-miklic-larics) said : 		#2

Hi, thanks for the quick reply.

For some reason, the command

sudo updatedb; locate libzmq*

provides absolutely no output on my system.

Leaving out the trailing asterisk:

sudo updatedb; locate libzmq

provides the following output:

/home/damjan/.virtualenvs/testenv/lib/python2.7/site-packages/zmq/backend/cython/libzmq.pxd
/home/damjan/devel/bbone/debian_7/usr/lib/arm-linux-gnueabihf/libzmq.so.1
/home/damjan/devel/bbone/debian_7/usr/lib/arm-linux-gnueabihf/libzmq.so.1.0.1
/home/damjan/devel/bbone/debian_7/usr/lib/python2.6/dist-packages/zmq/core/libzmq.pxd
/home/damjan/devel/bbone/debian_7/usr/lib/python2.7/dist-packages/zmq/core/libzmq.pxd
/home/damjan/devel/bbone/debian_7/usr/local/lib/libzmq.a
/home/damjan/devel/bbone/debian_7/usr/local/lib/libzmq.la
/home/damjan/devel/bbone/debian_7/usr/local/lib/libzmq.so
/home/damjan/devel/bbone/debian_7/usr/local/lib/libzmq.so.3
/home/damjan/devel/bbone/debian_7/usr/local/lib/libzmq.so.3.0.0
/home/damjan/devel/bbone/debian_7/usr/local/lib/pkgconfig/libzmq.pc
/home/damjan/devel/bbone/ubuntu_12_04/usr/lib/libzmq.so.1
/home/damjan/devel/bbone/ubuntu_12_04/usr/lib/libzmq.so.1.0.0
/home/damjan/devel/bbone/ubuntu_12_04/usr/lib/python2.7/dist-packages/zmq/core/libzmq.pxd
/home/damjan/devel/bbone/ubuntu_12_04/usr/local/lib/libzmq.a
/home/damjan/devel/bbone/ubuntu_12_04/usr/local/lib/libzmq.la
/home/damjan/devel/bbone/ubuntu_12_04/usr/local/lib/libzmq.so
/home/damjan/devel/bbone/ubuntu_12_04/usr/local/lib/libzmq.so.3
/home/damjan/devel/bbone/ubuntu_12_04/usr/local/lib/libzmq.so.3.0.0
/home/damjan/devel/bbone/ubuntu_12_04/usr/local/lib/pkgconfig/libzmq.pc
/mnt/ext/var/cache/apt/archives/libzmq3-dev_4.1.4-7_amd64.deb
/mnt/ext/var/cache/apt/archives/libzmqpp-dev_3.2.0-0ubuntu4_amd64.deb
/mnt/ext/var/cache/pbuilder/aptcache/libzmq3-dev_4.0.4+dfsg-2_amd64.deb
/mnt/ext/var/cache/pbuilder/aptcache/libzmq3_4.0.4+dfsg-2_amd64.deb
/mnt/ext/var/cache/pbuilder/aptcache/libzmqpp-dev_3.2.0-0ubuntu3_amd64.deb
/mnt/ext/var/cache/pbuilder/aptcache/libzmqpp3_3.2.0-0ubuntu3_amd64.deb
/mnt/ext/var/lib/dpkg/info/libzmq1:amd64.list
/mnt/ext/var/lib/dpkg/info/libzmq1:amd64.postrm
/mnt/ext/var/lib/dpkg/info/libzmq3-dev:amd64.list
/mnt/ext/var/lib/dpkg/info/libzmq3-dev:amd64.md5sums
/mnt/ext/var/lib/dpkg/info/libzmq3:amd64.list
/mnt/ext/var/lib/dpkg/info/libzmq3:amd64.postrm
/mnt/ext/var/lib/dpkg/info/libzmq5:amd64.list
/mnt/ext/var/lib/dpkg/info/libzmq5:amd64.md5sums
/mnt/ext/var/lib/dpkg/info/libzmq5:amd64.shlibs
/mnt/ext/var/lib/dpkg/info/libzmq5:amd64.symbols
/mnt/ext/var/lib/dpkg/info/libzmq5:amd64.triggers
/mnt/ext/var/lib/dpkg/info/libzmqpp-dev:amd64.list
/mnt/ext/var/lib/dpkg/info/libzmqpp-dev:amd64.md5sums
/mnt/ext/var/lib/dpkg/info/libzmqpp3:amd64.list
/mnt/ext/var/lib/dpkg/info/libzmqpp3:amd64.md5sums
/mnt/ext/var/lib/dpkg/info/libzmqpp3:amd64.shlibs
/mnt/ext/var/lib/dpkg/info/libzmqpp3:amd64.triggers
/mnt/storage/Backupz/mrcina-upgrade-2016-11-02/damjan/.virtualenvs/testenv/lib/python2.7/site-packages/zmq/backend/cython/libzmq.pxd
/usr/lib/python2.7/dist-packages/zmq/backend/cython/libzmq.pxd
/usr/lib/python3/dist-packages/zmq/backend/cython/libzmq.pxd
/usr/lib/x86_64-linux-gnu/libzmq.a
/usr/lib/x86_64-linux-gnu/libzmq.so
/usr/lib/x86_64-linux-gnu/libzmq.so.3
/usr/lib/x86_64-linux-gnu/libzmq.so.5
/usr/lib/x86_64-linux-gnu/libzmq.so.5.0.0
/usr/lib/x86_64-linux-gnu/libzmqpp.so
/usr/lib/x86_64-linux-gnu/libzmqpp.so.3
/usr/lib/x86_64-linux-gnu/libzmqpp.so.3.2.0
/usr/lib/x86_64-linux-gnu/pkgconfig/libzmq.pc
/usr/local/lib/python2.7/dist-packages/zmq/backend/cython/libzmq.pxd
/usr/share/doc/libzmq3-dev
/usr/share/doc/libzmq5
/usr/share/doc/libzmqpp-dev
/usr/share/doc/libzmqpp3
/usr/share/doc/libzmq3-dev/changelog.Debian.gz
/usr/share/doc/libzmq3-dev/copyright
/usr/share/doc/libzmq5/AUTHORS
/usr/share/doc/libzmq5/NEWS.gz
/usr/share/doc/libzmq5/changelog.Debian.gz
/usr/share/doc/libzmq5/copyright
/usr/share/doc/libzmqpp-dev/changelog.Debian.gz
/usr/share/doc/libzmqpp-dev/copyright
/usr/share/doc/libzmqpp3/changelog.Debian.gz
/usr/share/doc/libzmqpp3/copyright
/var/cache/apt/archives/libzmq3-dev_4.1.4-7_amd64.deb
/var/cache/apt/archives/libzmqpp-dev_3.2.0-0ubuntu4_amd64.deb
/var/cache/pbuilder/aptcache/libzmq3-dev_4.0.4+dfsg-2_amd64.deb
/var/cache/pbuilder/aptcache/libzmq3_4.0.4+dfsg-2_amd64.deb
/var/cache/pbuilder/aptcache/libzmqpp-dev_3.2.0-0ubuntu3_amd64.deb
/var/cache/pbuilder/aptcache/libzmqpp3_3.2.0-0ubuntu3_amd64.deb
/var/lib/dpkg/info/libzmq1:amd64.list
/var/lib/dpkg/info/libzmq1:amd64.postrm
/var/lib/dpkg/info/libzmq3-dev:amd64.list
/var/lib/dpkg/info/libzmq3-dev:amd64.md5sums
/var/lib/dpkg/info/libzmq3:amd64.list
/var/lib/dpkg/info/libzmq3:amd64.postrm
/var/lib/dpkg/info/libzmq5:amd64.list
/var/lib/dpkg/info/libzmq5:amd64.md5sums
/var/lib/dpkg/info/libzmq5:amd64.shlibs
/var/lib/dpkg/info/libzmq5:amd64.symbols
/var/lib/dpkg/info/libzmq5:amd64.triggers
/var/lib/dpkg/info/libzmqpp-dev:amd64.list
/var/lib/dpkg/info/libzmqpp-dev:amd64.md5sums
/var/lib/dpkg/info/libzmqpp3:amd64.list
/var/lib/dpkg/info/libzmqpp3:amd64.md5sums
/var/lib/dpkg/info/libzmqpp3:amd64.shlibs
/var/lib/dpkg/info/libzmqpp3:amd64.triggers

The file /usr/lib/x86_64-linux-gnu/libzmq.so.3 is a symbolic link created manually by me.

The /home/damjan/devel/* stuff are arm binaries I use for cross-compilation. Finally,

dpkg-query -L libzmq3-dev

gives:

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/pkgconfig
/usr/lib/x86_64-linux-gnu/pkgconfig/libzmq.pc
/usr/lib/x86_64-linux-gnu/libzmq.a
/usr/include
/usr/include/zmq_utils.h
/usr/include/zmq.h
/usr/include/zmq.hpp
/usr/share
/usr/share/doc
/usr/share/doc/libzmq3-dev
/usr/share/doc/libzmq3-dev/copyright
/usr/share/man
/usr/share/man/man7
/usr/share/man/man7/zmq_tcp.7.gz
/usr/share/man/man7/zmq_curve.7.gz
/usr/share/man/man7/zmq_ipc.7.gz
/usr/share/man/man7/zmq_pgm.7.gz
/usr/share/man/man7/zmq_null.7.gz
/usr/share/man/man7/zmq.7.gz
/usr/share/man/man7/zmq_inproc.7.gz
/usr/share/man/man7/zmq_tipc.7.gz
/usr/share/man/man7/zmq_plain.7.gz
/usr/share/man/man3
/usr/share/man/man3/zmq_unbind.3.gz
/usr/share/man/man3/zmq_msg_copy.3.gz
/usr/share/man/man3/zmq_send.3.gz
/usr/share/man/man3/zmq_version.3.gz
/usr/share/man/man3/zmq_close.3.gz
/usr/share/man/man3/zmq_msg_gets.3.gz
/usr/share/man/man3/zmq_msg_init.3.gz
/usr/share/man/man3/zmq_recvmsg.3.gz
/usr/share/man/man3/zmq_errno.3.gz
/usr/share/man/man3/zmq_ctx_term.3.gz
/usr/share/man/man3/zmq_msg_init_size.3.gz
/usr/share/man/man3/zmq_proxy.3.gz
/usr/share/man/man3/zmq_setsockopt.3.gz
/usr/share/man/man3/zmq_msg_set.3.gz
/usr/share/man/man3/zmq_send_const.3.gz
/usr/share/man/man3/zmq_ctx_shutdown.3.gz
/usr/share/man/man3/zmq_z85_decode.3.gz
/usr/share/man/man3/zmq_msg_more.3.gz
/usr/share/man/man3/zmq_msg_send.3.gz
/usr/share/man/man3/zmq_msg_close.3.gz
/usr/share/man/man3/zmq_msg_size.3.gz
/usr/share/man/man3/zmq_recv.3.gz
/usr/share/man/man3/zmq_msg_init_data.3.gz
/usr/share/man/man3/zmq_strerror.3.gz
/usr/share/man/man3/zmq_has.3.gz
/usr/share/man/man3/zmq_ctx_set.3.gz
/usr/share/man/man3/zmq_bind.3.gz
/usr/share/man/man3/zmq_msg_move.3.gz
/usr/share/man/man3/zmq_msg_data.3.gz
/usr/share/man/man3/zmq_socket_monitor.3.gz
/usr/share/man/man3/zmq_msg_recv.3.gz
/usr/share/man/man3/zmq_connect.3.gz
/usr/share/man/man3/zmq_sendmsg.3.gz
/usr/share/man/man3/zmq_ctx_new.3.gz
/usr/share/man/man3/zmq_z85_encode.3.gz
/usr/share/man/man3/zmq_getsockopt.3.gz
/usr/share/man/man3/zmq_msg_get.3.gz
/usr/share/man/man3/zmq_poll.3.gz
/usr/share/man/man3/zmq_socket.3.gz
/usr/share/man/man3/zmq_ctx_get.3.gz
/usr/share/man/man3/zmq_curve_keypair.3.gz
/usr/share/man/man3/zmq_proxy_steerable.3.gz
/usr/share/man/man3/zmq_disconnect.3.gz
/usr/lib/x86_64-linux-gnu/libzmq.so
/usr/share/doc/libzmq3-dev/changelog.Debian.gz

Revision history for this message
Manfred Hampl (m-hampl) said : 		#3

libzmq*3* seems to be a leftover from trusty, xenial has libzmq*5*

What output do you get for

apt-get --simulate autoremove

Revision history for this message
Damjan Miklic (damjan-miklic-larics) said : 		#4

Here is the output of sudo apt-get --simulate autoremove

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  ipython-notebook-common libjs-highlight libjs-highlight.js libjs-marked
0 upgraded, 0 newly installed, 4 to remove and 0 not upgraded.
Remv ipython-notebook-common [2.4.1-1]
Remv libjs-highlight [8.2+ds-4build1]
Remv libjs-highlight.js [8.2+ds-4build1]
Remv libjs-marked [0.3.2+dfsg-1]

As for libzmq*3*, apt-cache search libzmq gives

libjeromq-java - Java implementation of the ZeroMQ messaging library
libzeromq-perl - messaging library using ZeroMQ
libzmq-dbg - lightweight messaging kernel (debugging symbols)
libzmq-dev - lightweight messaging kernel (development files)
libzmq-java - ZeroMQ Java bindings (jzmq)
libzmq-java-doc - Documentation for ZeroMQ Java bindings (jzmq)
libzmq-jni - ZeroMQ Java bindings (jzmq)
libzmq1 - lightweight messaging kernel (shared library)
libzmq3-dev - lightweight messaging kernel (development files)
libzmq5 - lightweight messaging kernel (shared library)
libzmq5-dbg - lightweight messaging kernel (debugging symbols)
libzmqpp-dev - High-level C++ bindings for zeromq3 - development files
libzmqpp3 - High-level C++ bindings for zeromq3

and apt-cache show libzmq3-dev gives

Package: libzmq3-dev
Priority: optional
Section: universe/libdevel
Installed-Size: 1404
Maintainer: Ubuntu Developers <email address hidden>
Original-Maintainer: Laszlo Boszormenyi (GCS) <email address hidden>
Architecture: amd64
Source: zeromq3
Version: 4.1.4-7
Replaces: libzmq5-dev
Provides: libzmq5-dev
Depends: libzmq5 (= 4.1.4-7)
Conflicts: libzmq-dev, libzmq5-dev
Filename: pool/universe/z/zeromq3/libzmq3-dev_4.1.4-7_amd64.deb
Size: 282046
MD5sum: 5c421215f8fd6aed3727ed123e7556e9
SHA1: 043821803a86bbb91196e6cc9efcce20a99fbd77
SHA256: a618309da599c358b7a7e61f9cc2534854bed7e22c280a73eb70e4dd9f2cce5f
Description-en: lightweight messaging kernel (development files)
 ØMQ is a library which extends the standard socket interfaces with features
 traditionally provided by specialised messaging middleware products.
 .
 ØMQ sockets provide an abstraction of asynchronous message queues, multiple
 messaging patterns, message filtering (subscriptions), seamless access to
 multiple transport protocols and more.
 .
 This package contains the ZeroMQ development libraries and header files.
Description-md5: 7b2c1e47f6d305566aebc0e65e04f5ee
Multi-Arch: same
Homepage: http://www.zeromq.org/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 9m

Best,
damjan

Revision history for this message
Manfred Hampl (m-hampl) said : 		#5

What is the output of

apt-cache policy libzmq-dev libzmq3-dev libzmq5-dev libzmq3 libzmq5

Revision history for this message
Manfred Hampl (m-hampl) said : 		#6

And please also provide the output of

dpkg -l | grep zmq

Revision history for this message
Damjan Miklic (damjan-miklic-larics) said : 		#7

Here are the outputs.

apt-cache policy libzmq-dev libzmq3-dev libzmq5-dev libzmq3 libzmq5

libzmq-dev:
  Installed: (none)
  Candidate: 2.2.0+dfsg-7
  Version table:
     2.2.0+dfsg-7 500
        500 http://hr.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
libzmq3-dev:
  Installed: 4.1.4-7
  Candidate: 4.1.4-7
  Version table:
 *** 4.1.4-7 500
        500 http://hr.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        100 /var/lib/dpkg/status
libzmq5-dev:
  Installed: (none)
  Candidate: (none)
  Version table:
libzmq3:
  Installed: (none)
  Candidate: (none)
  Version table:
     4.0.4+dfsg-2 0
        100 /var/lib/dpkg/status
libzmq5:
  Installed: 4.1.4-7
  Candidate: 4.1.4-7
  Version table:
 *** 4.1.4-7 500
        500 http://hr.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
        100 /var/lib/dpkg/status

dpkg -l | grep zmq

rc libzmq1:amd64 2.2.0+dfsg-5 amd64 lightweight messaging kernel (shared library)
rc libzmq3:amd64 4.0.4+dfsg-2 amd64 lightweight messaging kernel (shared library)
ii libzmq3-dev:amd64 4.1.4-7 amd64 lightweight messaging kernel (development files)
ii libzmq5:amd64 4.1.4-7 amd64 lightweight messaging kernel (shared library)
ii libzmqpp-dev:amd64 3.2.0-0ubuntu4 amd64 High-level C++ bindings for zeromq3 - development files
ii libzmqpp3:amd64 3.2.0-0ubuntu4 amd64 High-level C++ bindings for zeromq3
ii python-zmq 15.2.0-0ubuntu4 amd64 Python bindings for 0MQ library
ii python3-zmq 15.2.0-0ubuntu4 amd64 Python3 bindings for 0MQ library

Revision history for this message
Manfred Hampl (m-hampl) said : 		#8

What do you get for the commands
ls -l /usr/lib/x86_64-linux-gnu/libzmq*
ldd /usr/lib/x86_64-linux-gnu/libzmq.so.3

As shown on http://packages.ubuntu.com/search?searchon=contents&keywords=libzmq.&mode=filename&suite=xenial&arch=any there should not be any libzmq*3* files on a xenial system.

You could try deleting (or renaming) the /usr/lib/x86_64-linux-gnu/libzmq.so.3 file
Maybe you have to reinstall libzmq3-dev, libzmq5 and/or python-zmq afterwards, and/or update the loader cache.

Revision history for this message
Damjan Miklic (damjan-miklic-larics) said : 		#9

Hi Manfred,

thanks for helping me with this. Like I wrote in my original report, libzmq.so.3 was not installed by any package, and that was my original problem, because without it, I could not import zmq in Python 2. So I created libzmq.so.3 by hand as a symlink to libzmq.so which allows me to import zmq in Python 2.

So to sum up, I installed python-zmq with apt, which pulled in libzmq5. However, it did not function properly until I created libzmq.so.3 by hand, which does not seem like proper behavior.

The output of ls -l /usr/lib/x86_64-linux-gnu/libzmq* is

-rw-r--r-- 1 root root 1244438 Mar 15 2016 /usr/lib/x86_64-linux-gnu/libzmq.a
lrwxrwxrwx 1 root root 17 Mar 14 2016 /usr/lib/x86_64-linux-gnu/libzmqpp.so -> libzmqpp.so.3.2.0
lrwxrwxrwx 1 root root 17 Mar 14 2016 /usr/lib/x86_64-linux-gnu/libzmqpp.so.3 -> libzmqpp.so.3.2.0
-rw-r--r-- 1 root root 60208 Mar 14 2016 /usr/lib/x86_64-linux-gnu/libzmqpp.so.3.2.0
lrwxrwxrwx 1 root root 15 Mar 15 2016 /usr/lib/x86_64-linux-gnu/libzmq.so -> libzmq.so.5.0.0
lrwxrwxrwx 1 root root 9 Dec 13 19:16 /usr/lib/x86_64-linux-gnu/libzmq.so.3 -> libzmq.so
lrwxrwxrwx 1 root root 15 Mar 15 2016 /usr/lib/x86_64-linux-gnu/libzmq.so.5 -> libzmq.so.5.0.0
-rw-r--r-- 1 root root 416880 Mar 15 2016 /usr/lib/x86_64-linux-gnu/libzmq.so.5.0.0

and for ldd /usr/lib/x86_64-linux-gnu/libzmq.so.3 it is

 linux-vdso.so.1 => (0x00007fffd3d63000)
 libsodium.so.18 => /usr/lib/x86_64-linux-gnu/libsodium.so.18 (0x00007f3262b2e000)
 librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3262926000)
 libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3262708000)
 libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3262386000)
 libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3261fbd000)
 libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3261da6000)
 /lib64/ld-linux-x86-64.so.2 (0x0000564a22d06000)
 libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3261a9d000)

Thanks and best,
damjan

Revision history for this message
Manfred Hampl (m-hampl) said : 		#10

So to sum up:
You had problems with "python import zmq", and no one knows why.
As a workaround you created a symlink for a file that should not be needed on xenial, and now it works, and no one knows why.

Revision history for this message
Damjan Miklic (damjan-miklic-larics) said : 		#11

Ok, I've figured it out :) I had a previous version of pyzmq installed by pip. I should have posted the full Python error message, as it gives the clue to the solution (the pip version is installed in /usr/local):

damjan@mrcina-ubuntu:/usr/lib/x86_64-linux-gnu$ python
Python 2.7.12 (default, Nov 19 2016, 06:48:10)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import zmq
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/zmq/__init__.py", line 34, in <module>
    from zmq import backend
  File "/usr/local/lib/python2.7/dist-packages/zmq/backend/__init__.py", line 40, in <module>
    reraise(*exc_info)
  File "/usr/local/lib/python2.7/dist-packages/zmq/backend/__init__.py", line 27, in <module>
    _ns = select_backend(first)
  File "/usr/local/lib/python2.7/dist-packages/zmq/backend/select.py", line 27, in select_backend
    mod = __import__(name, fromlist=public_api)
  File "/usr/local/lib/python2.7/dist-packages/zmq/backend/cython/__init__.py", line 6, in <module>
    from . import (constants, error, message, context,
ImportError: libzmq.so.3: cannot open shared object file: No such file or directory

Thanks for your help!

Best,
damjan