esys-escript

Problems compiling with MPI

Asked by liucheng83

Hi,
Sorry to add my similar problems as I compile my optional files with mpi = 'OPENMPI' in xenial_options.py
It comes out:
RuntimeError: Unable to link against ['mpi_cxx', 'mpi', 'open-rte', 'open-pal'] (paths: /usr/lib/openmpi/include, /usr/lib/openmpi/lib):
  File "/home/lc/win-share/vbshare/escript--FEM/src/SConstruct", line 495:
    env=checkOptionalLibraries(env)
  File "/home/lc/win-share/vbshare/escript--FEM/src/site_scons/dependencies.py", line 481:
    mpi_inc_path,mpi_lib_path=findLibWithHeader(env, env['mpi_libs'], 'mpi.h', env['mpi_prefix'], lang='c++')
  File "/home/lc/win-share/vbshare/escript--FEM/src/site_scons/site_init.py", line 77:
    raise RuntimeError('Unable to link against %s (paths: %s, %s)'%(libs,inc_path,lib_path))

Question information

Language:
English Edit question
Status:
Answered
For:
esys-escript Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
liucheng83 (lcheng83) said : 		#1

And the config.log is:

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/dependencies.py,line 48:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking whether the C++ compiler works...
.sconf_temp/conftest_0.cpp <-
  |
  |int main()
  |{
  | return 0;
  |}
  |
g++ -o .sconf_temp/conftest_0.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -Iinclude .sconf_temp/conftest_0.cpp
scons: Configure: yes

scons: Configure: Checking for C++ function gethostname()...
.sconf_temp/conftest_1.cpp <-
  |
  |
  |#include <assert.h>
  |
  |#ifdef __cplusplus
  |extern "C"
  |#endif
  |char gethostname();
  |
  |int main() {
  |#if defined (__stub_gethostname) || defined (__stub___gethostname)
  | fail fail fail
  |#else
  | gethostname();
  |#endif
  |
  | return 0;
  |}
  |
g++ -o .sconf_temp/conftest_1.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -Iinclude .sconf_temp/conftest_1.cpp
g++ -o .sconf_temp/conftest_1 -fopenmp .sconf_temp/conftest_1.o -Llib
scons: Configure: yes

scons: Configure: Checking for C++ header file byteswap.h...
.sconf_temp/conftest_2.cpp <-
  |
  |#include "byteswap.h"
  |
  |
g++ -o .sconf_temp/conftest_2.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -Iinclude .sconf_temp/conftest_2.cpp
scons: Configure: yes

scons: Configure: Checking for C++ function SCbswap32()...
.sconf_temp/conftest_3.cpp <-
  |
  |
  |#include <assert.h>
  |#include <byteswap.h>
  |#define SCbswap32() {int x=0;bswap_32(x);}
  |
  |int main() {
  |#if defined (__stub_SCbswap32) || defined (__stub___SCbswap32)
  | fail fail fail
  |#else
  | SCbswap32();
  |#endif
  |
  | return 0;
  |}
  |
g++ -o .sconf_temp/conftest_3.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -Iinclude .sconf_temp/conftest_3.cpp
g++ -o .sconf_temp/conftest_3 -fopenmp .sconf_temp/conftest_3.o -Llib
scons: Configure: yes

scons: Configure: Checking for C++ header file sys/endian.h...
.sconf_temp/conftest_4.cpp <-
  |
  |#include "sys/endian.h"
  |
  |
g++ -o .sconf_temp/conftest_4.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude .sconf_temp/conftest_4.cpp
.sconf_temp/conftest_4.cpp:2:24: fatal error: sys/endian.h: No such file or directory
compilation terminated.
scons: Configure: no

scons: Configure: Checking for C++ header file libkern/OSByteOrder.h...
.sconf_temp/conftest_5.cpp <-
  |
  |#include "libkern/OSByteOrder.h"
  |
  |
g++ -o .sconf_temp/conftest_5.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude .sconf_temp/conftest_5.cpp
.sconf_temp/conftest_5.cpp:2:33: fatal error: libkern/OSByteOrder.h: No such file or directory
compilation terminated.
scons: Configure: no

scons: Configure: Checking for working complex std::acos()...
.sconf_temp/conftest_6.cpp <-
  |
  |#include <complex>
  |int main() { std::complex<double> x(0,3.14159265359), y(1.5707963,-1.8622957);
  |return std::abs(std::acos(x)-y) < 1e-6 ? 0:-1;}
  |
g++ -o .sconf_temp/conftest_6.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude .sconf_temp/conftest_6.cpp
g++ -o .sconf_temp/conftest_6 -fopenmp .sconf_temp/conftest_6.o -Llib
.sconf_temp/conftest_6 > .sconf_temp/conftest_6.out
scons: Configure: yes

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/dependencies.py,line 167:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ header file Python.h...
.sconf_temp/conftest_7.cpp <-
  |
  |#include "Python.h"
  |
  |
g++ -o .sconf_temp/conftest_7.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude .sconf_temp/conftest_7.cpp
scons: Configure: yes

scons: Configure: Checking for C++ function Py_Exit()...
.sconf_temp/conftest_8.cpp <-
  |
  |
  |#include <assert.h>
  |
  |#ifdef __cplusplus
  |extern "C"
  |#endif
  |char Py_Exit();
  |
  |int main() {
  |#if defined (__stub_Py_Exit) || defined (__stub___Py_Exit)
  | fail fail fail
  |#else
  | Py_Exit();
  |#endif
  |
  | return 0;
  |}
  |
g++ -o .sconf_temp/conftest_8.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude .sconf_temp/conftest_8.cpp
g++ -o .sconf_temp/conftest_8 -fopenmp .sconf_temp/conftest_8.o -Llib -L/usr/lib/python2.7/config-x86_64-linux-gnu -lpython2.7
scons: Configure: yes

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/site_init.py,line 66:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ library boost_python-py27...
.sconf_temp/conftest_9.cpp <-
  |
  |
  |#include "boost/python.hpp"
  |
  |int
  |main() {
  |
  |return 0;
  |}
  |
g++ -o .sconf_temp/conftest_9.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude -I/usr/include .sconf_temp/conftest_9.cpp
g++ -o .sconf_temp/conftest_9 -fopenmp .sconf_temp/conftest_9.o -Llib -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -lboost_python-py27 -lboost_python-py27
scons: Configure: yes

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/dependencies.py,line 244:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ header file numpy/ndarrayobject.h...
.sconf_temp/conftest_10.cpp <-
  |#include "Python.h"
  |
  |#include "numpy/ndarrayobject.h"
  |
  |
g++ -o .sconf_temp/conftest_10.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -isystem /usr/include/boost -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -Iinclude .sconf_temp/conftest_10.cpp
In file included from /usr/include/python2.7/numpy/ndarraytypes.h:1777:0,
                 from /usr/include/python2.7/numpy/ndarrayobject.h:18,
                 from .sconf_temp/conftest_10.cpp:3:
/usr/include/python2.7/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
scons: Configure: yes

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/site_init.py,line 66:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ library netcdf_c++...
.sconf_temp/conftest_11.cpp <-
  |
  |
  |#include "netcdf.h"
  |
  |int
  |main() {
  |
  |return 0;
  |}
  |
g++ -o .sconf_temp/conftest_11.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -isystem /usr/include/boost -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -DESYS_HAVE_NUMPY_H -Iinclude -I/usr/include .sconf_temp/conftest_11.cpp
g++ -o .sconf_temp/conftest_11 -fopenmp .sconf_temp/conftest_11.o -Llib -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -lboost_python-py27 -lnetcdf_c++ -lnetcdf -lnetcdf_c++
scons: Configure: yes

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/site_init.py,line 66:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ library umfpack...
.sconf_temp/conftest_12.cpp <-
  |
  |
  |#include "umfpack.h"
  |
  |int
  |main() {
  |
  |return 0;
  |}
  |
g++ -o .sconf_temp/conftest_12.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -isystem /usr/include/boost -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -DESYS_HAVE_NUMPY_H -DESYS_HAVE_NETCDF -Iinclude -I/usr/include -I/usr/include/suitesparse .sconf_temp/conftest_12.cpp
g++ -o .sconf_temp/conftest_12 -fopenmp .sconf_temp/conftest_12.o -Llib -L/usr/lib/python2.7/config-x86_64-linux-gnu -L/usr/lib -lpython2.7 -lboost_python-py27 -lumfpack -lblas -lamd -lumfpack
scons: Configure: yes

file /home/lc/win-share/vbshare/escript--FEM/src/site_scons/site_init.py,line 66:
 Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ library mpi_cxx...
.sconf_temp/conftest_13.cpp <-
  |
  |
  |#include "mpi.h"
  |
  |int
  |main() {
  |
  |return 0;
  |}
  |
g++ -o .sconf_temp/conftest_13.o -c -fopenmp -O3 -std=c++11 -pedantic -Wall -fPIC -Wno-unknown-pragmas -Wno-sign-compare -Wno-system-headers -Wno-long-long -Wno-strict-aliasing -finline-functions -isystem /usr/include/python2.7 -isystem /usr/include/boost -DSVN_VERSION=-2 -DHAVE_GETHOSTNAME -DHAVE_BYTESWAP_H -DESYS_HAVE_NUMPY_H -DESYS_HAVE_NETCDF -DESYS_HAVE_UMFPACK -Iinclude -I/usr/include -I/usr/include/suitesparse -I/usr/lib/openmpi/include .sconf_temp/conftest_13.cpp
scons: Configure: Caught exception while building ".sconf_temp/conftest_13":

Traceback (most recent call last):
  File "/usr/lib/scons/SCons/Job.py", line 212, in start
    task.prepare()
  File "/usr/lib/scons/SCons/Taskmaster.py", line 197, in prepare
    t.prepare()
  File "/usr/lib/scons/SCons/Node/FS.py", line 3093, in prepare
    SCons.Node.Node.prepare(self)
  File "/usr/lib/scons/SCons/Node/__init__.py", line 723, in prepare
    raise SCons.Errors.StopError(msg % (i, self))
SCons.Errors.StopError: Implicit dependency `/usr/lib/libopen-rte.so' not found, needed by target `.sconf_temp/conftest_13'.
scons: Configure: no

Revision history for this message
liucheng83 (lcheng83) said : 		#2

$which mpirun
/usr/bin/mpirun
$ mpirun -V
mpirun (Open MPI) 1.10.2
$ dpkg -l | grep openmpi
ii libhdf5-openmpi-10:amd64 1.8.16+docs-4ubuntu1 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - OpenMPI version
ii libopenmpi-dev 1.10.2-8ubuntu1 amd64 high performance message passing library -- header files
ii libopenmpi1.10 1.10.2-8ubuntu1 amd64 high performance message passing library -- shared library
ii openmpi-bin 1.10.2-8ubuntu1 amd64 high performance message passing library -- binaries
ii openmpi-common 1.10.2-8ubuntu1 all high performance message passing library -- common files

Revision history for this message
liucheng83 (lcheng83) said : 		#3

>mpicxx -show
g++ -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent -I/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -pthread -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags -L/usr/lib/openmpi/lib -lmpi_cxx –lmpi

Revision history for this message
Manfred Hampl (m-hampl) said : 		#4

"... SCons.Errors.StopError: Implicit dependency `/usr/lib/libopen-rte.so' not found, ..."

What is the output of
ls -l /usr/lib/libopen-*

Revision history for this message
liucheng83 (lcheng83) said : 		#5

Manfred, Hi

$ ls -l /usr/lib/libopen-*
lrwxrwxrwx 1 root root 16 2月 26 2016 /usr/lib/libopen-pal.so -> libopen-pal.so.4
lrwxrwxrwx 1 root root 21 2月 26 2016 /usr/lib/libopen-pal.so.13 -> libopen-pal.so.13.0.2
lrwxrwxrwx 1 root root 33 2月 26 2016 /usr/lib/libopen-pal.so.13.0.2 -> openmpi/lib/libopen-pal.so.13.0.2
lrwxrwxrwx 1 root root 16 2月 26 2016 /usr/lib/libopen-rte.so -> libopen-rte.so.4
lrwxrwxrwx 1 root root 21 2月 26 2016 /usr/lib/libopen-rte.so.12 -> libopen-rte.so.12.0.2
lrwxrwxrwx 1 root root 33 2月 26 2016 /usr/lib/libopen-rte.so.12.0.2 -> openmpi/lib/libopen-rte.so.12.0.2

Revision history for this message
Bob (caltinay) said : 		#6

Something messed up your OpenMPI libraries:

 lrwxrwxrwx 1 root root 16 2月 26 2016 /usr/lib/libopen-rte.so -> libopen-rte.so.4

libopen-rte.so.4 does not exist.
I would try reinstalling the openmpi dev package, i.e.
apt-get install --reinstall libopenmpi-dev

Revision history for this message
liucheng83 (lcheng83) said : 		#7

Hi, caltinay
$ sudo apt-get install --reinstall libopenmpi-dev
$ ls -l /usr/lib/libopen-*
lrwxrwxrwx 1 root root 16 2月 26 2016 /usr/lib/libopen-pal.so -> libopen-pal.so.4
lrwxrwxrwx 1 root root 21 2月 26 2016 /usr/lib/libopen-pal.so.13 -> libopen-pal.so.13.0.2
lrwxrwxrwx 1 root root 33 2月 26 2016 /usr/lib/libopen-pal.so.13.0.2 -> openmpi/lib/libopen-pal.so.13.0.2
lrwxrwxrwx 1 root root 16 2月 26 2016 /usr/lib/libopen-rte.so -> libopen-rte.so.4
lrwxrwxrwx 1 root root 21 2月 26 2016 /usr/lib/libopen-rte.so.12 -> libopen-rte.so.12.0.2
lrwxrwxrwx 1 root root 33 2月 26 2016 /usr/lib/libopen-rte.so.12.0.2 -> openmpi/lib/libopen-rte.so.12.0.2

$ ls -l /usr/lib/openmpi/lib/libopen-*
lrwxrwxrwx 1 root root 21 2月 26 2016 /usr/lib/openmpi/lib/libopen-pal.so -> libopen-pal.so.13.0.2
-rw-r--r-- 1 root root 620720 2月 26 2016 /usr/lib/openmpi/lib/libopen-pal.so.13.0.2
lrwxrwxrwx 1 root root 21 2月 26 2016 /usr/lib/openmpi/lib/libopen-rte.so -> libopen-rte.so.12.0.2
-rw-r--r-- 1 root root 492464 2月 26 2016 /usr/lib/openmpi/lib/libopen-rte.so.12.0.2

Revision history for this message
Manfred Hampl (m-hampl) said : 		#8

This might be a bug in the packaging of libopenmpi.

As far as I can see the package http://archive.ubuntu.com/ubuntu/pool/universe/o/openmpi/libopenmpi-dev_1.10.2-8ubuntu1_amd64.deb installs links
"/usr/lib/libopen-pal.so -> libopen-pal.so.4" and
"/usr/lib/libopen-rte.so -> libopen-rte.so.4"
but the package http://archive.ubuntu.com/ubuntu/pool/universe/o/openmpi/libopenmpi1.10_1.10.2-8ubuntu1_amd64.deb delivers the files
/usr/lib/libopen-pal.so.13
/usr/lib/libopen-pal.so.13.0.2
/usr/lib/libopen-rte.so.12
/usr/lib/libopen-rte.so.12.0.2

It seems that for Debian this is covered in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=817020
For Ubuntu I found only bug #1597508 which is not related to the dangling links that you have, but some other problems covered in the mentioned Debian bug.

My suggestions:
1. try correcting the links on your system and check whether esys-script works then
2. create a bug report (or add information to bug #1597508) to signal the problem to the developers.

Can you help with this problem?

Provide an answer of your own, or ask liucheng83 for more information if necessary.

To post a message you must log in.