how to use trilinos to compiling escript?

Asked by Fan

hi sir,

I build escript on ubuntu18.04 (with python3.6), it is no problem. When I run "run-escript ~/gambit/bin/planeGravInv.py GinvInput_synth", the error is shown as follow:
________________
import imp
Configuration GinvInput_synth.py imported.
Mesh read from syth_testmesh.fly
Latitude range = 40000.0, 0.0
Gravity data variable 'gravity' read from test_grav.nc.
data range : -92.0727645507 - 55.2073226782
WARNING: This function is deprecated and is known to contain bugs.
extract data
Summary: inf=-4985.24 sup=698.284 data points=765992
Traceback (most recent call last):
  File "/home/fan/gambit/bin/planeGravInv.py", line 278, in <module>
    grav = ACGravity(dom, w_e, gz_e, rho_e, mu, m0, atol, rtol, iter_max, pdetol, config.output_name, config.VerboseLevel)
  File "/home/fan/gambit/bin/planeGravInv.py", line 44, in __init__
    self.pdeu = self.setupPDE()
  File "/home/fan/gambit/bin/planeGravInv.py", line 55, in setupPDE
    options.setPreconditioner(SolverOptions.AMG)
ValueError: escript was not compiled with Trilinos or MUMPS enabled
_____

I has use this command "scons options_file=scons/templates/bionic_py3_options.py Trilinos=True" try to install trilinos, but have error, the error is shown as follows:
__________________
scons: Reading SConscript files ...
3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
TypeError: a bytes-like object is required, not 'str':
  File "/home/fan/python_escript_5.6.2/src/SConstruct", line 173:
    ENV = {'PATH': os.environ['PATH']})
  File "/usr/lib/scons/SCons/Environment.py", line 965:
    variables.Update(self)
  File "/usr/lib/scons/SCons/Variables/__init__.py", line 184:
    exec(contents, {}, values)
  File "<string>", line 48:

  File "<string>", line 48:
__________________

I want to know how to slove it.

Sincerely.

Fan

Question information

Language:
English Edit question
Status:
Answered
For:
esys-escript Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Fan (fan188) said :
#1

I add my try.

firstly, I use sudo apt install trilinos-dev, it is successful.

Secondly, I Create a file name trilinos_options.py in the scons folder (i.e. scons/ubuntu_options.py) then paste in the following text:

escript_opts_version=203
pythoncmd='/usr/bin/python3'
boost_libs='libboost_python3-py36'
openmp=1
pythonlibname='python3.6m'
pythonlibpath='/usr/lib/x86_64-linux-gnu'
pythonincpath='/usr/include/'
trilinos=1
trilinos_prefix=['/usr/include/trilinos','/usr/lib/x86_64-linux-gnu/']

Then, run "scons -j4 options_file=scons/trillinos_options.py", has some errors, as follows:
__________________
scons: Reading SConscript files ...
3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
Using options in scons/trilinos_options.py.
/bin/sh: 1: svnversion: not found
Checking whether the C++ compiler works... (cached) yes
Checking for C++ function gethostname()... (cached) yes
Checking for C++ header file byteswap.h... (cached) yes
Checking for C++ function SCbswap32()... (cached) yes
Checking for C++ header file sys/endian.h... (cached) no
Checking for C++ header file libkern/OSByteOrder.h... (cached) no
Checking for working complex std::acos()... (cached) yes
Checking for C++ header file Python.h... (cached) yes
Checking for C++ function Py_Exit()... (cached) yes
Checking for C++ library libboost_python3-py36... (cached) yes
Checking for C++ library libboost_python3-py36... (cached) yes
Found boost/python/numpy.hpp. Building with boost numpy support.
Checking for C++ header file numpy/ndarrayobject.h... (cached) yes
Checking for C++ library cppunit... yes
Checking for C++ library boost_iostreams... no
RuntimeError: Amesos2.hpp not found under /usr/include/trilinos:
  File "/home/fan/python_escript_5.6.2/src/SConstruct", line 525:
    env=checkOptionalLibraries(env)
  File "/home/fan/python_escript_5.6.2/src/site_scons/dependencies.py", line 827:
    env = checkForTrilinos(env)
  File "/home/fan/python_escript_5.6.2/src/site_scons/dependencies.py", line 469:
    env['trilinos_prefix'], lang='c++', try_link=havelibs)
  File "/home/fan/python_escript_5.6.2/src/site_scons/site_init.py", line 65:
    raise RuntimeError('%s not found under %s'%(header,paths[0]))
______________________________

I don't know how to solve it. So, I need your help.

what should I do? I'm looking forward to your reply.

Thank you very much!

Revision history for this message
Adam Ellery (aellery) said :
#2

Hi Fan,

You will need to download and install Trilinos and then recompile escript. Installing Trilinos is difficult and not very user friendly but I can help you if you have any problems.

Installing Trilinos involves the following steps:
1. Clone the Trilinos code repository and set it to a stable release
git clone https://github.com/trilinos/trilinos trilinos
git reset --hard 4796b92fb0644ba8c531dd9953e7a4878b05c62d
2. Create a new folder somewhere
3. In this folder, create a file named configure.sh that contains the text at this link https://pastebin.com/7Urfx7PJ
4. Modify the variable TRILINOS_PATH in this file to point to your Trilinos source directory
5. Install the dependencies that Trilinos needs to operate
sudo apt-get install libmetis-dev libsuitesparse-dev gfortran
6. Compile and install Trilinos using
sudo make -j8 install
7. Finally, recompile escript using something like
scons -j8 options_file=scons/templates/bionic_py3_options.py trilinos=1 trilinos_prefix=/usr/local

- Adam

Revision history for this message
Fan (fan188) said :
#3

Hi Adam,

Thank you for your reply.

According to your guidance, there are has some errors and warns.

4: compiling Trilinos, as follows:
___________
Finished configuring Trilinos!

-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    Trilinos_ENABLE_MPI

-- Build files have been written to: /home/fan/Trilinos/build
___________

7: recompile escript using "scons -j8 options_file=scons/templates/bionic_py3_options.py trilinos=1 trilinos_prefix=/usr/local", the errors show:
 _______
scons: Reading SConscript files ...
3.6.9 (default, Oct 8 2020, 12:12:24)
[GCC 8.4.0]
TypeError: a bytes-like object is required, not 'str':
  File "/home/fan/python_escript_5.6.2/src/SConstruct", line 173:
    ENV = {'PATH': os.environ['PATH']})
  File "/usr/lib/scons/SCons/Environment.py", line 965:
    variables.Update(self)
  File "/usr/lib/scons/SCons/Variables/__init__.py", line 184:
    exec(contents, {}, values)
  File "<string>", line 48:

  File "<string>", line 48:
____________
I using "scons -j4 pythoncmd=/usr/bin/python3 boost_libs=libboost_python3-py36 openmp=1 pythonlibname=python3.6m pythonlibpath=/usr/lib/x86_64-linux-gnu pythonincpath=/usr/include/python3.6m trilinos=1 trilinos_prefix=/usr/local" to recompile esys-escript has show that:
_________

*** Config Summary (see config.log and <prefix>/lib/buildvars for details) ***
Escript revision -2
  Install prefix: /home/fan/python_escript_5.6.2/src
          Python: /usr/bin/python3 (Version 3.6.9)
           boost: /usr (Version 1.65.1)
     boost numpy: NO
        trilinos: /usr/local (Version 13.1.0)
           numpy: YES (with headers)
            gzip: YES
  Solver library: paso, trilinos
   Direct solver: YES (umfpack, trilinos)
         domains: dudley, finley, ripley, speckley
          netcdf: NO
           weipa: YES
          openmp: YES
         cppunit: YES
            gdal: YES
          pyproj: YES
           scipy: YES
           sympy: YES
         umfpack: YES

  DISABLED features: debug gmsh lapack mkl mpi mumps parmetis silo visit
  Treating warnings as errors

WARNING: Could not detect the svn revision number!
WARNING: gmsh not available. Skipping tests usersguide/trapezoid.py usersguide/quad.py usersguide/brick.py usersguide/refine.py cookbook/example04a.py cookbook/example04b.py cookbook/example05a.py cookbook/example05b.py cookbook/example05c.py cookbook/example06.py cookbook/example08c.py cookbook/example09m.py cookbook/example09a.py cookbook/example10m.py inversion/dc_forward.py!

ERROR: build stopped due to errors
_________

When I completed above step, my testing one example, it is still show that escript was not compiled with Trilinos enabled. the detail as follows:
______

Configuration GinvInput_synth.py imported.
Mesh read from syth_testmesh.fly
Latitude range = 40000.0, 0.0
Gravity data variable 'gravity' read from test_grav.nc.
data range : -92.0727645507 - 55.2073226782
WARNING: This function is deprecated and is known to contain bugs.
extract data
Summary: inf=-4985.24 sup=698.284 data points=765992
Traceback (most recent call last):
  File "/home/fan/gambit/bin/planeGravInv.py", line 278, in <module>
    grav = ACGravity(dom, w_e, gz_e, rho_e, mu, m0, atol, rtol, iter_max, pdetol, config.output_name, config.VerboseLevel)
  File "/home/fan/gambit/bin/planeGravInv.py", line 44, in __init__
    self.pdeu = self.setupPDE()
  File "/home/fan/gambit/bin/planeGravInv.py", line 54, in setupPDE
    options.setPackage(SolverOptions.TRILINOS)
ValueError: escript was not compiled with Trilinos enabled
_______

I want how to do solve it?

Thank you

Fan

Revision history for this message
Fan (fan188) said :
#4

Hi, Sir

I reinstall trilinos-12-12, it is no problem.

However, using trilinos to recompile escript has some errors, the errors as follows:

 _______

At global scope:
cc1plus: error: unrecognized command line option '-Wno-stringop-truncation' [-Werror]
cc1plus: all warnings being treated as errors
scons: *** [build/posix/trilinoswrap/src/PreconditionerFactory.os] Error 1
scons: building terminated because of errors.

*** Config Summary (see config.log and <prefix>/lib/buildvars for details) ***
Escript revision -2
  Install prefix: /home/fan/python_escript_5_6.1/src
          Python: /usr/bin/python3 (Version 3.6.9)
           boost: /usr (Version 1.65.1)
     boost numpy: NO
        trilinos: /usr/local (Version 12.12.1)
           numpy: YES (with headers)
            gzip: YES
  Solver library: paso, trilinos
   Direct solver: YES (trilinos)
         domains: dudley, finley, ripley, speckley
          netcdf: NO
           weipa: YES
          openmp: YES
         cppunit: YES
            gdal: YES
          pyproj: YES
           scipy: YES
           sympy: YES

  DISABLED features: debug gmsh lapack mkl mpi mumps parmetis silo umfpack visit
  Treating warnings as errors

WARNING: Could not detect the svn revision number!
WARNING: gmsh not available. Skipping tests usersguide/trapezoid.py usersguide/quad.py usersguide/brick.py usersguide/refine.py cookbook/example04a.py cookbook/example04b.py cookbook/example05a.py cookbook/example05b.py cookbook/example05c.py cookbook/example06.py cookbook/example08c.py cookbook/example09m.py cookbook/example09a.py cookbook/example10m.py inversion/dc_forward.py!

ERROR: build stopped due to errors
__________

Fan

Revision history for this message
Fan (fan188) said :
#5

add errors information:

Compiling build/posix/finley/src/Assemble_PDE_Single_C.os
trilinoswrap/src/PreconditionerFactory.cpp: In function 'Teuchos::RCP<Tpetra::Operator<ST, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > > esys_trilinos::createPreconditioner(Teuchos::RCP<const Tpetra::RowMatrix<ST, int, int, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP> > >, const escript::SolverBuddy&)':
trilinoswrap/src/PreconditionerFactory.cpp:183:44: error: ISO C++ forbids declaration of 'type name' with no type [-fpermissive]
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                            ^~~~~~~~~~~~~~
trilinoswrap/src/PreconditionerFactory.cpp:183:36: error: expected '>'
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                    ^~~~~~
trilinoswrap/src/PreconditionerFactory.cpp:183:36: error: expected '('
trilinoswrap/src/PreconditionerFactory.cpp:183:44: error: 'BlockCrsMatrix' is not a member of 'Tpetra'
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                            ^~~~~~~~~~~~~~
trilinoswrap/src/PreconditionerFactory.cpp:183:44: note: suggested alternative: 'CrsMatrix'
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                            ^~~~~~~~~~~~~~
                                            CrsMatrix
trilinoswrap/src/PreconditionerFactory.cpp:183:61: error: expected primary-expression before ',' token
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                                             ^
trilinoswrap/src/PreconditionerFactory.cpp:183:64: error: expected primary-expression before ',' token
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                                                ^
trilinoswrap/src/PreconditionerFactory.cpp:183:67: error: expected primary-expression before ',' token
             if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                                                   ^
trilinoswrap/src/PreconditionerFactory.cpp:183:70: error: expected primary-expression before '>' token
            if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                                                     ^
trilinoswrap/src/PreconditionerFactory.cpp:183:73: error: expected primary-expression before '>' token
         if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                                                     ^
trilinoswrap/src/PreconditionerFactory.cpp:183:87: error: expected ')' before '{' token
     if (dynamic_cast<const Tpetra::BlockCrsMatrix<ST,LO,GO,NT>* >(mat.get())) {
                                                                               ^
Compiling build/posix/finley/src/Assemble_PDE_System_1D.os
Compiling build/posix/finley/src/Assemble_PDE_System_2D.os
At global scope:
cc1plus: error: unrecognized command line option '-Wno-stringop-truncation' [-Werror]
cc1plus: all warnings being treated as errors
scons: *** [build/posix/trilinoswrap/src/PreconditionerFactory.os] Error 1
scons: building terminated because of errors.

*** Config Summary (see config.log and <prefix>/lib/buildvars for details) ***
Escript revision -2
  Install prefix: /home/fan/python_escript_5.6.2/src
          Python: /usr/bin/python3 (Version 3.6.9)
           boost: /usr (Version 1.65.1)
     boost numpy: NO
        trilinos: /usr/local (Version 12.12.1)
           numpy: YES (with headers)
            gzip: YES
  Solver library: paso, trilinos
   Direct solver: YES (umfpack, trilinos)
         domains: dudley, finley, ripley, speckley
          netcdf: NO
           weipa: YES
          openmp: YES
         cppunit: YES
            gdal: YES
          pyproj: YES
           scipy: YES
           sympy: YES
         umfpack: YES

  DISABLED features: debug gmsh lapack mkl mpi mumps parmetis silo visit
  Treating warnings as errors

WARNING: Could not detect the svn revision number!
WARNING: gmsh not available. Skipping tests usersguide/trapezoid.py usersguide/quad.py usersguide/brick.py usersguide/refine.py cookbook/example04a.py cookbook/example04b.py cookbook/example05a.py cookbook/example05b.py cookbook/example05c.py cookbook/example06.py cookbook/example08c.py cookbook/example09m.py cookbook/example09a.py cookbook/example10m.py inversion/dc_forward.py!

ERROR: build stopped due to errors

Revision history for this message
Adam Ellery (aellery) said :
#6

Hi fan,

You may need to uninstall trilinos-dev using apt-get.

Also, after compiling Trilinos, you need to run
sudo make install

To copy the files to /usr/local/, before you recompile escript.

- Adam

Revision history for this message
Fan (fan188) said :
#7

Hi Adam,

Thank you for your help.

About you guidance, I have one confuse:

To copy the files to /usr/local/, before my recompile escript. I should copy what file to /usr/local/???

Fan

Revision history for this message
Adam Ellery (aellery) said :
#8

Hi Fan,

After compiling Trilinos, you need to run

sudo make install

inside the build folder. This will install Trilinos (i.e. copy the compiled files to the appropriate folders in /usr/local).

Adam

Revision history for this message
Fan (fan188) said :
#9

Hi Adam,

I am still confused, because the compiled files is not the same with in /usr/local, Could you give me some example.

My build folder inside include show that:

CMakefiles folder; Packages folder, Testing folder

the usr/local/ the folders show that:
/usr/local$ ls
bin etc games include lib man pgplot sbin share src trilinos_nompi

Fan

Revision history for this message
Adam Ellery (aellery) said :
#10

Hi Fan,

Normally the `sudo make install` command only copies over some of the files in the build folder (such as library files, header files etc.). The build folder will contain other files that were used by make and cmake during compilation but which are not part of the installed program.

Could you please check that it is installing to the right place? In the file configure.sh, could you please confirm that you have the line:
-DCMAKE_INSTALL_PREFIX=/usr/local \

Adam

Revision history for this message
Fan (fan188) said :
#11

Hi Adam,

Thank you for your guidance.

In the file configure.sh, has the line: -D CMAKE_INSTALL_PREFIX=/usr/local/ \

So, I will copy the make and cmake file to /usr/local/ folder for try

fan

Revision history for this message
Adam Ellery (aellery) said :
#12

Hi Fan,

You shouldn't need to copy this file to /usr/local

The installation of Trilinos has three steps. The first is to run the configure.sh script in the build folder. This will generate the information cmake needs to compile the program. Then you run `make -j8` to compile the code. Then, finally, you should run `sudo make install` to install the compiled files in the folder /usr/local.

- Adam

Revision history for this message
Fan (fan188) said :
#13

Hi Adam,

Thank you for your guide.

I success recompile the escript, and my test an example, it show error, as follows:

run-escript ~/gambit/bin/planeGravInv.py GinvInput_synth
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat
Traceback (most recent call last):
  File "/home/fan/gambit/bin/planeGravInv.py", line 11, in <module>
    from esys.finley import ReadGmsh, ReadMesh
  File "finley/py_src/__init__.py", line 32, in <module>
ImportError: libteuchoscomm.so.12: cannot open shared object file: No such file or directory

I should how to slove it.

Fan

Revision history for this message
Adam Ellery (aellery) said :
#14

Hi Fan,

The error says that escript could not locate the library file libteuchoscomm.so.12. You can fix this by adding the path to this file to LD_LIBRARY_PATH. You can do this by running something like:

export LD_LIBRARY_PATH=[path to libteuchoscomm.so.12]:$LD_LIBRARY_PATH

and opening a new terminal window. For example, if Trilinos was installed to /usr/local then you need to run

export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

Or, if you wish for your terminal to do this automatically every time you open a new window, run

echo export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH >> ~/.bashrc

- Adam

Revision history for this message
Fan (fan188) said :
#15

Hi, Adam,

Thank you for you reply.

It is help me solved escript compiled question.

At present, I have the new questions, the question is how to build the Netcdf 3 file from the txt file.

Because I came across the following situation,

_______

/usr/lib/python3/dist-packages/osgeo/__init__.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Configuration GinvInput_synth.py imported.
Mesh read from syth_testmesh.fly
Latitude range = 40000.0, 0.0
Gravity data variable 'data' read from test_grav.nc.
Gravity data variable 'data_error' read from test_grav.nc.
data range : 3.0 - 3.0
extract data
Summary: inf=-174.866 sup=9962.25 data points=765992
PCG: initial residual norm = 6.882072e+11 (absolute tolerance = 6.882072e+07)
_______________

the data range is wrong, but I don't know how to solve it.

Fan

Revision history for this message
Adam Ellery (aellery) said :
#16

Hi Fan,

Have you tried running

ncdump test_grav.nc

to see what is in the file?

- Adam

Revision history for this message
Fan (fan188) said :
#17

Hi Adam,

I tried test_grav.nc data to test program.

The test_grav.nc is the netcdf format data.

I can't running ncdump test_grav.nc, because of my can't install the netcdf-bin in computer.

Fan

Can you help with this problem?

Provide an answer of your own, or ask Fan for more information if necessary.

To post a message you must log in.