extended precision with dolfin

Ok I was able to compile petsc and slepc for extended precision (__float128). When compiling dolfin it fails the slepc test run with the error below. Apparently it cannot link to libquadmath with gcc-4.6. Is there somewhere in dolfin where I can add that linking option?

--------------------------------------
Performing C++ SOURCE FILE Test SLEPC_TEST_RUNS failed with the following output:
Change Dir: /home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp

Run Build Command:/usr/bin/make "cmTryCompileExec/fast"
/usr/bin/make -f CMakeFiles/cmTryCompileExec.dir/build.make CMakeFiles/cmTryCompileExec.dir/build
make[1]: Entering directory `/home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp'
/usr/bin/cmake -E cmake_progress_report /home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp/CMakeFiles 1
Building CXX object CMakeFiles/cmTryCompileExec.dir/src.cxx.o
/usr/bin/c++ -DSLEPC_TEST_RUNS -fopenmp -I/usr/lib/slepcdir/3.2 -I/usr/lib/slepcdir/3.2/linux-gnu-c-opt/include -I/usr/lib/slepcdir/3.2/include -I/usr/lib/petscdir/3.2/include -I/usr/lib/petscdir/3.2/linux-gnu-c-opt/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -o CMakeFiles/cmTryCompileExec.dir/src.cxx.o -c /home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp/src.cxx
Linking CXX executable cmTryCompileExec
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTryCompileExec.dir/link.txt --verbose=1
/usr/bin/c++ -DSLEPC_TEST_RUNS -fopenmp CMakeFiles/cmTryCompileExec.dir/src.cxx.o -o cmTryCompileExec -rdynamic -Wl,-Bstatic -lslepc -Wl,-Bdynamic -lparpack -larpack /usr/lib/petscdir/3.2/linux-gnu-c-opt/lib/libpetsc.so /usr/lib/openmpi/lib/libmpi_cxx.so /usr/lib/openmpi/lib/libmpi.so /usr/lib/openmpi/lib/libopen-rte.so /usr/lib/openmpi/lib/libopen-pal.so -ldl -lnsl -lutil -lm -ldl -lnsl -lutil -lm -Wl,-rpath,/usr/lib/petscdir/3.2/linux-gnu-c-opt/lib:/usr/lib/openmpi/lib
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../../lib/libslepc.a(default.o): undefined reference to symbol 'sqrtq@@QUADMATH_1.0'
/usr/bin/ld: note: 'sqrtq@@QUADMATH_1.0' is defined in DSO /usr/lib/x86_64-linux-gnu/libquadmath.so.0 so try adding it to the linker command line
/usr/lib/x86_64-linux-gnu/libquadmath.so.0: could not read symbols: Invalid operation
collect2: ld returned 1 exit status
make[1]: Leaving directory `/home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp'
make[1]: *** [cmTryCompileExec] Error 1
make: *** [cmTryCompileExec/fast] Error 2

Return value: 1
Source file was:

#include "petsc.h"
#include "slepceps.h"
int main()
{
  PetscErrorCode ierr;
  int argc = 0;
  char** argv = NULL;
  ierr = SlepcInitialize(&argc, &argv, PETSC_NULL, PETSC_NULL);
  EPS eps;
  ierr = EPSCreate(PETSC_COMM_SELF, &eps); CHKERRQ(ierr);
  //ierr = EPSSetFromOptions(eps); CHKERRQ(ierr);
#if PETSC_VERSION_MAJOR == 3 && PETSC_VERSION_MINOR <= 1
  ierr = EPSDestroy(eps); CHKERRQ(ierr);
#else
  ierr = EPSDestroy(&eps); CHKERRQ(ierr);
#endif
  ierr = SlepcFinalize(); CHKERRQ(ierr);
  return 0;
}

You can alter the compiler flags by:

   CMAKE_CXX_FLAGS

and for the linker I think it is:

   CMAKE_SHARED_LINKER_FLAGS

but have in mind that dolfin does not support any high precision library
in its interface, nor does ufc or any of the form compilers. So any form
you ever assemble through the FEniCS interface will be of "normal"
precision. To change this you need to make changes in a number of
packages in FEniCS.

Johan

On 05/01/2012 12:25 AM, Chaffra Affouda wrote:
> Question #194957 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/194957
>
> Chaffra Affouda gave more information on the question:
> Ok I was able to compile petsc and slepc for extended precision
> (__float128). When compiling dolfin it fails the slepc test run with the
> error below. Apparently it cannot link to libquadmath with gcc-4.6. Is
> there somewhere in dolfin where I can add that linking option?
>
>
> --------------------------------------
> Performing C++ SOURCE FILE Test SLEPC_TEST_RUNS failed with the following output:
> Change Dir: /home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp
>
> Run Build Command:/usr/bin/make "cmTryCompileExec/fast"
> /usr/bin/make -f CMakeFiles/cmTryCompileExec.dir/build.make CMakeFiles/cmTryCompileExec.dir/build
> make[1]: Entering directory `/home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp'
> /usr/bin/cmake -E cmake_progress_report /home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp/CMakeFiles 1
> Building CXX object CMakeFiles/cmTryCompileExec.dir/src.cxx.o
> /usr/bin/c++ -DSLEPC_TEST_RUNS -fopenmp -I/usr/lib/slepcdir/3.2 -I/usr/lib/slepcdir/3.2/linux-gnu-c-opt/include -I/usr/lib/slepcdir/3.2/include -I/usr/lib/petscdir/3.2/include -I/usr/lib/petscdir/3.2/linux-gnu-c-opt/include -I/usr/lib/openmpi/include -I/usr/lib/openmpi/include/openmpi -o CMakeFiles/cmTryCompileExec.dir/src.cxx.o -c /home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp/src.cxx
> Linking CXX executable cmTryCompileExec
> /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTryCompileExec.dir/link.txt --verbose=1
> /usr/bin/c++ -DSLEPC_TEST_RUNS -fopenmp CMakeFiles/cmTryCompileExec.dir/src.cxx.o -o cmTryCompileExec -rdynamic -Wl,-Bstatic -lslepc -Wl,-Bdynamic -lparpack -larpack /usr/lib/petscdir/3.2/linux-gnu-c-opt/lib/libpetsc.so /usr/lib/openmpi/lib/libmpi_cxx.so /usr/lib/openmpi/lib/libmpi.so /usr/lib/openmpi/lib/libopen-rte.so /usr/lib/openmpi/lib/libopen-pal.so -ldl -lnsl -lutil -lm -ldl -lnsl -lutil -lm -Wl,-rpath,/usr/lib/petscdir/3.2/linux-gnu-c-opt/lib:/usr/lib/openmpi/lib
> /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.6.1/../../../../lib/libslepc.a(default.o): undefined reference to symbol 'sqrtq@@QUADMATH_1.0'
> /usr/bin/ld: note: 'sqrtq@@QUADMATH_1.0' is defined in DSO /usr/lib/x86_64-linux-gnu/libquadmath.so.0 so try adding it to the linker command line
> /usr/lib/x86_64-linux-gnu/libquadmath.so.0: could not read symbols: Invalid operation
> collect2: ld returned 1 exit status
> make[1]: Leaving directory `/home/chaffra/Projects/Fenics/dolfin/build/CMakeFiles/CMakeTmp'
> make[1]: *** [cmTryCompileExec] Error 1
> make: *** [cmTryCompileExec/fast] Error 2
>
> Return value: 1
> Source file was:
>
> #include "petsc.h"
> #include "slepceps.h"
> int main()
> {
> PetscErrorCode ierr;
> int argc = 0;
> char** argv = NULL;
> ierr = SlepcInitialize(&argc,&argv, PETSC_NULL, PETSC_NULL);
> EPS eps;
> ierr = EPSCreate(PETSC_COMM_SELF,&eps); CHKERRQ(ierr);
> //ierr = EPSSetFromOptions(eps); CHKERRQ(ierr);
> #if PETSC_VERSION_MAJOR == 3&& PETSC_VERSION_MINOR<= 1
> ierr = EPSDestroy(eps); CHKERRQ(ierr);
> #else
> ierr = EPSDestroy(&eps); CHKERRQ(ierr);
> #endif
> ierr = SlepcFinalize(); CHKERRQ(ierr);
> return 0;
> }
>

I understand but I'd like to find a way to solve linear algebra problems using extended precision from dolfin. I am running into very ill conditioned matrices for my solver and scaling does not help.

My plan was to change dolfin in order to feed petsc with extended precision vector and matrix, have it do the calculation at the extended precision and then take the result for conversion back to double. My first thinking would be to change PETScVector.cpp and PETScMatrix.cpp interface to accommodate the extended precision PETSCScalar type but maybe this is not enough.

I do not want to change too much so any other idea on how I might tackle this problem would be helpful (maybe using MTL4 since it's templated) .

Thanks,
Chaffra

Updating the linear algebra part is probably the easiest part, but is
that enough?

The precision of the la objects are limited by double precision from the
assembled ufc forms. To change this you need a major surgery of many
FEniCS components.

Johan

On 05/01/2012 03:35 PM, Chaffra Affouda wrote:
> Question #194957 on DOLFIN changed:
> https://answers.launchpad.net/dolfin/+question/194957
>
> Status: Answered => Open
>
> Chaffra Affouda is still having a problem:
> I understand but I'd like to find a way to solve linear algebra problems
> using extended precision from dolfin. I am running into very ill
> conditioned matrices for my solver and scaling does not help.
>
> My plan was to change dolfin in order to feed petsc with extended
> precision vector and matrix, have it do the calculation at the extended
> precision and then take the result for conversion back to double. My
> first thinking would be to change PETScVector.cpp and PETScMatrix.cpp
> interface to accommodate the extended precision PETSCScalar type but
> maybe this is not enough.
>
> I do not want to change too much so any other idea on how I might tackle
> this problem would be helpful (maybe using MTL4 since it's templated) .
>
> Thanks,
> Chaffra
>

I managed to change PETScVector.cpp and PETScMatrix.cpp so that it would work with the dev version of petsc compiled with __float128. Although dolfin still uses double precision I think all my petsc linear algebra is done in extended precision which greatly improve my newton solver convergence although it seems a little slow. Maybe this approach could be extended to the other linear solver libraries that have the option of extended precision calculations. I pushed my changes to lp:~chaffra/dolfin/main if interested.