How do I build Fluidity on Redhat Enterprise 5.X and derivatives?

PLEASE NOTE: Using RHEL 5.X and derivatives is not recommended for Fluidity users; an upgrade to Red Hat Enterprise 6 and its derivatives is highly recommended and supported by packages built by the Fluidity team. A FAQ for Red Hat 6 is available at https://answers.launchpad.net/fluidity/+faq/2169 . Users choosing the follow the instructions in this FAQ do so at their own risk, and central support from the Fluidity project is not available for this build method.

 * Background to this document

This is an initial draft of a FAQ for Fluidity users needing to build Fluidity on Redhat Enterprise 5.X and derivatives. At present this is in the state of a provisional document and methods are untested beyond initial development. Development was undertaken on CentOS 5.7 but it is expected that this should be relatively similar on any Redhat Enterprise 5.X derivative.

Redhat Enterprise 5.X is based on 2007 software releases. These are in some cases too old for building Fluidity. The critical issue is with compiler support, with gcc 4.1. There are issues with python 2.4 which cause issues with some Fluidity scripts, in particular with the testing infrastructure. Given the complexity of rebuilding python and subsequent packages this document takes the approach that retaining the pre-packaged python is the preferable option.

A significant amount of the required software is available through the Fedora Project's 'Extra Packages for Enterprise Linux' (EPEL) repository. Wherever possible these packages are used to save the time and effort of manual rebuilds.

 * Setting up the environment

Fluidity and its supporting software require some modification of the default environment through environment variables. It is suggested that you place the following in a file which can be sourced from your .bashrc, or sourced manually from the command line. Variables required are:

 export CPPFLAGS="$CPPFLAGS -I/usr/include/netcdf-3 -I/usr/include/udunits2"
 export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib/openmpi/lib:/usr/local/lib"
 export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib/vtk-5.6"
 export PATH="$PATH:/usr/local/lib/openmpi/bin"
 export PETSC_DIR="/usr/local"
 export PYTHONPATH="$PYTHONPATH:/usr/local/lib/python2.4/site-packages/"
 export VTK_INCLUDE="/usr/include/vtk-5.6"
 export VTK_LIBS="/usr/lib/vtk-5.6"
 export CC=gcc44
 export CXX=g++44
 export FC=gfortran44
 export F77=gfortran44
 export F90=gfortran44

 * Installing EPEL and EPEL packages

To enable EPEL on your system, one pacakge from the Fedora project is installed. This is obtained with:

 wget http://www.mirrorservice.org/sites/dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

and installed with:

 sudo rpm -ivh epel-release-5-4.noarch.rpm

The system is then updated:

 sudo yum update

Then, using yum, install packages from EPEL with the following, all on one line:

 yum install blas blas-devel bzr cmake fltk fltk-devel gcc44 gcc44-c++
      gcc44-gfortran gmp-devel lapack lapack-devel libtool
      mesa-libGL-devel mesa-libGLU-devel netcdf netcdf-devel netpbm
      netpbm-devel python-devel python-matplotlib scipy tcl-devel
      texinfo tk-devel transfig udunits2 udunits2-devel

Some builds fail to honour environment and expect 'gcc' to be present, therefore if you do not have any other gcc versions on your system it is suggested that you set a default gcc target with:

 sudo ln -s /usr/bin/gcc44 /usr/bin/gcc

 * Installing supporting software from source

 ** OpenMPI

OpenMPI is available from the default package set but this is built on top of the default gcc 4.1.2 compiler whereas Fluidity needs an MPI implementation on top of the EPEL-sourced gcc 4.4 compiler. The environment set up in the file above ensures that the OpenMPI build as follows will use the EPEL 4.4 compiler.

OpenMPI is obtained and uncompressed with:

 wget http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.4.tar.gz
 tar -zxvvf openmpi-1.6.4.tar.gz

Then built and installed with:

 cd openmpi-1.6.4
 ./configure --prefix=/usr/local
 make
 sudo make install

Some subsequent builds don't seem to honour path properly, so set up symlinks into /usr/bin:

sudo ln -s /usr/local/bin/mpicc /usr/bin/mpicc
sudo ln -s /usr/local/bin/mpicxx /usr/bin/mpicxx
sudo ln -s /usr/local/bin/mpif90 /usr/bin/mpif90

 ** ParMETIS

ParMETIS is obtained and uncompressed with:

 wget http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis/OLD/ParMetis-3.2.0.tar.gz
 tar -zxvvf ParMetis-3.2.0.tar.gz

And then built and installed with:

 cd ParMetis-3.2.0
 make
 sudo cp lib*.a /usr/local/lib
 sudo cp parmetis.h /usr/local/include

 ** PETSc

The PETSc build here uses an older version of PETSc, 3.1p8, as later versions require a METIS build as part of their configuration, which in turn requires a newer version of CMake than available by default with Redhat Enterprise 5.X. Even with the newer version, there are continuing problems with the build to be resolved, hence it is recommended to hold at 3.1p8.
FORTRAN_MODULES = \
        lb_user_const.mod \
        zoltan_types.mod \
        zoltan.mod \
        zoltan_user_data.mod
PETSc is obtained and uncompressed with:

 wget http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-3.1-p8.tar.gz
 tar -zxvvf petsc-3.1-p8.tar.gz
 cd petsc-3.1-p8
 export PETSC_DIR=$PWD

Then configured with the following, all on one line:

 ./configure --prefix=/usr/local --with-mpi-shared=1 --with-shared=1 --with-debugging=0
    --with-parmetis=1 --download-parmetis=1 --with-hypre=1 --download-hypre=1
    --with-prometheus=1 --download-prometheus=1 --with-fortran-interfaces=1

then compiled and installed with the make and make install as supplied by the configure and make all outputs. You may need to prefix the 'make install' command with 'sudo' to write to /usr/local.

And finally return PETSC_DIR to the original value:

 export PETSC_DIR=/usr/local

 ** Zoltan

The Zoltan build requires resetting the compiler environment to point to mpi compilers, with:

 export CC=mpicc
 export CXX=mpicxx
 export FC=mpif90
 export F90=mpif90

Zoltan is then obtained and uncompressed with:

 wget http://www.cs.sandia.gov/~kddevin/Zoltan_Distributions/zoltan_distrib_v3.6.tar.gz
 tar -zxvvf zoltan_distrib_v3.6.tar.gz

It is now necessary to fix a known issue with Zoltan Fortran modules by editing the file Zoltan_v3.6/src/Makefile.in: at the end of that file find lines looking like:

 #if F90_MODULE_LOWERCASE
 #FORTRAN_MODULES = \
 # lb_user_const.mod \
 # zoltan_types.mod \
 # zoltan.mod \
 # zoltan_user_data.mod
 #else
 #FORTRAN_MODULES = \
 # LB_USER_CONST.mod \
 # ZOLTAN_TYPES.mod \
 # ZOLTAN.mod \
 # ZOLTAN_USER_DATA.mod
 #endif

Delete lines and leading # characters appropriately so you end up with the above replaced by:

 FORTRAN_MODULES = \
        lb_user_const.mod \
        zoltan_types.mod \
        zoltan.mod \
        zoltan_user_data.mod

Zoltan is built in a separate build directory; create this and change into it with:

 mkdir Zoltan_build
 cd Zoltan_build

Zoltan is then configured with the following all on one line:

 ../Zoltan_v3.6/configure x86_64-linux-gnu --prefix=/usr/local --enable-mpi
     --with-mpi-compilers --with-parmetis --enable-f90interface --disable-examples
     --enable-zoltan-cppdriver --with-parmetis-libdir=/usr/local/lib

and built with:

 make
 sudo make install

 ** VTK

For this VTK build, use the legacy version (5.6.1) rather than the latest version (5.10.1) as any releases later than 5.6.1 require a later version of CMake than supplied by RHEL 5.X.

VTK is obtained and uncompressed with:

 wget http://www.vtk.org/files/release/5.6/vtk-5.6.1.tar.gz
 tar -zxvvf vtk-5.6.1.tar.gz

It is built in a separate build directory, which is made and changed into with:

 mkdir VTK_Build
 cd VTK_Build

Then configured with:

 ccmake -i ../VTK

and press 'c' for the initial configuration. This brings up a configuration options interface in which you should set values as follows.

Note that the install prefix would ideally be /usr/local but the 5.6 install of vtk python fails if it's not installing to the default python location, as .pth files as required by this install fail to work. Hence this build installs into /usr to get round that issue.

 BUILD_EXAMPLES *OFF
 BUILD_SHARED_LIBS *ON
 BUILD_TESTING *ON
 CMAKE_BACKWARDS_COMPATIBILITY *2.4
 CMAKE_BUILD_TYPE *Debug
 CMAKE_INSTALL_PREFIX */usr
 VTK_DATA_ROOT *VTK_DATA_ROOT-NOTFOUND
 VTK_EXTRA_COMPILER_WARNINGS *OFF
 VTK_LARGE_DATA_ROOT *VTK_LARGE_DATA_ROOT-NOTFOUND
 VTK_USE_CHARTS *OFF
 VTK_USE_CHEMISTRY *OFF
 VTK_USE_GEOVIS *OFF
 VTK_USE_INFOVIS *OFF
 VTK_USE_N_WAY_ARRAYS *OFF
 VTK_USE_PARALLEL *OFF
 VTK_USE_QT *OFF
 VTK_USE_RENDERING *OFF
 VTK_USE_TEXT_ANALYSIS *OFF
 VTK_USE_VIEWS *OFF
 VTK_WRAP_JAVA *OFF
 VTK_WRAP_PYTHON *ON
 VTK_WRAP_PYTHON_SIP *OFF
 VTK_WRAP_TCL *OFF

Then 'c' to configure, 'c' again to configure, 'g' to generate.

VTK is then built and installed with:

 make
 sudo make install

 ** GMSH

GMSH is obtained and uncompressed with:

 wget http://geuz.org/gmsh/src/gmsh-2.6.1-source.tgz
 tar -zxvvf gmsh-2.6.1-source.tgz

It is then built in a separate build directory made and changed into with:

 mkdir gmsh-build
 cd gmsh-build

GMSH is configured with:

 ccmake -i ../gmsh-2.6.1-source

Then 'c' to configure, 'c' again to configure, 'g' to generate. If you
encounter 'help' screens, press 'e' to exit from them.

GMSH is then compiled and installed with:

 make
 sudo make install

 ** Spud

Spud is obtained with:

 bzr co lp:spud

and then configured, built, and installed with:

 cd spud
 ./configure
 make
 sudo make install

 * Fluidity build

Before building Fluidity, ensure that the build environment as described at the start of this document. Be particularly careful if you have worked through this document that the variables reset in the Zoltan build do not persist at this stage - it is recommended that you start a new session and re-source the initial environment variables as described at the start of this docunment to ensure a clean Fluidity build environment.

Fluidity is obtained with:

 bzr co lp:fluidity/4.1 fluidity-4.1

Then configured and built with:

 cd fluidity-4.1
 ./configure --enable-2d-adaptivity
 make
 make fltools

The Fluidity install is very dependent on environment, which needs to be present during the install process. To ensure this, become root, then set up the environment as described at the start of the document, then run:

 make install

During the Fluidity build and install you may experience some warnings relating to multiple versions of libgfortran being linked in as version 3 from Fluidity and version 1 from liblapack; these don't appear to cause problems later and look to be able to be safely installed.

Due to the issues with older system Python the install can only be partially tested, and given the provisional nature of this FAQ this build should probably be regarded as potentially broken until tests can be carried out. Use at your own risk!