Intel 2018 + OSX + Siesta Compilation Errors when linking

Asked by Jesse Quinn on 2018-04-06

Using the latest 2018 intel compilers with openmpi on OSX i get numerous mkl reference errors during linking of siesta. I have included the arch.make and a log file that contains the FFLAGS = -O0 -g -check bounds -traceback. Any information would be appreciated.

https://jessequinn-my.sharepoint.com/:u:/g/personal/me_jessequinn_info/EZUK_6HTvOdOo_QeZ1-mqMwBudsBPNWDKJ2Ip6TxwOiCdQ?e=fafo6i

siesta version 4.1-b3

Question information

Language:
English Edit question
Status:
Answered
For:
Siesta Edit question
Assignee:
No assignee Edit question
Last query:
2018-04-06
Last reply:
2018-04-06
Nick Papior (nickpapior) said : #1

1)
Please note that the order of libraries is important:

scalapack blacs lapack blas (other intel libraries) hdf5 netcdf libz mpi

2) You have -DNCDF_PARALLEL but not -DNCDF and -DNCDF_4. You need all 3!

3) You can't mix MPICH and OpenMPI libraries. Please consult MKL suite on details for using either of them (but not both!)

Jesse Quinn (jessequinn) said : #2

Hi Nick,

Thank you for the quick answer.

I have changed the following order for LIBS with some new libraries

LIBS = -qopenmp $(INTEL_LIBS) -Wl,-rpath,$(MKL)/lib $(NETCDF_LIBS) -Wl,-rpath,$(NETCDF_ROOT)/lib $(NETCDFF_LIBS) -Wl,-rpath,$(NETCDFF_ROOT)/lib $(HDF5_LIBS) -Wl,-rpath,$(HDF5_ROOT)/lib $(OTHER_LIBS) -Wl,-rpath,$(HOME)/Packages/zlib-1.2.11/lib $(MPI_LIBS) -Wl,-rpath,$(HOME)/Packages/openmpi-3.0.1/lib \
-lmkl_blas95_lp64 -lmkl_intel_thread -lmkl_lapack95_lp64 -lmkl_scalapack_lp64 -lmkl_blacs_mpich_lp64 -lmkl_core -liomp5 -lpthread -lm -ldl -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lnetcdf -lnetcdff -lz -lmpi

I have included the new FPPFLAGS

FPPFLAGS = $(DEFS_PREFIX)-DFC_HAVE_ABORT -DMPI -DCDF -DNCDF_PARALLEL -DNCDF -DNCDF_4

I have used the "intel link advisor" https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor to help select the libraries. In 2018, there is only the BLACS MPICH library, which apparently you need to include along with the openmpi library -liomp5.

Anyhow, now, with the addition of the new FPPFLAGS, I believe, I get new errors before even getting to the linking stage.

Find attached the arch and log file.

https://jessequinn-my.sharepoint.com/:u:/g/personal/me_jessequinn_info/EYEZwmnX-EFFomXnrtR0dWsBAMD5Kmw4Z0QeMetQXB3L3w?e=mJQTgO

Regards,

Jesse Quinn (jessequinn) said : #3

Sorry I used

LIBS = -qopenmp $(INTEL_LIBS) -Wl,-rpath,$(MKL)/lib $(NETCDF_LIBS) -Wl,-rpath,$(NETCDF_ROOT)/lib $(NETCDFF_LIBS) -Wl,-rpath,$(NETCDFF_ROOT)/lib $(HDF5_LIBS) \
-Wl,-rpath,$(HDF5_ROOT)/lib $(OTHER_LIBS) -Wl,-rpath,$(HOME)/Packages/zlib-1.2.11/lib $(MPI_LIBS) -Wl,-rpath,$(HOME)/Packages/openmpi-3.0.1/lib \
-lmkl_scalapack_lp64 -lmkl_blacs_mpich_lp64 -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lnetcdf -lnetcdff -lz -lmpi

regardless of that order the results remains the same. I am attempting to compile without the parallel netcdf flags.

Jesse Quinn (jessequinn) said : #4

I have changed over to mpich. Let's see what happens.

Jesse Quinn (jessequinn) said : #5

Using mpich-3.1.4 and the following

FPPFLAGS = $(DEFS_PREFIX)-DFC_HAVE_ABORT -DMPI -DCDF

MKL = /opt/intel/mkl
INTEL_LIBS = -L$(MKL)/lib
MKL_INCLUDE = -I$(MKL)/include

NETCDF_ROOT = $(HOME)/Packages/netcdf-4.6.1
NETCDF_LIBS = -L$(NETCDF_ROOT)/lib
NETCDF_INCFLAGS = -I$(NETCDF_ROOT)/include

NETCDFF_ROOT = $(HOME)/Packages/netcdf-fortran-4.4.4
NETCDFF_LIBS = -L$(NETCDFF_ROOT)/lib
NETCDFF_INCFLAGS = -I$(NETCDFF_ROOT)/include

HDF5_ROOT = $(HOME)/Packages/hdf5-1.10.2
HDF5_LIBS = -L$(HDF5_ROOT)/lib
HDF5_INCFLAGS = -I$(HDF5_ROOT)/include

OTHER_LIBS = -L$(HOME)/Packages/zlib-1.2.11/lib
OTHER_INCLUDE = -L$(HOME)/Packages/zlib-1.2.11/include

MPI_INTERFACE = libmpi_f90.a
MPI_LIBS = -L$(HOME)/Packages/mpich-3.1.4/lib
MPI_INCLUDE = -I$(HOME)/Packages/mpich-3.1.4/include

LIBS = -qopenmp $(INTEL_LIBS) -Wl,-rpath,$(MKL)/lib $(NETCDF_LIBS) -Wl,-rpath,$(NETCDF_ROOT)/lib $(NETCDFF_LIBS) -Wl,-rpath,$(NETCDFF_ROOT)/lib $(HDF5_LIBS) \
-Wl,-rpath,$(HDF5_ROOT)/lib $(OTHER_LIBS) -Wl,-rpath,$(HOME)/Packages/zlib-1.2.11/lib $(MPI_LIBS) -Wl,-rpath,$(HOME)/Packages/mpich-3.1.4/lib \
-lmkl_scalapack_lp64 -lmkl_blacs_mpich_lp64 -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -ldl -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 -lnetcdf -lnetcdff -lz -lmpi

INCFLAGS = $(NETCDF_INCFLAGS) $(NETCDFF_INCFLAGS) $(HDF5_INCFLAGS) $(MPI_INCLUDE) $(MKL_INCLUDE) $(OTHER_INCLUDE)

same result as originally. MKL reference errors.

Please advise.

Nick Papior (nickpapior) said : #6

I would advice you to ask this question with the intel MKL people.

Linking errors are related to linking flags. So if you ask about your MacOSX and your specific intel version they can tell you what to do.

Jesse Quinn (jessequinn) said : #7

will do.

thanks

Jesse Quinn (jessequinn) said : #8

For those how are interested the following arch will compile Siesta 4.1-b3 on OSX with intel 2018.0 mkl and compilers

You need to build zlib, netcdf, netcdf fortran, hdf5 yourself along with mpich.

#
# Copyright (C) 1996-2016 The SIESTA group
# This file is distributed under the terms of the
# GNU General Public License: see COPYING in the top directory
# or http://www.gnu.org/copyleft/gpl.txt.
# See Docs/Contributors.txt for a list of contributors.
#
#-------------------------------------------------------------------
# arch.make file for gfortran compiler.
# To use this arch.make file you should rename it to
# arch.make
# or make a sym-link.
# For an explanation of the flags see DOCUMENTED-TEMPLATE.make

.SUFFIXES:
.SUFFIXES: .f .F .o .c .a .f90 .F90

SIESTA_ARCH = OSX

CC = mpicc
FPP = $(FC) -E -P
FC = mpif90
FC_SERIAL = ifort

FFLAGS = -O2 -xcore-avx2 -ip -mp1 -fpp -heap-arrays 1024 -warn unused,truncated_source,uncalled,declarations,usage
#FFLAGS = -O0 -g -check bounds -traceback
CFLAGS = -O2 -xcore-avx2 -ip -mp1 -Wall

AR = ar
RANLIB = ranlib

SYS = nag

SP_KIND = 4
DP_KIND = 8
KINDS = $(SP_KIND) $(DP_KIND)

LDFLAGS =

COMP_LIBS += libncdf.a libfdict.a

# Preprocessor definitions or flags.
# Here we use FPPFLAGS (as 'configure' calls them), but historically
# it was very common to use DEFS. Try to use only FPPFLAGS from now on,
# converting any old arch.make files you might have lying around, and
# remember that you have to change the final building rules at the end
# to use only FPPFLAGS. DEFS is deprecated.

# CDF and MPI are self-explanatory.
#FPPFLAGS_CDF = -DFC_HAVE_FLUSH -DFC_HAVE_ABORT -DCDF
FPPFLAGS_CDF = $(DEFS_PREFIX)-DFC_HAVE_ABORT -DCDF -DNCDF -DNCDF_4
FPPFLAGS_MPI = -DMPI

# Other definitions might be needed to work around some glitch in the
# compiler. For old versions of gfortran, add -DGFORTRAN.
FPPFLAGS = $(FPPFLAGS_CDF) $(FPPFLAGS_MPI) -DUSE_GEMM3M

MKL = /opt/intel/mkl
INTEL_LIBS = -L$(MKL)/lib
MKL_INCLUDE = -I$(MKL)/include

NETCDF_ROOT = $(HOME)/Packages/netcdf-4.6.1
NETCDF_LIBS = -L$(NETCDF_ROOT)/lib
NETCDF_INCFLAGS = -I$(NETCDF_ROOT)/include

NETCDFF_ROOT = $(HOME)/Packages/netcdf-fortran-4.4.4
NETCDFF_LIBS = -L$(NETCDFF_ROOT)/lib
NETCDFF_INCFLAGS = -I$(NETCDFF_ROOT)/include

HDF5_ROOT = $(HOME)/Packages/hdf5-1.10.2
HDF5_LIBS = -L$(HDF5_ROOT)/lib
HDF5_INCFLAGS = -I$(HDF5_ROOT)/include

OTHER_LIBS = -L$(HOME)/Packages/zlib-1.2.11/lib
OTHER_INCLUDE = -L$(HOME)/Packages/zlib-1.2.11/include

MPI_INTERFACE = libmpi_f90.a
MPI_LIBS = -L$(HOME)/Packages/mpich-3.1.4/lib
MPI_INCLUDE = -I$(HOME)/Packages/mpich-3.1.4/include

LIBS = -qopenmp $(MKL)/lib/libmkl_blas95_ilp64.a $(MKL)/lib/libmkl_lapack95_ilp64.a $(INTEL_LIBS) -Wl,-rpath,$(MKL)/lib \
-lmkl_scalapack_ilp64 -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_mpich_ilp64 -liomp5 -lpthread -lm -ldl \
$(HDF5_LIBS) -Wl,-rpath,$(HDF5_ROOT)/lib -lhdf5hl_fortran -lhdf5_fortran -lhdf5_hl -lhdf5 \
$(NETCDF_LIBS) -Wl,-rpath,$(NETCDF_ROOT)/lib -lnetcdf $(NETCDFF_LIBS) -Wl,-rpath,$(NETCDFF_ROOT)/lib -lnetcdff \
$(OTHER_LIBS) -Wl,-rpath,$(HOME)/Packages/zlib-1.2.11/lib -lz \
$(MPI_LIBS) -Wl,-rpath,$(HOME)/Packages/mpich-3.1.4/lib -lmpi $(COMP_LIBS)

INCFLAGS = $(MKL_INCLUDE) $(HDF5_INCFLAGS) $(NETCDF_INCFLAGS) $(NETCDFF_INCFLAGS) $(MPI_INCLUDE) $(OTHER_INCLUDE)

# Dependency rules ---------

FFLAGS_DEBUG = -O0 -xcore-avx2 -ip -mp1 -fpp -heap-arrays 1024 -warn unused,truncated_source,uncalled,declarations,usage
# your appropriate flags here...

# The atom.f code is very vulnerable. Particularly the Intel compiler
# will make an erroneous compilation of atom.f with high optimization
# levels.
atom.o: atom.F
 $(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<

.c.o:
 $(CC) -c $(CFLAGS) $(INCFLAGS) $(CPPFLAGS) $<
.F.o:
 $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
.F90.o:
 $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $<
.f.o:
 $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<
.f90.o:
 $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<

Nick Papior (nickpapior) said : #9

Thanks for the follow up!

Can you help with this problem?

Provide an answer of your own, or ask Jesse Quinn for more information if necessary.

To post a message you must log in.