error in compailing siesta by using MPI bur works for openMP+MPI

Asked by Yusuf Wicaksono

Dear Siesta Users/Developers,

I would like to ask your kind help.

I successfully compiled siesta-4.1-b4 by using openMP+ MPI and runs well. The arch.make file is as follows

#
# Copyright (C) 1996-2016 The SIESTA group
# This file is distributed under the terms of the
# GNU General Public License: see COPYING in the top directory
# or http://www.gnu.org/copyleft/gpl.txt.
# See Docs/Contributors.txt for a list of contributors.
#
#-------------------------------------------------------------------
# arch.make file for gfortran compiler.
# To use this arch.make file you should rename it to
# arch.make
# or make a sym-link.
# For an explanation of the flags see DOCUMENTED-TEMPLATE.make

.SUFFIXES:
.SUFFIXES: .f .F .o .c .a .f90 .F90

SIESTA_ARCH = unknown

CC = mpicc
FPP = $(FC) -E -P
FC = mpiifort
FC_SERIAL = ifort

FFLAGS = -O2 -fPIC -fp-model source

AR = ar
RANLIB = ranlib

SYS = nag

SP_KIND = 4
DP_KIND = 8
KINDS = $(SP_KIND) $(DP_KIND)

LDFLAGS =

MKL = /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl
LIBS += -L$(MKL)/lib/intel64 -lmkl_blas95_lp64 -lmkl_lapack95_lp64 -lmkl_intel_thread -lmkl_intel_lp64 -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64

FFLAGS += -qopenmp
LIBS += -qopenmp

FPPFLAGS = $(DEFS_PREFIX)-DFC_HAVE_ABORT -DMPI -DSCALAPACK

MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE=.

# Dependency rules ---------

FFLAGS_DEBUG = -g -O1 -fp-model source # your appropriate flags here...

# The atom.f code is very vulnerable. Particularly the Intel compiler
# will make an erroneous compilation of atom.f with high optimization
# levels.

atom.o: atom.F
        $(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
state_analysis.o: state_analysis.F
        $(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<

.c.o:
        $(CC) -c $(CFLAGS) $(INCFLAGS) $(CPPFLAGS) $<
.F.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
.F90.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $<
.f.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<
.f90.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<

INCFLAGS += -I/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/netcdf/include
LDFLAGS += -L/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/zlib/lib -Wl,-rpath=/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/zlib/lib
LDFLAGS += -L/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/hdf5/lib -Wl,-rpath=/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/hdf5/lib
LDFLAGS += -L/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/netcdf/lib -Wl,-rpath=/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/netcdf/lib
LIBS += -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lz
COMP_LIBS += libncdf.a libfdict.a
FPPFLAGS += -DCDF -DNCDF -DNCDF_4

However, when I compiled siesta-4.1-b4 only by using MPI (I compiled for the use of a system with medium to large size),
the following errors appear

denmat.o: ?? `m_denmat_mp_denmat_' ?:
denmat.F:(.text+0x1395): `dscal_' ????????????????
denmat.F:(.text+0x13b4): `dscal_' ????????????????
dhscf.o: ?? `m_dhscf_mp_dhscf_' ?:
dhscf.F:(.text+0x7343): `ddot_' ????????????????
dhscf.F:(.text+0x7370): `ddot_' ????????????????
dhscf.F:(.text+0x73e3): `ddot_' ????????????????
dhscf.F:(.text+0x7410): `ddot_' ????????????????
dhscf.F:(.text+0x7544): `ddot_' ????????????????
dhscf.o:dhscf.F:(.text+0x756d): `ddot_' ????????????????????????
inver.o: ?? `inver_' ?:
inver.f:(.text+0x1b0): `dgetrf_' ????????????????
inver.f:(.text+0x1ea): `dgetri_' ????????????????
ksv.o: ?? `m_ksv_mp_ksv_pol_' ?:
ksv.f:(.text+0xcb3): `ddot_' ????????????????
ksv.f:(.text+0xe92): `ddot_' ????????????????
ksv.f:(.text+0xee6): `ddot_' ????????????????
ksv.f:(.text+0x2db7): `ddot_' ????????????????
linpack.o: ?? `m_linpack_mp_zgedi_' ?:
linpack.F:(.text+0x168): `zscal_' ????????????????
linpack.F:(.text+0x1ec): `zaxpy_' ????????????????
linpack.F:(.text+0x4fb): `zaxpy_' ????????????????
linpack.F:(.text+0x563): `zswap_' ????????????????
linpack.o: ?? `m_linpack_mp_zgefa_' ?:
linpack.F:(.text+0x835): `zscal_' ????????????????
linpack.F:(.text+0x92a): `zaxpy_' ????????????????
m_new_dm.o: ?? `m_new_dm_mp_new_dm_' ?:
m_new_dm.F90:(.text+0x1b8b): `daxpy_' ????????????????
siesta_forces.o: ?? `m_siesta_forcessiesta_forces_mp_transiesta_switch_' ?:
siesta_forces.F:(.text+0x3ead): `daxpy_' ????????????????
siesta_forces.F:(.text+0x4c9a): `daxpy_' ????????????????
m_broyddj.o: ?? `m_broyddj_mp_dlinds_' ?:
m_broyddj.f90:(.text+0x7a6d): `dgetrf_' ????????????????
m_broyddj.f90:(.text+0x7b0c): `dgetri_' ????????????????
mneighb.o: ?? `neighbour_mp_mranger_' ?:
mneighb.f:(.text+0x75d): `ddot_' ????????????????
mneighb.f:(.text+0x900): `ddot_' ????????????????
mneighb.f:(.text+0x1629): `ddot_' ????????????????
mneighb.f:(.text+0x1ecf): `ddot_' ????????????????
conjgr.o: ?? `m_conjgr_mp_conjgr_' ?:
conjgr.f:(.text+0x10a): `ddot_' ????????????????
conjgr.o:conjgr.f:(.text+0x132): `ddot_' ????????????????????????
m_broyddj_nocomm.o: ?? `m_broyddj_nocomm_mp_dlinds_' ?:
m_broyddj_nocomm.f90:(.text+0x860d): `dgetrf_' ????????????????
m_broyddj_nocomm.f90:(.text+0x86ac): `dgetri_' ????????????????
final_H_f_stress.o: ?? `m_final_h_f_stress_mp_final_h_f_stress_' ?:
final_H_f_stress.F:(.text+0x6428): `daxpy_' ????????????????
final_H_f_stress.F:(.text+0x66b8): `daxpy_' ????????????????
save_density_matrix.o: ?? `m_save_density_matrix_mp_save_density_matrix_' ?:
save_density_matrix.F90:(.text+0x473): `daxpy_' ????????????????
save_density_matrix.F90:(.text+0x5b5): `daxpy_' ????????????????
m_filter.o: ?? `m_filter_mp_filter_rdiag_' ?:
m_filter.f90:(.text+0x5476): `ilaenv_' ????????????????
m_filter.f90:(.text+0x58d6): `dsyevx_' ????????????????
m_dminim.o: ?? `m_dminim_mp_minim_cg_sparse_' ?:
m_dminim.F90:(.text+0x9e61): `ddot_' ????????????????
m_dminim.F90:(.text+0x9ea9): `ddot_' ????????????????
m_dminim.F90:(.text+0xb3c6): `ddot_' ????????????????
m_dminim.F90:(.text+0xd4ab): `ddot_' ????????????????
m_dminim.F90:(.text+0xd4f7): `ddot_' ????????????????
m_dminim.o:m_dminim.F90:(.text+0x14687): `ddot_' ????????????????????????
m_svd.o: ?? `m_svd_mp_solve_with_svd_' ?:
m_svd.F90:(.text+0x8c9): `dgelss_' ????????????????
diag.o: ?? `m_diag_mp_diag_c_' ?:
diag.F90:(.text+0x132c): `zpotrf_' ????????????????
diag.F90:(.text+0x1b14): `zhegst_' ????????????????
diag.F90:(.text+0x2ee8): `zheevx_' ????????????????
diag.F90:(.text+0x320b): `zheev_' ????????????????
diag.F90:(.text+0x347d): `zcopy_' ????????????????
diag.F90:(.text+0x3941): `zheevd_' ????????????????
diag.F90:(.text+0x3cf5): `zcopy_' ????????????????
diag.F90:(.text+0x7167): `ztrsm_' ????????????????
diag.F90:(.text+0x812a): `dscal_' ????????????????
diag.o: ?? `m_diagdiag_c_mp_work_query_' ?:
diag.F90:(.text+0x8bf7): `zheevx_' ????????????????
diag.F90:(.text+0x8fb6): `zheev_' ????????????????
diag.F90:(.text+0x9774): `zheevd_' ????????????????
diag.o: ?? `m_diag_mp_diag_r_' ?:
diag.F90:(.text+0xfa54): `dpotrf_' ????????????????
diag.F90:(.text+0x1034b): `dsygst_' ????????????????
diag.F90:(.text+0x11bbe): `dsyevx_' ????????????????
diag.F90:(.text+0x120cb): `dsyev_' ????????????????
diag.F90:(.text+0x12224): `dcopy_' ????????????????
diag.F90:(.text+0x12552): `dsyevd_' ????????????????
diag.F90:(.text+0x127d6): `dcopy_' ????????????????
diag.F90:(.text+0x15cf5): `dtrsm_' ????????????????
diag.F90:(.text+0x16e14): `dscal_' ????????????????
diag.o: ?? `m_diagdiag_r_mp_work_query_' ?:
diag.F90:(.text+0x17e1e): `dsyevx_' ????????????????
diag.F90:(.text+0x183d3): `dsyev_' ????????????????
diag.F90:(.text+0x18870): `dsyevd_' ????????????????
m_mixing.o: ?? `m_mixing_mp_svd_' ?:
m_mixing.F90:(.text+0x7304): `dgelss_' ????????????????
m_mixing.o: ?? `m_mixing_mp_inverse_' ?:
m_mixing.F90:(.text+0x7862): `dgetrf_' ????????????????
m_mixing.F90:(.text+0x789c): `dgetri_' ????????????????
m_mixing.o: ?? `m_mixing_mp_update_f_' ?:
m_mixing.F90:(.text+0xc4c9): `dcopy_' ????????????????
m_mixing.o: ?? `m_mixing_mp_push_f_' ?:
m_mixing.F90:(.text+0xdc0b): `dcopy_' ????????????????
m_trimat_invert.o: ?? `m_trimat_invert_mp_calc_xn_div_cn_p1_' ?:
m_trimat_invert.F90:(.text+0x1887): `zgemm_' ????????????????
m_trimat_invert.F90:(.text+0x1c35): `zgesv_' ????????????????
m_trimat_invert.o: ?? `m_trimat_invert_mp_calc_yn_div_bn_m1_' ?:
m_trimat_invert.F90:(.text+0x296f): `zgemm_' ????????????????
m_trimat_invert.F90:(.text+0x2d19): `zgesv_' ????????????????
m_trimat_invert.o: ?? `m_trimat_invert_mp_calc_mnn_inv_' ?:
m_trimat_invert.F90:(.text+0x370f): `zgemm_' ????????????????
m_trimat_invert.F90:(.text+0x4027): `zgemm_' ????????????????
m_trimat_invert.F90:(.text+0x4911): `zgemm_' ????????????????
m_trimat_invert.F90:(.text+0x51f1): `zgesv_' ????????????????
m_trimat_invert.F90:(.text+0x59d4): `zgemm_' ????????????????
m_trimat_invert.o: ?? `m_trimat_invert_mp_calc_mnm1n_inv_' ?:
m_trimat_invert.F90:(.text+0x615e): `zcopy_' ????????????????
m_trimat_invert.F90:(.text+0x6880): `zgemm_' ????????????????
m_trimat_invert.o: ?? `m_trimat_invert_mp_calc_mnp1n_inv_' ?:
m_trimat_invert.F90:(.text+0x7006): `zcopy_' ????????????????
m_trimat_invert.F90:(.text+0x7724): `zgemm_' ????????????????
m_diis.o: ?? `m_diisdiis_mp_inverse_' ?:
m_diis.F90:(.text+0x2065): `dgetrf_' ????????????????
m_diis.F90:(.text+0x2116): `dgetri_' ????????????????
m_gauss_quad.o: ?? `m_gauss_quad_mp_solve_golub_welsch_' ?:
m_gauss_quad.f90:(.text+0x26f9): `dsyev_' ????????????????
m_gauss_quad.f90:(.text+0x286e): `dsyev_' ????????????????
m_mat_invert.o: ?? `m_mat_invert_mp_mat_invert_recursive_' ?:
m_mat_invert.F90:(.text+0x369): `zgesv_' ????????????????
m_mat_invert.F90:(.text+0x3d9): `zgesv_' ????????????????
m_mat_invert.F90:(.text+0x498): `zgemm_' ????????????????
m_mat_invert.F90:(.text+0x523): `zgemm_' ????????????????
m_mat_invert.F90:(.text+0x553): `zgetrf_' ????????????????
m_mat_invert.F90:(.text+0x5a8): `zgetrf_' ????????????????
m_mat_invert.F90:(.text+0x752): `zgetri_' ????????????????
m_mat_invert.F90:(.text+0x79c): `zgetri_' ????????????????
m_mat_invert.F90:(.text+0x84a): `zgemm_' ????????????????
m_mat_invert.F90:(.text+0x8c6): `zgemm_' ????????????????
m_mat_invert.o: ?? `m_mat_invert_mp_mat_invert_' ?:
m_mat_invert.F90:(.text+0xd79): `zgesv_' ????????????????
m_mat_invert.F90:(.text+0xddb): `zgetrf_' ????????????????
m_mat_invert.F90:(.text+0xe14): `zgetri_' ????????????????
m_geom_box.o: ?? `m_geom_box_mp_fgeo_read_box_delta_' ?:
m_geom_box.f90:(.text+0x543): `dgetrf_' ????????????????
m_geom_box.f90:(.text+0x591): `dgetri_' ????????????????
m_geom_square.o: ?? `m_geom_square_mp_p_voxel_square_' ?:
m_geom_square.f90:(.text+0x147): `dgesv_' ????????????????
m_ts_electrode.o: ?? `m_ts_electrode_mp_ssr_sgreen_nodos_' ?:
m_ts_electrode.F90:(.text+0xd55a): `zgesv_' ????????????????
m_ts_electrode.F90:(.text+0xd8fe): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0xdb05): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0xdd20): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0xde59): `zgemm_' ????????????????
m_ts_electrode.o: ?? `m_ts_electrode_mp_ssr_sgreen_dos_' ?:
m_ts_electrode.F90:(.text+0xfc17): `zgesv_' ????????????????
m_ts_electrode.F90:(.text+0xffba): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x101d0): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x103f8): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x108c5): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x11f55): `zgemm_' ????????????????
m_ts_electrode.o:m_ts_electrode.F90:(.text+0x123ac): `zgemm_' ????????????????????????
m_ts_electrode.o: ?? `m_ts_electrode_mp_ssr_sgreen_dos_' ?:
m_ts_electrode.F90:(.text+0x12813): `zdotu_' ????????????????
m_ts_electrode.F90:(.text+0x13479): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x13751): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x139fd): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x13d54): `zgemm_' ????????????????
m_ts_electrode.F90:(.text+0x13ebf): `zdotu_' ????????????????
m_ts_electrode.F90:(.text+0x13efc): `zdotc_' ????????????????
m_ts_electrode.F90:(.text+0x13f38): `zdotu_' ????????????????
m_ts_electype.o: ?? `m_ts_electype_mp_elec_box2grididx_' ?:
m_ts_electype.F90:(.text+0x8362): `dgesv_' ????????????????
m_ts_electype.F90:(.text+0x854e): `dgesv_' ????????????????
m_ts_electype.F90:(.text+0x8737): `dgesv_' ????????????????
m_ts_electype.F90:(.text+0x8936): `dgesv_' ????????????????
m_ts_electype.F90:(.text+0x8b31): `dgesv_' ????????????????
m_ts_electype.o:m_ts_electype.F90:(.text+0x8d32): `dgesv_' ????????????????????????
m_ts_electype.o: ?? `m_ts_electype_mp_copy_dm_' ?:
m_ts_electype.F90:(.text+0xd16b): `daxpy_' ????????????????
m_ts_electype.o: ?? `m_ts_electype_mp_init_elec_sim_' ?:
m_ts_electype.F90:(.text+0x1044b): `dgetrf_' ????????????????
m_ts_electype.F90:(.text+0x10499): `dgetri_' ????????????????
m_ts_elec_se.o: ?? `m_ts_elec_se_mp_uc_expansion_sigma_' ?:
m_ts_elec_se.F90:(.text+0x1f66): `zgetrf_' ????????????????
m_ts_elec_se.F90:(.text+0x2019): `zgetri_' ????????????????
m_ts_elec_se.F90:(.text+0x2209): `zcopy_' ????????????????
m_ts_elec_se.o: ?? `m_ts_elec_se_mp_uc_expansion_sigma_bulk_' ?:
m_ts_elec_se.F90:(.text+0x426f): `zgesv_' ????????????????
m_ts_elec_se.F90:(.text+0x450e): `zcopy_' ????????????????
m_ts_elec_se.o: ?? `m_ts_elec_se_mp_uc_expansion_sigma_gammat_' ?:
m_ts_elec_se.F90:(.text+0x6911): `zgetrf_' ????????????????
m_ts_elec_se.F90:(.text+0x69cc): `zgetri_' ????????????????
m_ts_elec_se.F90:(.text+0x72b6): `zcopy_' ????????????????
m_transiesta.o: ?? `m_transiesta_mp_transiesta_' ?:
m_transiesta.F90:(.text+0x1a13): `daxpy_' ????????????????
m_ts_contour_eq.o: ?? `m_ts_contour_eq_mp_contour_continued_fraction_' ?:
m_ts_contour_eq.f90:(.text+0xd95a): `dsygv_' ????????????????
m_ts_dm_update.o: ?? `m_ts_dm_update_mp_update_dm_' ?:
m_ts_dm_update.F90:(.text+0x418c): `daxpy_' ????????????????
m_ts_dm_update.o: ?? `m_ts_dm_update_mp_update_zdm_' ?:
m_ts_dm_update.F90:(.text+0x5271): `zaxpy_' ????????????????
m_ts_pivot.o: ?? `m_ts_pivot_mp_ts_pivot_' ?:
m_ts_pivot.F90:(.text+0x5dc4): `dgels_' ????????????????
m_ts_full_scat.o: ?? `m_ts_full_scat_mp_gf_gamma_gf_' ?:
m_ts_full_scat.F90:(.text+0x441): `zgemm_' ????????????????
m_ts_full_scat.F90:(.text+0x4b5): `zgemm_' ????????????????
m_ts_full_scat.F90:(.text+0x885): `zgemm_' ????????????????
m_ts_full_scat.F90:(.text+0x8f2): `zgemm_' ????????????????
m_ts_full_scat.o: ?? `m_ts_full_scat_mp_calc_gf_' ?:
m_ts_full_scat.F90:(.text+0x9b4): `zgesv_' ????????????????
m_ts_full_scat.o: ?? `m_ts_full_scat_mp_calc_gf_bias_' ?:
m_ts_full_scat.F90:(.text+0xe9b): `zgesv_' ????????????????
m_ts_full_scat.o: ?? `m_ts_full_scat_mp_calc_gf_part_' ?:
m_ts_full_scat.F90:(.text+0x155a): `zgesv_' ????????????????
m_ts_trimat_invert.o: ?? `m_ts_trimat_invert_mp_calc_mnn_inv_cols_' ?:
m_ts_trimat_invert.F90:(.text+0x10ac): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x1a18): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x232d): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x2a53): `zgesv_' ????????????????
m_ts_trimat_invert.F90:(.text+0x3134): `zgemm_' ????????????????
m_ts_trimat_invert.o: ?? `m_ts_trimat_invert_mp_invert_biastrimat_col_' ?:
m_ts_trimat_invert.F90:(.text+0x3acb): `zcopy_' ????????????????
m_ts_trimat_invert.F90:(.text+0x3e90): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x43a2): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x4941): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x4e30): `zgemm_' ????????????????
m_ts_trimat_invert.o: ?? `m_ts_trimat_invert_mp_invert_biastrimat_rgn_' ?:
m_ts_trimat_invert.F90:(.text+0x5821): `zcopy_' ????????????????
m_ts_trimat_invert.F90:(.text+0x5b8b): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x617f): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x6783): `zgemm_' ????????????????
m_ts_trimat_invert.F90:(.text+0x6ca5): `zgemm_' ????????????????
m_ts_tri_scat.o: ?? `m_ts_tri_scat_mp_gf_gamma_gf_' ?:
m_ts_tri_scat.F90:(.text+0x66d): `zgemm_' ????????????????
m_ts_tri_scat.o:m_ts_tri_scat.F90:(.text+0xc0b): `zgemm_' ????????????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssteqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsyr2k_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cscal_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `MKL_Dimatcopy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `caxpy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsteqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cbdsqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `clanhs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sger_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlartg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssyr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dbdsqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssytrd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `MKL_Zomatcopy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssyr2k_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slasq2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slasr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlartg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ieeeck_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dpttrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `drot_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dstedc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ztrtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sgetrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarra_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsymm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlapy3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zrot_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dswap_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_domatcopy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `classq_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slamc3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zdotc' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dgemm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dgemm' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_iface_allocate' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlacpy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ctrmm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarfg' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlabad_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_iface_deallocate' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ztrmv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlacpy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cher_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slasrt_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `strtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `scnrm2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlasr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_print' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dnrm2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zpbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlasrt_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dtrsv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgetrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlae2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlacgv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `strsv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlanhs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlassq_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slapy2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zcopy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlacpy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slassq_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `lsame_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlasy2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sswap_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `disnan_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaqr0_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `xerbla_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarrc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarrk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sscal_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_free_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlange_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_set_progress_interface' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sdot_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slascl_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgeru_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ztbtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsyr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so: `MKL_calloc' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarra_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaswp_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `csscal_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ccopy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaqr0_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `chetrd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `stbtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaqr1_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaset_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgemm' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlarfg' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sgbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slamch_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsytrd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssymm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaqr4_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsyr2' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ztrmm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_malloc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slapy3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgemv' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaq6_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ctbtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlacpy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slange_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `csymm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlapy2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zhemv' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarrb_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaev2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zsyrk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaq5_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_memcpy_unbounded_s' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `daxpy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarnv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaswp_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlassq_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cherk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaqr4_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaq6_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgetrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgemv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssymv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cpbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlanst_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgemv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaqr1_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zhemv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_printf_s' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaq5_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `clacpy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zherk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `chemm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slae2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cher2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgetrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgerc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `clasr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zsymm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dtbtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlascl_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dgbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so: `mkl_serv_iface_malloc' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zher_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaset' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlamc3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaqr3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slacpy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `scasum_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `clarfg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarrv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zaxpy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dtrmm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssterf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `scsum1_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarrv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsymv' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `clacgv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zher2k_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so: `MKL_free' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssyrk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ctrsv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlasq2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sbdsqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_allocate' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sstedc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so: `mkl_serv_vsprintf_s' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `claswp_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dznrm2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sgetrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlaset_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarfx_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ddot' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarrd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slasy2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaqr3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `izmax1_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dger_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so: `MKL_malloc' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `MKL_Domatcopy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_lsame' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_fprintf_stdout_s' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zbdsqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlarfg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `icmax1_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slagts_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlaswp_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgemm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `idamax_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ctrsm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sasum_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ztrmm' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `clartg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ctrtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlanv2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slahqr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `crot_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zsyr2k_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cpotrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ztrsv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlagtf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dcopy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_memcpy_s' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaebz_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `spotrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sisnan_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_get_mpi_wrappers' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zlaset' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `strmm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cpttrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `isamax_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarfg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `chemv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dgemv' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_domatadd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zpttrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgerc' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlasr_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dtrtrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `saxpy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarnv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dapply_2hv' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `MKL_Zimatcopy' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dger' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sgemm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlagts_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dpbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaebz_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsyrk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zhetrd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `snrm2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dasum_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slasr3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slartg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `srot_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zgeru_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `izamax_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zhemm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ctrmv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dtrmv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaev2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dgetrs_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `icamax_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `strmv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slanst_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarfg_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarrd_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsterf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `spttrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dzasum_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zher2' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so: `mkl_serv_iface_free' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `claset_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `strsm_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dtrmm' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zdotu' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `sgemv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cgerc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `csyrk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_deallocate' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `ssyr2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slagtf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarfx_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slaed4_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaed4_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `csyr2k_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_fprintf_stderr_s' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarrb_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlaset_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsymv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `scopy_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `mkl_serv_progress' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `spbtrf_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlarrc_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dsyr2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slarrk_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slanv2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlasr3_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zdscal_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cher2k_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dgemv_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zher2_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dzsum1_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlamch_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cdecl_progress' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `slabad_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `cswap_' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `zapply_2hv' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_scalapack_lp64.so: `dlahqr_' ????????????????
make: *** [siesta] Error 1

The arch.make file is as follows

#
# Copyright (C) 1996-2016 The SIESTA group
# This file is distributed under the terms of the
# GNU General Public License: see COPYING in the top directory
# or http://www.gnu.org/copyleft/gpl.txt.
# See Docs/Contributors.txt for a list of contributors.
#
#-------------------------------------------------------------------
# arch.make file for gfortran compiler.
# To use this arch.make file you should rename it to
# arch.make
# or make a sym-link.
# For an explanation of the flags see DOCUMENTED-TEMPLATE.make

.SUFFIXES:
.SUFFIXES: .f .F .o .c .a .f90 .F90

SIESTA_ARCH = unknown

CC = mpicc
FPP = $(FC) -E -P
FC = mpiifort
FC_SERIAL = ifort

FFLAGS = -O2 -fPIC -fp-model source

AR = ar
RANLIB = ranlib

SYS = nag

SP_KIND = 4
DP_KIND = 8
KINDS = $(SP_KIND) $(DP_KIND)

LDFLAGS =

MKL = /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl
LIBS += -L$(MKL)/lib/intel64 -lmkl_blas95_lp64
LIBS += -L$(MKL)/lib/intel64 -lmkl_lapack95_lp64
LIBS += -L$(MKL)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64

FPPFLAGS = $(DEFS_PREFIX)-DFC_HAVE_ABORT

MPI_INTERFACE=libmpi_f90.a
MPI_INCLUDE =.
FPPFLAGS += -DMPI

# Dependency rules ---------

FFLAGS_DEBUG = -g -O1 -fp-model source # your appropriate flags here...

# The atom.f code is very vulnerable. Particularly the Intel compiler
# will make an erroneous compilation of atom.f with high optimization
# levels.

atom.o: atom.F
        $(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
state_analysis.o: state_analysis.F
        $(FC) -c $(FFLAGS_DEBUG) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<

.c.o:
        $(CC) -c $(CFLAGS) $(INCFLAGS) $(CPPFLAGS) $<
.F.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<
.F90.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $<
.f.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<
.f90.o:
        $(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<

INCFLAGS += -I/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/netcdf/include
LDFLAGS += -L/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/zlib/lib -Wl,-rpath=/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/zlib/lib
LDFLAGS += -L/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/hdf5/lib -Wl,-rpath=/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/hdf5/lib
LDFLAGS += -L/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/netcdf/lib -Wl,-rpath=/home/usr5/m70315e/siesta-4.1-b4/Obj_intel/netcdf_install/netcdf/lib
LIBS += -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lz
COMP_LIBS += libncdf.a libfdict.a
FPPFLAGS += -DCDF -DNCDF -DNCDF_4

Could you kindly help me to solve this problem? I appreciate your kind help.

Best,

Yusuf Wicaksono
Master Students
Department of Materials Science and Engineering
Osaka University

Question information

Language:
English Edit question
Status:
Solved
For:
Siesta Edit question
Assignee:
No assignee Edit question
Solved by:
Nick Papior
Solved:
Last query:
Last reply:
Revision history for this message
Nick Papior (nickpapior) said :
#1

The order of libraries matters:

Change this line:
LIBS += -L$(MKL)/lib/intel64 -lmkl_blas95_lp64 -lmkl_lapack95_lp64 -lmkl_intel_thread -lmkl_intel_lp64 -lmkl_core -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64

Into:

LIBS += -L$(MKL)/lib/intel64 -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64 -lmkl_lapack95_lp64 -lmkl_blas95_lp64 -lmkl_intel_thread -lmkl_intel_lp64 -lmkl_core

Revision history for this message
Yusuf Wicaksono (yusufwicaksono15) said :
#2

Dear Dr. Nick Papior,

Thank you very much for your kind answer and help.

I tried your suggestion. However, the errors still occur but now are reduced as follows,

/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_get_nested' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_critical' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_for_static_init_8' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_float8_add' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_dispatch_next_4' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_master' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_for_static_fini' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_reduce' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_flush' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_for_static_init_8u' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_for_static_init_4' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_fork_call' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_dispatch_next_8' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_push_num_threads' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_get_num_procs' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_ordered' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_omp_task_alloc' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_barrier' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_cmplx8_add' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_omp_task' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_global_thread_num' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_dispatch_init_4' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_get_max_threads' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_critical' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_serialized_parallel' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_fixed4_add' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_omp_task_with_deps' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_reduce_nowait' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_float4_max' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_master' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_single' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_dispatch_fini_8' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_set_nested' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_in_parallel' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_dispatch_init_8' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_fixed8_add' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_cmplx4_add' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_ok_to_fork' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_single' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_get_thread_num' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_ordered' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_end_reduce_nowait' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_serialized_parallel' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_float4_add' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `omp_get_num_threads' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_atomic_float8_max' ????????????????
/home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so: `__kmpc_reduce' ????????????????
make: *** [siesta] Error 1

Could you help me again to fix this problem?

Thank you very much for your kind consideration.

Revision history for this message
Nick Papior (nickpapior) said :
#3

Add -qopenmp to your flags and libraries.

On Mon, 26 Aug 2019, 16:22 Yusuf Wicaksono, <
<email address hidden>> wrote:

> Question #683326 on Siesta changed:
> https://answers.launchpad.net/siesta/+question/683326
>
> Status: Answered => Open
>
> Yusuf Wicaksono is still having a problem:
> Dear Dr. Nick Papior,
>
> Thank you very much for your kind answer and help.
>
> I tried your suggestion. However, the errors still occur but now are
> reduced as follows,
>
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_get_nested' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_critical' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_for_static_init_8' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_float8_add' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_dispatch_next_4' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_master' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_for_static_fini' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_reduce' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_flush' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_for_static_init_8u' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_for_static_init_4' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_fork_call' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_dispatch_next_8' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_push_num_threads' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_get_num_procs' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_ordered' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_omp_task_alloc' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_barrier' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_cmplx8_add' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_omp_task' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_global_thread_num' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_dispatch_init_4' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_get_max_threads' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_critical' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_serialized_parallel' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_fixed4_add' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_omp_task_with_deps' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_reduce_nowait' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_float4_max' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_master' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_single' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_dispatch_fini_8' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_set_nested' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_in_parallel' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_dispatch_init_8' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_fixed8_add' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_cmplx4_add' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_ok_to_fork' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_single' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_get_thread_num' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_ordered' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_end_reduce_nowait' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_serialized_parallel' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_float4_add' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `omp_get_num_threads' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_atomic_float8_max' ????????????????
> /home/app/intel/intel2018_up3/compilers_and_libraries_2018.3.222/linux/mkl/lib/intel64/libmkl_intel_thread.so:
> `__kmpc_reduce' ????????????????
> make: *** [siesta] Error 1
>
>
> Could you help me again to fix this problem?
>
> Thank you very much for your kind consideration.
>
> --
> You received this question notification because you are an answer
> contact for Siesta.
>

Revision history for this message
Yusuf Wicaksono (yusufwicaksono15) said :
#4

Dear Dr. Nick Papior,

Thank you very much for your suggestion.

By adding -qopenmp to the flags and libraries, the compilation works well.

However, does it mean the compilation of siesta using MPI + OpenMP?

Is it possible for me to compile siesta only by using MPI?

Revision history for this message
Best Nick Papior (nickpapior) said :
#5

Please read the manual, it describes clearly the difference between an openmp and non openmp compilation

Revision history for this message
Yusuf Wicaksono (yusufwicaksono15) said :
#6

Thanks Nick Papior, that solved my question.