Some cluster runs fail while others seem to run fine

Hi Peter,

According to your log this is a compilation problem. My guess is that some of the node miss some of the gfortran library that you need to compile the code.

This being said, I have a couple of comment/concern:
1) You are not using the recommended way to submit jobs on a cluster.
The recommend way is to change to tell MadGraph to do the cluster submission:
for this you can edit the file input/mg5_configuration.txt
and set
run_mode on 1
and
cluster_type on pbs

2) Did you specify the seed for your runs? if not, it is likely that you generate each times the same sample of events (depend on how exactly how you generate your sample of events).
So please check that point before using your multiple samples.

Cheers,

Olivier

On Jul 6, 2014, at 6:41 PM, Peter Winslow <email address hidden> wrote:

> New question #251271 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/251271
>
> Hi,
>
> I am running MG5_aMC_v2_1_1 and trying to generate 1,000,000 events for a simple process on my local cluster. I'm finding a very strange error. I submit 20 separate jobs (instances of MG which run this particular process) to our local PBS cluster. Each job is identical to the other, I simply submit the same 50,000 event generation run 20 separate times to eventually get the desired 1,000,000 events in the end. Some number of these jobs will run (at least they are continuing to run so far) but others are quickly stopped in error. I've attached the 1) output.log, 2) error.log, and 3) debug.log files below. Do you happen to have any idea what could be causing such errors in only some of the runs?
>
> Peter
>
> ** output.log file **
>
> seed: 6
> HOSTNAME=titan34
> PBS_VERSION=TORQUE-2.3.7
> SHELL=/bin/bash
> HISTSIZE=1000
> PBS_JOBNAME=job6
> PBS_ENVIRONMENT=PBS_BATCH
> QTDIR=/usr/lib64/qt-3.3
> QTINC=/usr/lib64/qt-3.3/include
> PBS_O_WORKDIR=/scratch/pwinslow/MGjobs/jjWWBG/job6
> USER=pwinslow
> PBS_TASKNUM=1
> PBS_O_HOME=/home/pwinslow
> PBS_MOMPORT=15003
> VIRTUAL_ENV=/scratch/pwinslow/CosmoTransitions/User
> PBS_O_QUEUE=feed
> PATH=/scratch/pwinslow/CosmoTransitions/User/bin:/home/pwinslow/.local/bin:/home/pwinslow/ActivePython-2.7/bin:/home/pwinslow/bin:/usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/pwinslow/bin
> PBS_O_LOGNAME=pwinslow
> MAIL=/var/spool/mail/pwinslow
> PBS_O_LANG=en_US.UTF-8
> PBS_JOBCOOKIE=DA569D966BD5993EEFB5F6ACFE571D52
> PWD=/home/pwinslow
> LANG=en_US.UTF-8
> PBS_NODENUM=0
> MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles
> LOADEDMODULES=
> PBS_O_SHELL=/bin/bash
> PBS_SERVER=titan.physics.umass.edu
> PBS_JOBID=920893.titan.physics.umass.edu
> ENVIRONMENT=BATCH
> HISTCONTROL=ignoredups
> SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
> HOME=/home/pwinslow
> SHLVL=2
> PBS_O_HOST=titan.physics.umass.edu
> PBS_VNODENUM=0
> LOGNAME=pwinslow
> CVS_RSH=ssh
> QTLIB=/usr/lib64/qt-3.3/lib
> PBS_QUEUE=express
> MODULESHOME=/usr/share/Modules
> PBS_O_MAIL=/var/spool/mail/pwinslow
> LESSOPEN=|/usr/bin/lesspipe.sh %s
> PBS_NODEFILE=/var/spool/torque/aux//920893.titan.physics.umass.edu
> G_BROKEN_FILENAMES=1
> PBS_O_PATH=/scratch/pwinslow/CosmoTransitions/User/bin:/home/pwinslow/.local/bin:/home/pwinslow/ActivePython-2.7/bin:/home/pwinslow/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/pwinslow/bin
> module=() { eval `/usr/bin/modulecmd bash $*`
> }
> _=/usr/bin/printenv
> ************************************************************
> * *
> * W E L C O M E to *
> * M A D G R A P H 5 _ a M C @ N L O *
> * M A D E V E N T *
> * *
> * * * *
> * * * * * *
> * * * * * 5 * * * * *
> * * * * * *
> * * * *
> * *
> * VERSION 5.2.1.2 *
> * *
> * The MadGraph5_aMC@NLO Development Team - Find us at *
> * https://server06.fynu.ucl.ac.be/projects/madgraph *
> * *
> * Type 'help' for in-line help. *
> * *
> ************************************************************
> INFO: load configuration from /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Cards/me5_configuration.txt
> INFO: load configuration from /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/input/mg5_configuration.txt
> INFO: load configuration from /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Cards/me5_configuration.txt
> Using default text editor "vi". Set another one in ./input/mg5_configuration.txt
> Using default eps viewer "evince". Set another one in ./input/mg5_configuration.txt
> Using default web browser "firefox". Set another one in ./input/mg5_configuration.txt
> No valid Delphes path found
> generate_events -f run6 --nb_core=1
> Generating 50000 events with run name run6
> survey run6
> INFO: compile directory
> Since icckw>0, We change the value of 'drjl' to 0
> quit
> INFO:
>
> INFO:
>
> launch in debug mode
>
>
>
> ** error.log file **
>
> Error detected in "generate_events -f run6 --nb_core=1"
> write debug file /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/run6_tag_1_debug.log
> If you need help with this issue please contact us on https://answers.launchpad.net/madgraph5
> MadGraph5Error : A compilation Error occurs when trying to compile /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Source.
> The compilation fails with the following output message:
> gfortran -O -w -fbounds-check -ffixed-line-length-132 -c -o gen_ximprove.o gen_ximprove.f
> gfortran -O -w -fbounds-check -ffixed-line-length-132 -o ../bin/internal/gen_ximprove gen_ximprove.o ranmar.o rw_routines.o open_file.o
> ranmar.o: In function `rmarin_':
> ranmar.f:(.text+0xb1): undefined reference to `_gfortran_copy_string'
> ranmar.f:(.text+0x136): undefined reference to `_gfortran_copy_string'
> ranmar.o: In function `get_base_':
> ranmar.f:(.text+0x9bf): undefined reference to `_gfortran_copy_string'
> ranmar.f:(.text+0xa8b): undefined reference to `_gfortran_copy_string'
> ranmar.f:(.text+0xac9): undefined reference to `_gfortran_copy_string'
> ranmar.o:ranmar.f:(.text+0xb8d): more undefined references to `_gfortran_copy_string' follow
> rw_routines.o: In function `load_para_':
> rw_routines.f:(.text+0xe22): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0xe6f): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0xe77): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0xe9a): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0xeca): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1050): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1078): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x10b2): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x10cc): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x1114): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x111e): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1146): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1180): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1423): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x144b): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1485): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x149c): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x14e6): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x14f0): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1518): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1552): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x16de): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1706): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1740): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x175b): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x17a2): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x17ab): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x17d3): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x180d): undefined reference to `_gfortran_copy_string'
> rw_routines.o: In function `load_gridpack_para_':
> rw_routines.f:(.text+0x1899): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x18b0): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1b18): undefined reference to `_gfortran_copy_string'
> rw_routines.o:rw_routines.f:(.text+0x1b3c): more undefined references to `_gfortran_copy_string' follow
> rw_routines.o: In function `load_gridpack_para_':
> rw_routines.f:(.text+0x1b8f): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x1bd7): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1be1): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1c05): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1c3b): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1dbf): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1de3): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1e19): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1e36): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x1e7d): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1e87): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1eab): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1ee1): undefined reference to `_gfortran_copy_string'
> open_file.o: In function `open_file_':
> open_file.f:(.text+0x4b): undefined reference to `_gfortran_copy_string'
> open_file.f:(.text+0x193): undefined reference to `_gfortran_internal_malloc'
> open_file.f:(.text+0x1cc): undefined reference to `_gfortran_copy_string'
> open_file.f:(.text+0x1d4): undefined reference to `_gfortran_internal_free'
> open_file.f:(.text+0x282): undefined reference to `_gfortran_copy_string'
> collect2: ld returned 1 exit status
> make: *** [../bin/internal/gen_ximprove] Error 1
>
> Please try to fix this compilations issue and retry.
> Help might be found at https://answers.launchpad.net/madgraph5.
> If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
>
>
>
>
> ** debug.log file **
>
> #************************************************************
> #* MadGraph5_aMC@NLO/MadEvent *
> #* *
> #* * * *
> #* * * * * *
> #* * * * * 5 * * * * *
> #* * * * * *
> #* * * *
> #* *
> #* *
> #* VERSION 5.2.1.2 *
> #* *
> #* The MadGraph5_aMC@NLO Development Team - Find us at *
> #* https://server06.fynu.ucl.ac.be/projects/madgraph *
> #* *
> #************************************************************
> #* *
> #* Command File for MadEvent *
> #* *
> #* run as ./bin/madevent.py filename *
> #* *
> #************************************************************
> generate_events -f run6 --nb_core=1
> Traceback (most recent call last):
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 879, in onecmd
> return self.onecmd_orig(line, **opt)
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 872, in onecmd_orig
> return func(arg, **opt)
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 2107, in do_generate_events
> postcmd=False)
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 919, in exec_cmd
> stop = Cmd.onecmd_orig(current_interface, line, **opt)
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 872, in onecmd_orig
> return func(arg, **opt)
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 2476, in do_survey
> self.configure_directory()
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 3570, in configure_directory
> self.compile(arg=[name], cwd=os.path.join(self.me_dir, 'Source'))
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 987, in compile
> return misc.compile(nb_core=self.options['nb_core'], *args, **opts)
> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/misc.py", line 293, in compile
> raise MadGraph5Error, error_text
> MadGraph5Error: A compilation Error occurs when trying to compile /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Source.
> The compilation fails with the following output message:
> gfortran -O -w -fbounds-check -ffixed-line-length-132 -c -o gen_ximprove.o gen_ximprove.f
> gfortran -O -w -fbounds-check -ffixed-line-length-132 -o ../bin/internal/gen_ximprove gen_ximprove.o ranmar.o rw_routines.o open_file.o
> ranmar.o: In function `rmarin_':
> ranmar.f:(.text+0xb1): undefined reference to `_gfortran_copy_string'
> ranmar.f:(.text+0x136): undefined reference to `_gfortran_copy_string'
> ranmar.o: In function `get_base_':
> ranmar.f:(.text+0x9bf): undefined reference to `_gfortran_copy_string'
> ranmar.f:(.text+0xa8b): undefined reference to `_gfortran_copy_string'
> ranmar.f:(.text+0xac9): undefined reference to `_gfortran_copy_string'
> ranmar.o:ranmar.f:(.text+0xb8d): more undefined references to `_gfortran_copy_string' follow
> rw_routines.o: In function `load_para_':
> rw_routines.f:(.text+0xe22): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0xe6f): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0xe77): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0xe9a): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0xeca): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1050): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1078): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x10b2): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x10cc): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x1114): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x111e): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1146): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1180): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1423): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x144b): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1485): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x149c): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x14e6): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x14f0): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1518): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1552): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x16de): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1706): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1740): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x175b): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x17a2): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x17ab): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x17d3): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x180d): undefined reference to `_gfortran_copy_string'
> rw_routines.o: In function `load_gridpack_para_':
> rw_routines.f:(.text+0x1899): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x18b0): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1b18): undefined reference to `_gfortran_copy_string'
> rw_routines.o:rw_routines.f:(.text+0x1b3c): more undefined references to `_gfortran_copy_string' follow
> rw_routines.o: In function `load_gridpack_para_':
> rw_routines.f:(.text+0x1b8f): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x1bd7): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1be1): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1c05): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1c3b): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1dbf): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1de3): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1e19): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1e36): undefined reference to `_gfortran_internal_malloc'
> rw_routines.f:(.text+0x1e7d): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1e87): undefined reference to `_gfortran_internal_free'
> rw_routines.f:(.text+0x1eab): undefined reference to `_gfortran_copy_string'
> rw_routines.f:(.text+0x1ee1): undefined reference to `_gfortran_copy_string'
> open_file.o: In function `open_file_':
> open_file.f:(.text+0x4b): undefined reference to `_gfortran_copy_string'
> open_file.f:(.text+0x193): undefined reference to `_gfortran_internal_malloc'
> open_file.f:(.text+0x1cc): undefined reference to `_gfortran_copy_string'
> open_file.f:(.text+0x1d4): undefined reference to `_gfortran_internal_free'
> open_file.f:(.text+0x282): undefined reference to `_gfortran_copy_string'
> collect2: ld returned 1 exit status
> make: *** [../bin/internal/gen_ximprove] Error 1
>
> Please try to fix this compilations issue and retry.
> Help might be found at https://answers.launchpad.net/madgraph5.
> If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
> Run Options
> -----------
> stdout_level : None
>
> MadEvent Options
> ----------------
> automatic_html_opening : False (user set)
> cluster_temp_path : None
> cluster_queue : madgraph
> cluster_memory : None
> cluster_time : None
> nb_core : 1 (user set)
> run_mode : 2
>
> Configuration Options
> ---------------------
> text_editor : None
> cluster_status_update : (600, 30)
> pythia8_path : None (user set)
> hwpp_path : None (user set)
> pythia-pgs_path : /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/pythia-pgs (user set)
> td_path : None (user set)
> delphes_path : None (user set)
> thepeg_path : None (user set)
> cluster_type : condor
> exrootanalysis_path : /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/ExRootAnalysis (user set)
> eps_viewer : None
> web_browser : None
> syscalc_path : None (user set)
> madanalysis_path : None (user set)
> lhapdf : lhapdf-config
> hepmc_path : None (user set)
> cluster_retry_wait : 300
> fortran_compiler : None
> auto_update : 7 (user set)
> cluster_nb_retry : 1
> timeout : 60
> cpp_compiler : None
>
>
>
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Hi Olivier,

Thanks for your suggestion that some of the nodes may not have the gfortran library necessary to compile the code. However, I don't think this is the case. I was running all the jobs with this method just fine about a week ago but using an older version of PGS. After updating the pythia-pgs version (so that I could use the antikt jet-finding algorithm), these errors started to occur. I'm also not sure how this could be the source of the problem though as the runs that fail fail right away, long before pythia or pgs start to run. Is it possible that the new version of pythia-pgs requires different gfortran files and that these would be required at the beginning of the parton level compilation?

1) After having tried for almost 6 months now to get mad graph to submit jobs to my local cluster through this method I have given up. Every time I tried, it never actually submitted to my cluster, i.e,. the job never showed up on the cluster job list (using qstat for a PBS cluster). It just ran on my local account, forcing me to wait until that run was done before generating anymore events. Because of this, I've moved on to the current method. Using this method does run multiple instances of MG on different nodes at the same time but, as I've described, still needs some tweaking to get it working completely right with the new version of PGS and the antikt algorithm.

2) I do modify the seed for each run by modifying the run_card.dat before submitting the corresponding job.

Cheers,
Peter

On 2014-07-06, at 5:07 PM, Olivier Mattelaer <email address hidden> wrote:

> Your question #251271 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/251271
>
> Status: Open => Answered
>
> Olivier Mattelaer proposed the following answer:
> Hi Peter,
>
> According to your log this is a compilation problem. My guess is that
> some of the node miss some of the gfortran library that you need to
> compile the code.
>
> This being said, I have a couple of comment/concern:
> 1) You are not using the recommended way to submit jobs on a cluster.
> The recommend way is to change to tell MadGraph to do the cluster submission:
> for this you can edit the file input/mg5_configuration.txt
> and set
> run_mode on 1
> and
> cluster_type on pbs
>
> 2) Did you specify the seed for your runs? if not, it is likely that you generate each times the same sample of events (depend on how exactly how you generate your sample of events).
> So please check that point before using your multiple samples.
>
> Cheers,
>
> Olivier
>
>
>
> On Jul 6, 2014, at 6:41 PM, Peter Winslow <email address hidden> wrote:
>
>> New question #251271 on MadGraph5_aMC@NLO:
>> https://answers.launchpad.net/mg5amcnlo/+question/251271
>>
>> Hi,
>>
>> I am running MG5_aMC_v2_1_1 and trying to generate 1,000,000 events for a simple process on my local cluster. I'm finding a very strange error. I submit 20 separate jobs (instances of MG which run this particular process) to our local PBS cluster. Each job is identical to the other, I simply submit the same 50,000 event generation run 20 separate times to eventually get the desired 1,000,000 events in the end. Some number of these jobs will run (at least they are continuing to run so far) but others are quickly stopped in error. I've attached the 1) output.log, 2) error.log, and 3) debug.log files below. Do you happen to have any idea what could be causing such errors in only some of the runs?
>>
>> Peter
>>
>> ** output.log file **
>>
>> seed: 6
>> HOSTNAME=titan34
>> PBS_VERSION=TORQUE-2.3.7
>> SHELL=/bin/bash
>> HISTSIZE=1000
>> PBS_JOBNAME=job6
>> PBS_ENVIRONMENT=PBS_BATCH
>> QTDIR=/usr/lib64/qt-3.3
>> QTINC=/usr/lib64/qt-3.3/include
>> PBS_O_WORKDIR=/scratch/pwinslow/MGjobs/jjWWBG/job6
>> USER=pwinslow
>> PBS_TASKNUM=1
>> PBS_O_HOME=/home/pwinslow
>> PBS_MOMPORT=15003
>> VIRTUAL_ENV=/scratch/pwinslow/CosmoTransitions/User
>> PBS_O_QUEUE=feed
>> PATH=/scratch/pwinslow/CosmoTransitions/User/bin:/home/pwinslow/.local/bin:/home/pwinslow/ActivePython-2.7/bin:/home/pwinslow/bin:/usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/pwinslow/bin
>> PBS_O_LOGNAME=pwinslow
>> MAIL=/var/spool/mail/pwinslow
>> PBS_O_LANG=en_US.UTF-8
>> PBS_JOBCOOKIE=DA569D966BD5993EEFB5F6ACFE571D52
>> PWD=/home/pwinslow
>> LANG=en_US.UTF-8
>> PBS_NODENUM=0
>> MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles
>> LOADEDMODULES=
>> PBS_O_SHELL=/bin/bash
>> PBS_SERVER=titan.physics.umass.edu
>> PBS_JOBID=920893.titan.physics.umass.edu
>> ENVIRONMENT=BATCH
>> HISTCONTROL=ignoredups
>> SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
>> HOME=/home/pwinslow
>> SHLVL=2
>> PBS_O_HOST=titan.physics.umass.edu
>> PBS_VNODENUM=0
>> LOGNAME=pwinslow
>> CVS_RSH=ssh
>> QTLIB=/usr/lib64/qt-3.3/lib
>> PBS_QUEUE=express
>> MODULESHOME=/usr/share/Modules
>> PBS_O_MAIL=/var/spool/mail/pwinslow
>> LESSOPEN=|/usr/bin/lesspipe.sh %s
>> PBS_NODEFILE=/var/spool/torque/aux//920893.titan.physics.umass.edu
>> G_BROKEN_FILENAMES=1
>> PBS_O_PATH=/scratch/pwinslow/CosmoTransitions/User/bin:/home/pwinslow/.local/bin:/home/pwinslow/ActivePython-2.7/bin:/home/pwinslow/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/pwinslow/bin
>> module=() { eval `/usr/bin/modulecmd bash $*`
>> }
>> _=/usr/bin/printenv
>> ************************************************************
>> * *
>> * W E L C O M E to *
>> * M A D G R A P H 5 _ a M C @ N L O *
>> * M A D E V E N T *
>> * *
>> * * * *
>> * * * * * *
>> * * * * * 5 * * * * *
>> * * * * * *
>> * * * *
>> * *
>> * VERSION 5.2.1.2 *
>> * *
>> * The MadGraph5_aMC@NLO Development Team - Find us at *
>> * https://server06.fynu.ucl.ac.be/projects/madgraph *
>> * *
>> * Type 'help' for in-line help. *
>> * *
>> ************************************************************
>> INFO: load configuration from /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Cards/me5_configuration.txt
>> INFO: load configuration from /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/input/mg5_configuration.txt
>> INFO: load configuration from /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Cards/me5_configuration.txt
>> Using default text editor "vi". Set another one in ./input/mg5_configuration.txt
>> Using default eps viewer "evince". Set another one in ./input/mg5_configuration.txt
>> Using default web browser "firefox". Set another one in ./input/mg5_configuration.txt
>> No valid Delphes path found
>> generate_events -f run6 --nb_core=1
>> Generating 50000 events with run name run6
>> survey run6
>> INFO: compile directory
>> Since icckw>0, We change the value of 'drjl' to 0
>> quit
>> INFO:
>>
>> INFO:
>>
>> launch in debug mode
>>
>>
>>
>> ** error.log file **
>>
>> Error detected in "generate_events -f run6 --nb_core=1"
>> write debug file /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/run6_tag_1_debug.log
>> If you need help with this issue please contact us on https://answers.launchpad.net/madgraph5
>> MadGraph5Error : A compilation Error occurs when trying to compile /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Source.
>> The compilation fails with the following output message:
>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -c -o gen_ximprove.o gen_ximprove.f
>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -o ../bin/internal/gen_ximprove gen_ximprove.o ranmar.o rw_routines.o open_file.o
>> ranmar.o: In function `rmarin_':
>> ranmar.f:(.text+0xb1): undefined reference to `_gfortran_copy_string'
>> ranmar.f:(.text+0x136): undefined reference to `_gfortran_copy_string'
>> ranmar.o: In function `get_base_':
>> ranmar.f:(.text+0x9bf): undefined reference to `_gfortran_copy_string'
>> ranmar.f:(.text+0xa8b): undefined reference to `_gfortran_copy_string'
>> ranmar.f:(.text+0xac9): undefined reference to `_gfortran_copy_string'
>> ranmar.o:ranmar.f:(.text+0xb8d): more undefined references to `_gfortran_copy_string' follow
>> rw_routines.o: In function `load_para_':
>> rw_routines.f:(.text+0xe22): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0xe6f): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0xe77): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0xe9a): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0xeca): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1050): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1078): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x10b2): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x10cc): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x1114): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x111e): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1146): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1180): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1423): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x144b): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1485): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x149c): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x14e6): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x14f0): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1518): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1552): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x16de): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1706): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1740): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x175b): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x17a2): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x17ab): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x17d3): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x180d): undefined reference to `_gfortran_copy_string'
>> rw_routines.o: In function `load_gridpack_para_':
>> rw_routines.f:(.text+0x1899): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x18b0): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1b18): undefined reference to `_gfortran_copy_string'
>> rw_routines.o:rw_routines.f:(.text+0x1b3c): more undefined references to `_gfortran_copy_string' follow
>> rw_routines.o: In function `load_gridpack_para_':
>> rw_routines.f:(.text+0x1b8f): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x1bd7): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1be1): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1c05): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1c3b): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1dbf): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1de3): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1e19): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1e36): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x1e7d): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1e87): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1eab): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1ee1): undefined reference to `_gfortran_copy_string'
>> open_file.o: In function `open_file_':
>> open_file.f:(.text+0x4b): undefined reference to `_gfortran_copy_string'
>> open_file.f:(.text+0x193): undefined reference to `_gfortran_internal_malloc'
>> open_file.f:(.text+0x1cc): undefined reference to `_gfortran_copy_string'
>> open_file.f:(.text+0x1d4): undefined reference to `_gfortran_internal_free'
>> open_file.f:(.text+0x282): undefined reference to `_gfortran_copy_string'
>> collect2: ld returned 1 exit status
>> make: *** [../bin/internal/gen_ximprove] Error 1
>>
>> Please try to fix this compilations issue and retry.
>> Help might be found at https://answers.launchpad.net/madgraph5.
>> If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
>>
>>
>>
>>
>> ** debug.log file **
>>
>> #************************************************************
>> #* MadGraph5_aMC@NLO/MadEvent *
>> #* *
>> #* * * *
>> #* * * * * *
>> #* * * * * 5 * * * * *
>> #* * * * * *
>> #* * * *
>> #* *
>> #* *
>> #* VERSION 5.2.1.2 *
>> #* *
>> #* The MadGraph5_aMC@NLO Development Team - Find us at *
>> #* https://server06.fynu.ucl.ac.be/projects/madgraph *
>> #* *
>> #************************************************************
>> #* *
>> #* Command File for MadEvent *
>> #* *
>> #* run as ./bin/madevent.py filename *
>> #* *
>> #************************************************************
>> generate_events -f run6 --nb_core=1
>> Traceback (most recent call last):
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 879, in onecmd
>> return self.onecmd_orig(line, **opt)
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 872, in onecmd_orig
>> return func(arg, **opt)
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 2107, in do_generate_events
>> postcmd=False)
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 919, in exec_cmd
>> stop = Cmd.onecmd_orig(current_interface, line, **opt)
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 872, in onecmd_orig
>> return func(arg, **opt)
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 2476, in do_survey
>> self.configure_directory()
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 3570, in configure_directory
>> self.compile(arg=[name], cwd=os.path.join(self.me_dir, 'Source'))
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 987, in compile
>> return misc.compile(nb_core=self.options['nb_core'], *args, **opts)
>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/misc.py", line 293, in compile
>> raise MadGraph5Error, error_text
>> MadGraph5Error: A compilation Error occurs when trying to compile /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Source.
>> The compilation fails with the following output message:
>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -c -o gen_ximprove.o gen_ximprove.f
>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -o ../bin/internal/gen_ximprove gen_ximprove.o ranmar.o rw_routines.o open_file.o
>> ranmar.o: In function `rmarin_':
>> ranmar.f:(.text+0xb1): undefined reference to `_gfortran_copy_string'
>> ranmar.f:(.text+0x136): undefined reference to `_gfortran_copy_string'
>> ranmar.o: In function `get_base_':
>> ranmar.f:(.text+0x9bf): undefined reference to `_gfortran_copy_string'
>> ranmar.f:(.text+0xa8b): undefined reference to `_gfortran_copy_string'
>> ranmar.f:(.text+0xac9): undefined reference to `_gfortran_copy_string'
>> ranmar.o:ranmar.f:(.text+0xb8d): more undefined references to `_gfortran_copy_string' follow
>> rw_routines.o: In function `load_para_':
>> rw_routines.f:(.text+0xe22): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0xe6f): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0xe77): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0xe9a): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0xeca): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1050): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1078): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x10b2): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x10cc): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x1114): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x111e): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1146): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1180): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1423): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x144b): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1485): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x149c): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x14e6): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x14f0): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1518): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1552): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x16de): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1706): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1740): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x175b): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x17a2): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x17ab): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x17d3): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x180d): undefined reference to `_gfortran_copy_string'
>> rw_routines.o: In function `load_gridpack_para_':
>> rw_routines.f:(.text+0x1899): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x18b0): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1b18): undefined reference to `_gfortran_copy_string'
>> rw_routines.o:rw_routines.f:(.text+0x1b3c): more undefined references to `_gfortran_copy_string' follow
>> rw_routines.o: In function `load_gridpack_para_':
>> rw_routines.f:(.text+0x1b8f): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x1bd7): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1be1): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1c05): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1c3b): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1dbf): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1de3): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1e19): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1e36): undefined reference to `_gfortran_internal_malloc'
>> rw_routines.f:(.text+0x1e7d): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1e87): undefined reference to `_gfortran_internal_free'
>> rw_routines.f:(.text+0x1eab): undefined reference to `_gfortran_copy_string'
>> rw_routines.f:(.text+0x1ee1): undefined reference to `_gfortran_copy_string'
>> open_file.o: In function `open_file_':
>> open_file.f:(.text+0x4b): undefined reference to `_gfortran_copy_string'
>> open_file.f:(.text+0x193): undefined reference to `_gfortran_internal_malloc'
>> open_file.f:(.text+0x1cc): undefined reference to `_gfortran_copy_string'
>> open_file.f:(.text+0x1d4): undefined reference to `_gfortran_internal_free'
>> open_file.f:(.text+0x282): undefined reference to `_gfortran_copy_string'
>> collect2: ld returned 1 exit status
>> make: *** [../bin/internal/gen_ximprove] Error 1
>>
>> Please try to fix this compilations issue and retry.
>> Help might be found at https://answers.launchpad.net/madgraph5.
>> If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
>> Run Options
>> -----------
>> stdout_level : None
>>
>> MadEvent Options
>> ----------------
>> automatic_html_opening : False (user set)
>> cluster_temp_path : None
>> cluster_queue : madgraph
>> cluster_memory : None
>> cluster_time : None
>> nb_core : 1 (user set)
>> run_mode : 2
>>
>> Configuration Options
>> ---------------------
>> text_editor : None
>> cluster_status_update : (600, 30)
>> pythia8_path : None (user set)
>> hwpp_path : None (user set)
>> pythia-pgs_path : /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/pythia-pgs (user set)
>> td_path : None (user set)
>> delphes_path : None (user set)
>> thepeg_path : None (user set)
>> cluster_type : condor
>> exrootanalysis_path : /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/ExRootAnalysis (user set)
>> eps_viewer : None
>> web_browser : None
>> syscalc_path : None (user set)
>> madanalysis_path : None (user set)
>> lhapdf : lhapdf-config
>> hepmc_path : None (user set)
>> cluster_retry_wait : 300
>> fortran_compiler : None
>> auto_update : 7 (user set)
>> cluster_nb_retry : 1
>> timeout : 60
>> cpp_compiler : None
>>
>>
>>
>>
>> --
>> You received this question notification because you are an answer
>> contact for MadGraph5_aMC@NLO.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/mg5amcnlo/+question/251271/+confirm?answer_id=0
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/mg5amcnlo/+question/251271
>
> You received this question notification because you asked the question.

Hi Peter,

I don’t think that this is linked to pythia-pgs, as you said the compilation error is not linked to that package.
One possibility is that part of your code is compile with g77 and part of it is compile with gfortran.

This can happen if you use a old v4 model, where the compiler is often hardcoded in the makefile of the model
(and hardcoded to f77).
If some of your node have f77 installed and some other where f77 is just a symlink to gfortran it could explain your difference depending of the node.

Finally I’m not aware of any change in PGS since more than 5 years. So I’m not sure that any version of PGS use anti-kt. (I doubt about that).
So please check carefully this point. I would personally recommend to use Delphes which use fast jet and the anti-it algorithm.

Cheers,

Olivier

On Jul 7, 2014, at 2:52 AM, Peter Winslow <email address hidden> wrote:

> Question #251271 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/251271
>
> Status: Answered => Open
>
> Peter Winslow is still having a problem:
> Hi Olivier,
>
> Thanks for your suggestion that some of the nodes may not have the
> gfortran library necessary to compile the code. However, I don't think
> this is the case. I was running all the jobs with this method just fine
> about a week ago but using an older version of PGS. After updating the
> pythia-pgs version (so that I could use the antikt jet-finding
> algorithm), these errors started to occur. I'm also not sure how this
> could be the source of the problem though as the runs that fail fail
> right away, long before pythia or pgs start to run. Is it possible that
> the new version of pythia-pgs requires different gfortran files and that
> these would be required at the beginning of the parton level
> compilation?
>
> 1) After having tried for almost 6 months now to get mad graph to submit
> jobs to my local cluster through this method I have given up. Every time
> I tried, it never actually submitted to my cluster, i.e,. the job never
> showed up on the cluster job list (using qstat for a PBS cluster). It
> just ran on my local account, forcing me to wait until that run was done
> before generating anymore events. Because of this, I've moved on to the
> current method. Using this method does run multiple instances of MG on
> different nodes at the same time but, as I've described, still needs
> some tweaking to get it working completely right with the new version of
> PGS and the antikt algorithm.
>
> 2) I do modify the seed for each run by modifying the run_card.dat
> before submitting the corresponding job.
>
> Cheers,
> Peter
>
> On 2014-07-06, at 5:07 PM, Olivier Mattelaer
> <email address hidden> wrote:
>
>> Your question #251271 on MadGraph5_aMC@NLO changed:
>> https://answers.launchpad.net/mg5amcnlo/+question/251271
>>
>> Status: Open => Answered
>>
>> Olivier Mattelaer proposed the following answer:
>> Hi Peter,
>>
>> According to your log this is a compilation problem. My guess is that
>> some of the node miss some of the gfortran library that you need to
>> compile the code.
>>
>> This being said, I have a couple of comment/concern:
>> 1) You are not using the recommended way to submit jobs on a cluster.
>> The recommend way is to change to tell MadGraph to do the cluster submission:
>> for this you can edit the file input/mg5_configuration.txt
>> and set
>> run_mode on 1
>> and
>> cluster_type on pbs
>>
>> 2) Did you specify the seed for your runs? if not, it is likely that you generate each times the same sample of events (depend on how exactly how you generate your sample of events).
>> So please check that point before using your multiple samples.
>>
>> Cheers,
>>
>> Olivier
>>
>>
>>
>> On Jul 6, 2014, at 6:41 PM, Peter Winslow <email address hidden> wrote:
>>
>>> New question #251271 on MadGraph5_aMC@NLO:
>>> https://answers.launchpad.net/mg5amcnlo/+question/251271
>>>
>>> Hi,
>>>
>>> I am running MG5_aMC_v2_1_1 and trying to generate 1,000,000 events for a simple process on my local cluster. I'm finding a very strange error. I submit 20 separate jobs (instances of MG which run this particular process) to our local PBS cluster. Each job is identical to the other, I simply submit the same 50,000 event generation run 20 separate times to eventually get the desired 1,000,000 events in the end. Some number of these jobs will run (at least they are continuing to run so far) but others are quickly stopped in error. I've attached the 1) output.log, 2) error.log, and 3) debug.log files below. Do you happen to have any idea what could be causing such errors in only some of the runs?
>>>
>>> Peter
>>>
>>> ** output.log file **
>>>
>>> seed: 6
>>> HOSTNAME=titan34
>>> PBS_VERSION=TORQUE-2.3.7
>>> SHELL=/bin/bash
>>> HISTSIZE=1000
>>> PBS_JOBNAME=job6
>>> PBS_ENVIRONMENT=PBS_BATCH
>>> QTDIR=/usr/lib64/qt-3.3
>>> QTINC=/usr/lib64/qt-3.3/include
>>> PBS_O_WORKDIR=/scratch/pwinslow/MGjobs/jjWWBG/job6
>>> USER=pwinslow
>>> PBS_TASKNUM=1
>>> PBS_O_HOME=/home/pwinslow
>>> PBS_MOMPORT=15003
>>> VIRTUAL_ENV=/scratch/pwinslow/CosmoTransitions/User
>>> PBS_O_QUEUE=feed
>>> PATH=/scratch/pwinslow/CosmoTransitions/User/bin:/home/pwinslow/.local/bin:/home/pwinslow/ActivePython-2.7/bin:/home/pwinslow/bin:/usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/pwinslow/bin
>>> PBS_O_LOGNAME=pwinslow
>>> MAIL=/var/spool/mail/pwinslow
>>> PBS_O_LANG=en_US.UTF-8
>>> PBS_JOBCOOKIE=DA569D966BD5993EEFB5F6ACFE571D52
>>> PWD=/home/pwinslow
>>> LANG=en_US.UTF-8
>>> PBS_NODENUM=0
>>> MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles
>>> LOADEDMODULES=
>>> PBS_O_SHELL=/bin/bash
>>> PBS_SERVER=titan.physics.umass.edu
>>> PBS_JOBID=920893.titan.physics.umass.edu
>>> ENVIRONMENT=BATCH
>>> HISTCONTROL=ignoredups
>>> SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
>>> HOME=/home/pwinslow
>>> SHLVL=2
>>> PBS_O_HOST=titan.physics.umass.edu
>>> PBS_VNODENUM=0
>>> LOGNAME=pwinslow
>>> CVS_RSH=ssh
>>> QTLIB=/usr/lib64/qt-3.3/lib
>>> PBS_QUEUE=express
>>> MODULESHOME=/usr/share/Modules
>>> PBS_O_MAIL=/var/spool/mail/pwinslow
>>> LESSOPEN=|/usr/bin/lesspipe.sh %s
>>> PBS_NODEFILE=/var/spool/torque/aux//920893.titan.physics.umass.edu
>>> G_BROKEN_FILENAMES=1
>>> PBS_O_PATH=/scratch/pwinslow/CosmoTransitions/User/bin:/home/pwinslow/.local/bin:/home/pwinslow/ActivePython-2.7/bin:/home/pwinslow/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/pwinslow/bin
>>> module=() { eval `/usr/bin/modulecmd bash $*`
>>> }
>>> _=/usr/bin/printenv
>>> ************************************************************
>>> * *
>>> * W E L C O M E to *
>>> * M A D G R A P H 5 _ a M C @ N L O *
>>> * M A D E V E N T *
>>> * *
>>> * * * *
>>> * * * * * *
>>> * * * * * 5 * * * * *
>>> * * * * * *
>>> * * * *
>>> * *
>>> * VERSION 5.2.1.2 *
>>> * *
>>> * The MadGraph5_aMC@NLO Development Team - Find us at *
>>> * https://server06.fynu.ucl.ac.be/projects/madgraph *
>>> * *
>>> * Type 'help' for in-line help. *
>>> * *
>>> ************************************************************
>>> INFO: load configuration from /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Cards/me5_configuration.txt
>>> INFO: load configuration from /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/input/mg5_configuration.txt
>>> INFO: load configuration from /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Cards/me5_configuration.txt
>>> Using default text editor "vi". Set another one in ./input/mg5_configuration.txt
>>> Using default eps viewer "evince". Set another one in ./input/mg5_configuration.txt
>>> Using default web browser "firefox". Set another one in ./input/mg5_configuration.txt
>>> No valid Delphes path found
>>> generate_events -f run6 --nb_core=1
>>> Generating 50000 events with run name run6
>>> survey run6
>>> INFO: compile directory
>>> Since icckw>0, We change the value of 'drjl' to 0
>>> quit
>>> INFO:
>>>
>>> INFO:
>>>
>>> launch in debug mode
>>>
>>>
>>>
>>> ** error.log file **
>>>
>>> Error detected in "generate_events -f run6 --nb_core=1"
>>> write debug file /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/run6_tag_1_debug.log
>>> If you need help with this issue please contact us on https://answers.launchpad.net/madgraph5
>>> MadGraph5Error : A compilation Error occurs when trying to compile /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Source.
>>> The compilation fails with the following output message:
>>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -c -o gen_ximprove.o gen_ximprove.f
>>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -o ../bin/internal/gen_ximprove gen_ximprove.o ranmar.o rw_routines.o open_file.o
>>> ranmar.o: In function `rmarin_':
>>> ranmar.f:(.text+0xb1): undefined reference to `_gfortran_copy_string'
>>> ranmar.f:(.text+0x136): undefined reference to `_gfortran_copy_string'
>>> ranmar.o: In function `get_base_':
>>> ranmar.f:(.text+0x9bf): undefined reference to `_gfortran_copy_string'
>>> ranmar.f:(.text+0xa8b): undefined reference to `_gfortran_copy_string'
>>> ranmar.f:(.text+0xac9): undefined reference to `_gfortran_copy_string'
>>> ranmar.o:ranmar.f:(.text+0xb8d): more undefined references to `_gfortran_copy_string' follow
>>> rw_routines.o: In function `load_para_':
>>> rw_routines.f:(.text+0xe22): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0xe6f): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0xe77): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0xe9a): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0xeca): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1050): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1078): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x10b2): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x10cc): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x1114): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x111e): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1146): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1180): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1423): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x144b): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1485): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x149c): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x14e6): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x14f0): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1518): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1552): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x16de): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1706): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1740): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x175b): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x17a2): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x17ab): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x17d3): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x180d): undefined reference to `_gfortran_copy_string'
>>> rw_routines.o: In function `load_gridpack_para_':
>>> rw_routines.f:(.text+0x1899): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x18b0): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1b18): undefined reference to `_gfortran_copy_string'
>>> rw_routines.o:rw_routines.f:(.text+0x1b3c): more undefined references to `_gfortran_copy_string' follow
>>> rw_routines.o: In function `load_gridpack_para_':
>>> rw_routines.f:(.text+0x1b8f): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x1bd7): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1be1): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1c05): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1c3b): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1dbf): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1de3): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1e19): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1e36): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x1e7d): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1e87): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1eab): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1ee1): undefined reference to `_gfortran_copy_string'
>>> open_file.o: In function `open_file_':
>>> open_file.f:(.text+0x4b): undefined reference to `_gfortran_copy_string'
>>> open_file.f:(.text+0x193): undefined reference to `_gfortran_internal_malloc'
>>> open_file.f:(.text+0x1cc): undefined reference to `_gfortran_copy_string'
>>> open_file.f:(.text+0x1d4): undefined reference to `_gfortran_internal_free'
>>> open_file.f:(.text+0x282): undefined reference to `_gfortran_copy_string'
>>> collect2: ld returned 1 exit status
>>> make: *** [../bin/internal/gen_ximprove] Error 1
>>>
>>> Please try to fix this compilations issue and retry.
>>> Help might be found at https://answers.launchpad.net/madgraph5.
>>> If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
>>>
>>>
>>>
>>>
>>> ** debug.log file **
>>>
>>> #************************************************************
>>> #* MadGraph5_aMC@NLO/MadEvent *
>>> #* *
>>> #* * * *
>>> #* * * * * *
>>> #* * * * * 5 * * * * *
>>> #* * * * * *
>>> #* * * *
>>> #* *
>>> #* *
>>> #* VERSION 5.2.1.2 *
>>> #* *
>>> #* The MadGraph5_aMC@NLO Development Team - Find us at *
>>> #* https://server06.fynu.ucl.ac.be/projects/madgraph *
>>> #* *
>>> #************************************************************
>>> #* *
>>> #* Command File for MadEvent *
>>> #* *
>>> #* run as ./bin/madevent.py filename *
>>> #* *
>>> #************************************************************
>>> generate_events -f run6 --nb_core=1
>>> Traceback (most recent call last):
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 879, in onecmd
>>> return self.onecmd_orig(line, **opt)
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 872, in onecmd_orig
>>> return func(arg, **opt)
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 2107, in do_generate_events
>>> postcmd=False)
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 919, in exec_cmd
>>> stop = Cmd.onecmd_orig(current_interface, line, **opt)
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 872, in onecmd_orig
>>> return func(arg, **opt)
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 2476, in do_survey
>>> self.configure_directory()
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/madevent_interface.py", line 3570, in configure_directory
>>> self.compile(arg=[name], cwd=os.path.join(self.me_dir, 'Source'))
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/extended_cmd.py", line 987, in compile
>>> return misc.compile(nb_core=self.options['nb_core'], *args, **opts)
>>> File "/nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/bin/internal/misc.py", line 293, in compile
>>> raise MadGraph5Error, error_text
>>> MadGraph5Error: A compilation Error occurs when trying to compile /nfs/scratch/pwinslow/MGjobs/jjWWBG/job6/MG5_aMC_v2_1_1/jjWWClusterBG/Source.
>>> The compilation fails with the following output message:
>>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -c -o gen_ximprove.o gen_ximprove.f
>>> gfortran -O -w -fbounds-check -ffixed-line-length-132 -o ../bin/internal/gen_ximprove gen_ximprove.o ranmar.o rw_routines.o open_file.o
>>> ranmar.o: In function `rmarin_':
>>> ranmar.f:(.text+0xb1): undefined reference to `_gfortran_copy_string'
>>> ranmar.f:(.text+0x136): undefined reference to `_gfortran_copy_string'
>>> ranmar.o: In function `get_base_':
>>> ranmar.f:(.text+0x9bf): undefined reference to `_gfortran_copy_string'
>>> ranmar.f:(.text+0xa8b): undefined reference to `_gfortran_copy_string'
>>> ranmar.f:(.text+0xac9): undefined reference to `_gfortran_copy_string'
>>> ranmar.o:ranmar.f:(.text+0xb8d): more undefined references to `_gfortran_copy_string' follow
>>> rw_routines.o: In function `load_para_':
>>> rw_routines.f:(.text+0xe22): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0xe6f): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0xe77): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0xe9a): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0xeca): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1050): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1078): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x10b2): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x10cc): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x1114): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x111e): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1146): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1180): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1423): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x144b): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1485): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x149c): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x14e6): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x14f0): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1518): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1552): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x16de): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1706): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1740): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x175b): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x17a2): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x17ab): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x17d3): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x180d): undefined reference to `_gfortran_copy_string'
>>> rw_routines.o: In function `load_gridpack_para_':
>>> rw_routines.f:(.text+0x1899): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x18b0): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1b18): undefined reference to `_gfortran_copy_string'
>>> rw_routines.o:rw_routines.f:(.text+0x1b3c): more undefined references to `_gfortran_copy_string' follow
>>> rw_routines.o: In function `load_gridpack_para_':
>>> rw_routines.f:(.text+0x1b8f): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x1bd7): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1be1): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1c05): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1c3b): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1dbf): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1de3): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1e19): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1e36): undefined reference to `_gfortran_internal_malloc'
>>> rw_routines.f:(.text+0x1e7d): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1e87): undefined reference to `_gfortran_internal_free'
>>> rw_routines.f:(.text+0x1eab): undefined reference to `_gfortran_copy_string'
>>> rw_routines.f:(.text+0x1ee1): undefined reference to `_gfortran_copy_string'
>>> open_file.o: In function `open_file_':
>>> open_file.f:(.text+0x4b): undefined reference to `_gfortran_copy_string'
>>> open_file.f:(.text+0x193): undefined reference to `_gfortran_internal_malloc'
>>> open_file.f:(.text+0x1cc): undefined reference to `_gfortran_copy_string'
>>> open_file.f:(.text+0x1d4): undefined reference to `_gfortran_internal_free'
>>> open_file.f:(.text+0x282): undefined reference to `_gfortran_copy_string'
>>> collect2: ld returned 1 exit status
>>> make: *** [../bin/internal/gen_ximprove] Error 1
>>>
>>> Please try to fix this compilations issue and retry.
>>> Help might be found at https://answers.launchpad.net/madgraph5.
>>> If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
>>> Run Options
>>> -----------
>>> stdout_level : None
>>>
>>> MadEvent Options
>>> ----------------
>>> automatic_html_opening : False (user set)
>>> cluster_temp_path : None
>>> cluster_queue : madgraph
>>> cluster_memory : None
>>> cluster_time : None
>>> nb_core : 1 (user set)
>>> run_mode : 2
>>>
>>> Configuration Options
>>> ---------------------
>>> text_editor : None
>>> cluster_status_update : (600, 30)
>>> pythia8_path : None (user set)
>>> hwpp_path : None (user set)
>>> pythia-pgs_path : /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/pythia-pgs (user set)
>>> td_path : None (user set)
>>> delphes_path : None (user set)
>>> thepeg_path : None (user set)
>>> cluster_type : condor
>>> exrootanalysis_path : /nfs/scratch/pwinslow/MG5_aMC_v2_1_1/ExRootAnalysis (user set)
>>> eps_viewer : None
>>> web_browser : None
>>> syscalc_path : None (user set)
>>> madanalysis_path : None (user set)
>>> lhapdf : lhapdf-config
>>> hepmc_path : None (user set)
>>> cluster_retry_wait : 300
>>> fortran_compiler : None
>>> auto_update : 7 (user set)
>>> cluster_nb_retry : 1
>>> timeout : 60
>>> cpp_compiler : None
>>>
>>>
>>>
>>>
>>> --
>>> You received this question notification because you are an answer
>>> contact for MadGraph5_aMC@NLO.
>>
>> --
>> If this answers your question, please go to the following page to let us
>> know that it is solved:
>> https://answers.launchpad.net/mg5amcnlo/+question/251271/+confirm?answer_id=0
>>
>> If you still need help, you can reply to this email or go to the
>> following page to enter your feedback:
>> https://answers.launchpad.net/mg5amcnlo/+question/251271
>>
>> You received this question notification because you asked the question.
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.