MadGraph5_aMC@NLO

MCatNLO compilation fails for pythia8

Asked by Prudhvi Bhattiprolu

Dear Madgraph Team,

I am trying to generate events at NLO (in QCD) using Madgraph v3.4.2 (with pythia8 installed through madgraph interface) for the command

        generate e+ e- > t t~ [QCD]

and I was able to successfully generate the .lhe (parton-level events) file but the pythia8 showering seems to fail due to a segmentation fault during the compilation of MCatNLO with the following output:

        /home/prudhvib/HEPTools/testmg5/MCatNLO/RUN_PYTHIA8_1/shower.sh: line 44: 824834 Segmentation fault (core dumped) ./Pythia8.exe Pythia8.cmd > mcatnlo_run.log 2>&1
        mv: cannot stat 'Pythia8.hep': No such file or directory

And, "mcatnlo_run.log" reads:

        ./Pythia8.exe: Symbol `_ZTVN7Pythia810StringFlavE' has different size in shared object, consider re-linking
        ./Pythia8.exe: Symbol `_ZTVN7Pythia87StringZE' has different size in shared object, consider re-linking
        ./Pythia8.exe: Symbol `_ZTVN7Pythia89UserHooksE' has different size in shared object, consider re-linking

and the content of debug.log is pasted at the end. I've tried using both g++ (gcc 8.4.1, also tried gcc 11.3.0) and clang (2022.1.2) compilers, but the issue still persists. Also, in case it is useful to know, I'm using Red Hat Linux 8 (with Intel Xeon Gold 6154 processor).

Any help to resolve this would be great. Thank you very much!

Cheers,
Prudhvi

Content of "run_01_tag_1_debug.log" is pasted below:

launch auto
Traceback (most recent call last):
  File "/sw/pkgs/arc/python3.10-anaconda/2023.03/lib/python3.10/shutil.py", line 816, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/home/prudhvib/HEPTools/testmg5/MCatNLO/RUN_PYTHIA8_1/events.hepmc.gz' -> '/home/prudhvib/HEPTools/testmg5/Events/run_01/events_PYTHIA8_0.hepmc.gz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/prudhvib/HEPTools/MG5_aMC_v3_4_2/madgraph/interface/amcatnlo_run_interface.py", line 4094, in run_mcatnlo
    files.mv(os.path.join(rundir, 'events.%s.gz' % ext), hep_file)
  File "/home/prudhvib/HEPTools/MG5_aMC_v3_4_2/madgraph/iolibs/files.py", line 186, in mv
    shutil.move(path1, path2)
  File "/sw/pkgs/arc/python3.10-anaconda/2023.03/lib/python3.10/shutil.py", line 836, in move
    copy_function(src, real_dst)
  File "/sw/pkgs/arc/python3.10-anaconda/2023.03/lib/python3.10/shutil.py", line 434, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/sw/pkgs/arc/python3.10-anaconda/2023.03/lib/python3.10/shutil.py", line 254, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/home/prudhvib/HEPTools/testmg5/MCatNLO/RUN_PYTHIA8_1/events.hepmc.gz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/prudhvib/HEPTools/MG5_aMC_v3_4_2/madgraph/interface/extended_cmd.py", line 1544, in onecmd
    return self.onecmd_orig(line, **opt)
  File "/home/prudhvib/HEPTools/MG5_aMC_v3_4_2/madgraph/interface/extended_cmd.py", line 1493, in onecmd_orig
    return func(arg, **opt)
  File "/home/prudhvib/HEPTools/MG5_aMC_v3_4_2/madgraph/interface/amcatnlo_run_interface.py", line 1805, in do_launch
    self.run_mcatnlo(evt_file, options)
  File "/home/prudhvib/HEPTools/MG5_aMC_v3_4_2/madgraph/interface/amcatnlo_run_interface.py", line 4110, in run_mcatnlo
    raise aMCatNLOError('No file has been generated, an error occurred.'+\
madgraph.interface.amcatnlo_run_interface.aMCatNLOError: No file has been generated, an error occurred. More information in /home/prudhvib/HEPTools/amcatnlo_run.log
Value of current Options:
             pythia8_path : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/pythia8
                hwpp_path : None
              thepeg_path : None
               hepmc_path : None
         madanalysis_path : None
        madanalysis5_path : None
          pythia-pgs_path : None
               rivet_path : None
                yoda_path : None
              contur_path : None
                  td_path : None
             delphes_path : None
      exrootanalysis_path : None
             syscalc_path : None
                  timeout : 60
              web_browser : None
               eps_viewer : None
              text_editor : None
         fortran_compiler : None
            f2py_compiler : None
        f2py_compiler_py2 : None
        f2py_compiler_py3 : None
             cpp_compiler : None
             cluster_type : condor
            cluster_queue : None
    cluster_status_update : (600, 30)
                  fastjet : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/fastjet/bin/fastjet-config
                    golem : None
                  samurai : None
                    ninja : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/lib
                  collier : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/lib
                   lhapdf : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/lhapdf6_py3/bin/lhapdf-config
                 pineappl : pineappl
               lhapdf_py2 : None
               lhapdf_py3 : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/lhapdf6_py3/bin/lhapdf-config
        cluster_temp_path : None
mg5amc_py8_interface_path : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2/HEPTools/MG5aMC_PY8_interface
       cluster_local_path : None
                      OLP : MadLoop
         cluster_nb_retry : 1
       cluster_retry_wait : 300
             cluster_size : 100
      output_dependencies : external
           crash_on_error : False
       auto_convert_model : False
 acknowledged_v3.1_syntax : False
       group_subprocesses : Auto
ignore_six_quark_processes : False
low_mem_multicore_nlo_generation : False
      complex_mass_scheme : False
include_lepton_initiated_processes : True
                    gauge : unitary
             stdout_level : 20
    loop_optimized_output : True
         loop_color_flows : False
   max_npoint_for_channel : 0
  default_unset_couplings : 99
        max_t_for_channel : 99
       zerowidth_tchannel : True
      nlo_mixed_expansion : True
              auto_update : 7
   automatic_html_opening : False
                 run_mode : 2
                  nb_core : 72
      notification_center : True
                 mg5_path : /home/prudhvib/HEPTools/MG5_aMC_v3_4_2

Question information

Language:
English Edit question
Status:
Solved
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Solved by:
Prudhvi Bhattiprolu
Solved:
Last query:
Last reply:
Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said : 		#1

If I understand correctly the error, you will need to recompile pythia8 (I guess). And compile it with the same version of gcc with which you are compiling the rest. Did you try that?

Cheers,

Olivier

Revision history for this message
Prudhvi Bhattiprolu (prudhvibhattiprolu) said : 		#2

Hi Olivier,

Sorry for my late reply. Yes, I am compiling pythia8 with the same version of gcc with which I am compiling the rest. And there were no issues when running pythia8 for showering events at LO, but the issue only seems to be there for NLO events. It seems like a very strange compiler issue that I am unable to resolve, however, when I followed exactly the same steps in installing madgraph and pythia8 on a different linux cluster I am now able to generate NLO events without any compiler issues. I will go ahead and mark this question as solved and will add a comment with details in case I manage to resolve the compiler issue (but any further insights would be great too!).

Thanks a lot,
Prudhvi

Revision history for this message
Jahred (jahreda) said : 		#3

Hi all

I'm noticing that I am getting this same error. I'm running
generate p p > a a [QCD]
and would like to shower with Pythia8. But I'm really confused by the claim that I need to recompile pythia8; it was compiled inside of Madgraph, ie installed using the Madgraph interface. I am trying to set up my environment first using LCG configurations, ie:
source /cvmfs/sft.cern.ch/lcg/views/setupViews.sh LCG_102 x86_64-centos9-gcc11-opt
or (I've tried both)
source /cvmfs/sft.cern.ch/lcg/views/LCG_105/x86_64-el9-gcc12-opt/setup.sh
But I don't know how to find a configuration that matches what MG5@NLO was compiled with. Is there a recommendation that I can use?

Thanks!

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said : 		#4

Hi,

I do know nothing about LCG configuration... So cannot comment on that.
For the above error, I have never faced it myself, it might be another issue when multiple version of pythia are configured with some environment variable pointing to a second version of pythia8.
In that case, maybe the best is to remove the internal version of pythia8 and to use only the external one (or remove the environment linked to the external one).

Maybe the simples is to not use any of those LCG and check if this works.

Cheers,

Olivier

Revision history for this message
Jahred (jahreda) said : 		#5

The LCG configuration is a way to set up stable environments across machines. I don't have to use it, but it simplifies things. The alternative is to know what gcc and gFortran versions are needed. That is not at all obvious; how should a user know that?

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said : 		#6

Hi Jahred,

You can use the version of gcc and gfortran that you want. (as long as it support quadruple precision which should be the case since version 4.6), I'm personally running gfortran13.

If LCG helps, you can use it, but if it does setup pythia8 for you then you can not use the install pythia8 command of madgraph since this will likely create issue. And in that case, you should either to edit the mg5_configuration.txt to point the version of pythia8 setup in your LCG (and in that case as well it is not trivial for a user to know that --but should be possible by looking at the environment variable set by LCG), or edit the LCG environment variable to avoid interference with the local install (which is likely even more complicated).

Cheers,

Olivier

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said : 		#7

Just checked on an account on lxplus and indeed your command seems to setup pythia8 environment variable:

printenv | grep pythia -i
PYTHIA8DATA=/cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-96556/x86_64-centos9-gcc11-opt/share/Pythia8/xmldoc
PYTHIA8=/cvmfs/sft.cern.ch/lcg/releases/MCGenerators/pythia8/307p1-96556/x86_64-centos9-gcc11-opt

Maybe they should also define a MADGRAPH_BASE environment varible to define a path
where they can have a file mg5_configuration.txt where the path to pythia8 would be correctly setup.
This way you could use this LCG out of the box.

Cheers,

Olivier

Revision history for this message
Jahred (jahreda) said : 		#8

Thanks for the help, Olivier. It is a lot easier to set up things using LCG since our local machine doesn't set up gfortran otherwise. What I did was set it up and then unset the PYTHIA environment variables. I tried a fresh installation of PYTHIA8 and then ran and I got further! But now I have a new error? See below. The error in run_01_tag_1_debug.log seems to be:

launch auto
Traceback (most recent call last):
  File "/cvmfs/sft.cern.ch/lcg/releases/Python/3.9.12-9a1bc/x86_64-el9-gcc12-opt/lib/python3.9/shutil.py", line 823, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/MCatNLO/RUN_PYTHIA8_1/events.hepmc.gz' -> '/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/Events/run_01/events_PYTHIA8_0.hepmc.gz'

The full tail of my output to the screen:
INFO: Idle: 0, Running: 0, Completed: 4 [ 0.11s ]
INFO: Collecting events
INFO:
   --------------------------------------------------------------
      Summary:
      Process p p > a a [QCD]
      Run at p-p collider (7000.0 + 7000.0 GeV)
      Number of events generated: 10000
      Total cross section: 1.934e+02 +- 2.2e+00 pb
   --------------------------------------------------------------
      Scale variation (computed from LHE events):
          Dynamical_scale_choice -1 (envelope of 9 values):
              1.929e+02 pb +18.1% -18.7%
   --------------------------------------------------------------

INFO: The /data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/Events/run_01/events.lhe.gz file has been generated.

INFO: Events generated
reweight -from_cards
decay_events -from_cards
INFO: Preparing MCatNLO run
INFO: Using LHAPDF v6.5.3 interface for PDFs
INFO: Compiling MCatNLO for PYTHIA8...
INFO: ... done
INFO: Showering events...
INFO: (Running in /data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/MCatNLO/RUN_PYTHIA8_1)
INFO: Idle: 1, Running: 0, Completed: 0 [ current time: 09h42 ]
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/MCatNLO/RUN_PYTHIA8_1/shower.sh: line 44: 2786185 Segmentation fault (core dumped) ./Pythia8.exe Pythia8.cmd > mcatnlo_run.log 2>&1
mv: cannot stat 'Pythia8.hep': No such file or directory
gzip: events.hepmc: No such file or directory
INFO: Idle: 0, Running: 0, Completed: 1 [ 0.39s ]
Error detected in "launch auto "
write debug file /data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/run_01_tag_1_debug.log
If you need help with this issue please contact us on https://answers.launchpad.net/mg5amcnlo
str : No file has been generated, an error occurred. More information in /data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/amcatnlo_run.log

Thanks again for your help
Jahred

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said : 		#9

I guess that they are no relevant warning/information in
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/amcatnlo_run.log ?

You will likely need to report this to pythia8 author and check if they know why a segfault is occuring her.

Cheers,

Olivier

Revision history for this message
Jahred (jahreda) said : 		#10

That log file doesn't exist - does that point to anything? If I search in MG5_aMC_v2_9_19 for all the log files I see only the ones below:

(so the closest seems to be /data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/mcatnlo.log)

[08:34] login01.af.uchicago.edu:/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19 $ find /data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/ -name '*.log'
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/run_01_tag_1_debug.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uux_aa/gensym.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uux_aa/check_poles.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uux_aa/test_MC.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uux_aa/test_ME.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_dxd_aa/gensym.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_dxd_aa/check_poles.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_dxd_aa/test_MC.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_dxd_aa/test_ME.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uxu_aa/gensym.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uxu_aa/check_poles.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uxu_aa/test_MC.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_uxu_aa/test_ME.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_ddx_aa/gensym.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_ddx_aa/check_poles.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_ddx_aa/test_MC.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/SubProcesses/P0_ddx_aa/test_ME.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/MCatNLO/RUN_PYTHIA8_1/mcatnlo_run.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/yyTest/mcatnlo.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/vendor/CutTools/includects/compiler_version.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/tests/input_files/loop_MSSM/MSSM_NLO.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/tests/input_files/full_sm_UFO/Standard_Model_UFO.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/collier/collier_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/hepmc/hepmc_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/ninja/Ninja/config.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/ninja/ninja_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/pythia8/pythia8_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/MG5aMC_PY8_interface/mg5amc_py8_interface_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/zlib/zlib_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/HEPTools/oneloop/oneloop_install.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/models/hgg_plugin/HEFT_UFO.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/models/taudecay_UFO/taudecay_UFO.log
/data/jahreda/MG5_aMC_v2.9.19/MG5_aMC_v2_9_19/models/MSSM_SLHA2/MSSM_UFO.log