W+0,1,2jets NLO gridpacks are failing

Hi,

The generation of the code requires typically a lot of RAM available.
To clean that memory space, the easiest is to close MG5aMC and then run the code via another instance.
If you really want to do everything inside the same session, importing a new model should free most of the RAM consumed by the generation of the code.

Since 2.4.0, we also introduced a mode using less RAM for the generation (see the message of the UpdatesNotes below):
        MZ: new NLO generation mode. It is more efficient from the memory and CPU point of
            view, in particular for high-multiplicity processes.
            Many thanks to Josh Bendavid for his fundamental contribution for this.
            The mode can be enabled with
            > set low_mem_multicore_nlo_generation True
            before generating the process.

Did you try that?

Cheers,

Olivier

> On Nov 7, 2017, at 20:23, Marco Trovato <email address hidden> wrote:
>
> New question #660459 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/660459
>
> Hi,
>
> I've been trying to generate W+0,1,2jets samples at NLO from
>
> 1) lxplus - locally
> 2) lxplus - lsf jobs
> 3) lxplus - condor jobs
> 4) login.uscms.org - CMS connect
>
> Madgrapg version is 5.2.4
>
> All jobs are either stuck or have failed because of memory issues [1] and no gridpacks were produced. I tried to increase the minimum desired memory to 4Gb but no luck. Do you have any suggestion on that?
> Cards are here [2], Logs are here [3]
>
> The last attempt I made is to raise the jet pt cuts from 10 GeV to 30 GeV and place a maximum 3.0 jet abs(eta) cut. With those changes I went a bit further in the generation (first cross section estimation was seen), but the gridpack production crashed here [4]. Do you know what the issue is? I get "Fortran runtime error: End of file" for the P2_dxux_tapvtuxux log file. Cards are here [5]
>
> Please let me know if I can provide you with more info.
> Thanks a lot
> Best,
> Marco
>
>
>
> [1]:
> ***WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_codegen.log***
> ...
> INFO: Generating Helas calls for FKS process: d g > mu- vm~ u g [ all = QCD ] $$ t t~ h @5 (1461 / 2340)
> ^[[1;31mCommand "import /srv/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_gridpack/work/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_proc_card.dat" interrupted in sub-command:
> "output WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX -nojpeg" with error:
> MemoryError :
> Please report this bug on https://bugs.launchpad.net/mg5amcnlo
> More information is found in 'MG5_debug'.
> Please attach this file to your report.^[[0m
> quit
> ...
>
> [2] :
> http://nuhep.northwestern.edu/~mtrovato/W012J_NLOcards/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX/
>
> [3]:
> http://nuhep.northwestern.edu/~mtrovato/W012J_NLOlogs/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX/
>
> [4]
> ***gridpack_generation.log***
> ...
> INFO: Idle: 0, Running: 1, Completed: 543 [ 2h 4m ]
> INFO: Idle: 0, Running: 1, Completed: 543 [ 2h 9m ]
> INFO: All jobs finished
> INFO: Idle: 0, Running: 0, Completed: 544 [ 2h 14m ]
> ^[[1;31mError detected in "launch -n pilotrun"
> write debug file /stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/pilotrun_tag_1_debug.log
> If you need help with this issue please contact us on https://answers.launchpad.net/mg5amcnlo
> aMCatNLOError : An error occurred during the collection of results.
> Please check the .log files inside the directories which failed:
> /stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/SubProcesses/P2_dxux_tapvtuxux/GF1/log.txt
> ^[[0m
> ...
> ***pilotrun_tag_1_debug.log****
> ...
> launch -n pilotrun
> Traceback (most recent call last):
> File "/stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/bin/internal/extended_cmd.py", line 1009, in \
> onecmd
> return self.onecmd_orig(line, **opt)
> File "/stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/bin/internal/extended_cmd.py", line 964, in o\
> necmd_orig
> return func(arg, **opt)
> File "/stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/bin/internal/amcatnlo_run_interface.py", line\
> 1205, in do_launch
> evt_file = self.run(mode, options)
> File "/stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/bin/internal/amcatnlo_run_interface.py", line\
> 1433, in run
> jobs_to_collect,mint_step,mode,mode_dict[mode],fixed_order=False)
> File "/stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/bin/internal/amcatnlo_run_interface.py", line\
> 1640, in collect_the_results
> self.append_the_results(jobs_to_run,integration_step)
> File "/stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/bin/internal/amcatnlo_run_interface.py", line\
> 1883, in append_the_results
> '\n'.join(error_log)+'\n')
> aMCatNLOError: An error occurred during the collection of results.
> Please check the .log files inside the directories which failed:
> /stash2/user/mtrovato/genproductions/bin/MadGraph5_aMCatNLO/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight_gridpack/work/processtmp/SubProcesses/P2_dxux_tapvtuxux/GF1/log.txt
> ...
> ***process/SubProcesses/P2_dxux_tapvtuxux/GF1/log.txt***
> ...
> Enter alpha, beta for G_azi
> Enter alpha>0 to set G_azi=0 (no azi corr)
> At line 244 of file driver_mintMC.f (unit = 12, file = 'mint_grids')
> Fortran runtime error: End of file
> Thanks for using LHAPDF 6.2.1. Please make sure to cite the paper:
> Eur.Phys.J. C75 (2015) 3, 132 (http://arxiv.org/abs/1412.7420)
> for G_azi: alpha= -1.0000000000000000 , beta= -0.10000000000000001
> ...
>
>
> [5]:
> http://nuhep.northwestern.edu/~mtrovato/W012J_NLOcards/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/
>
> [6]:
> http://nuhep.northwestern.edu/~mtrovato/W012J_NLOlogs/WJetsToLNu_012j_Wpt-150ToInf_5f_NLO_FXFX_tight/
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Hi Olivier,

I followed your suggestion, but I now seem to exhaust the disk space on the host site. I followed up with the appropriate experts. I will keep you posted
Thanks a lot
Marco