MadSpin very memory hungry for ttWj

Hi,

I can try to see what I can do. But I have to set this as low priority for the moment.
The only thing that I can try is to try to free some memory from the python executable.

To see if this will work, one method would be to test if you can compile the directory manually.
If you can, reducing the python memory would help.

On that matter did you use the low memory mode for the NLO generation?
Did you try to split the generation of the matrix element and of the run?
Splitting them would obviously reduce the memory.
If you do then the problem is a pure MadSpin issue which would be difficult to solve.
(But I doubt that you reach 12 Gb only for MadSpin for such process)

Cheers,

Olivier

> On 9 Jul 2018, at 14:02, Josh McFayden <email address hidden> wrote:
>
> New question #670762 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/670762
>
> Hi all,
>
> We are trying to generate ttW+0,1j@NLO with FxFx interfaced with MadSpin for the top and W decays.
>
> I have tried this several times in the last ~5 days and each time the integration steps run fine, but the MadSpin step does not. It seems to be incredibly memory hungry (~12GB of virtual memory according to top). Is this expected?
>
> The crash is typically something like:
>
> INFO: generate p p > t t~ w, (t~ > b~ w- , w- > all all QCD=99), w- > all all QCD=99, w+ > all all QCD=99, (t > b w+ , w+ > all all QCD=99) --no_warning=duplicate;define pert_QCD = -5 -4 -3 -2 -1 1 2 3 4 5 21;add process p p > t t~ w pert_QCD, (t~ > b~ w- , w- > all all QCD=99), w- > all all QCD=99, w+ > all all QCD=99, (t > b w+ , w+ > all all QCD=99) --no_warning=duplicate;add process p p > t t~ w j, (t~ > b~ w- , w- > all all QCD=99), w- > all all QCD=99, w+ > all all QCD=99, (t > b w+ , w+ > all all QCD=99) --no_warning=duplicate;define pert_QCD = -5 -4 -3 -2 -1 1 2 3 4 5 21;add process p p > t t~ w j pert_QCD, (t~ > b~ w- , w- > all all QCD=99), w- > all all QCD=99, w+ > all all QCD=99, (t > b w+ , w+ > all all QCD=99) --no_warning=duplicate;
> Command "launch --name=Test" interrupted with error:
> MadGraph5Error : Impossible to compile /afs/cern.ch/work/m/mcfayden/mcgen/MadGraphControl/MC16ttVFxFx/run/ttW01j_FxFx_int_wMadSpin/PROCNLO_loop_sm-no_b_mass_0/MadSpin/full_me/Source directory
> Trying to launch make command returns:
> [Errno 12] Cannot allocate memory
> In general this means that your computer is not able to compile.
> Please report this bug on https://bugs.launchpad.net/mg5amcnlo
> More information is found in '/afs/cern.ch/work/m/mcfayden/mcgen/MadGraphControl/MC16ttVFxFx/run/ttW01j_FxFx_int_wMadSpin/PROCNLO_loop_sm-no_b_mass_0/Test_tag_1_debug.log'.
> Please attach this file to your report.
>
> Is there any way to reduce the load? I am running on a machine with 30GB of RAM, but but it's shared across many concurrent users...
>
> Best,
>
> Josh
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Hi Olivier,

I did not yet try the low memory mode, how do I turn it on? Somehow I missed this :-)

The memory issue definitely is in the MadSpin step. I know this from tracking the job in real time and in fact it peaks at 26GB!
It starts from the "generate p p > t t~ w, (t~ > b~ w- , w- > all all QCD=99), w- > all all ......." step and continues into "Compiling code".

Best,

Josh

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Hi Olivier,
Finally getting back to this. How should I turn on the low memory option?
Best,
Josh

before the generation you have to run the following command:
set low_mem_multicore_nlo_generation

Did you make the test to run MadSpin independently of the event generation?

Cheers,

Olivier