Avoid recompilation for generate events?

Asked by Joshua Ellis

Hi,

It appears that each call of `generate_events` causes all subprocesses to be recompiled which can takes up a lot of time.
 Is it possible to avoid the recompilation if the subprocesses are already compiled?

In particular, I wish to update certain parameters in `param_card.dat` with everything else remaining the same. It would seem to me that the compilation step is unnecessary and thus I would greatly speed up the generation of events if it weren't present.

Regards,

Josh

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#1

Hi,

To avoid that each run of the executable have to read the param_card and parse it. (Which is a IO bottleneck on a lot of machines)
We prefer to convert the param_card to an fortran file which is compile within the executable.
Such that the parsing is done only once. This indeed force a recompilation. But only from one file (+the final linking).
This also ensure that the user can modify the param_card for the next run without creating a possible problem with the current run.

So this step is really necessary.

Cheers,

Olivier

> On 23 May 2017, at 16:04, Joshua Ellis <email address hidden> wrote:
>
> New question #633706 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/633706
>
> Hi,
>
> It appears that each call of `generate_events` causes all subprocesses to be recompiled which can takes up a lot of time.
> Is it possible to avoid the recompilation if the subprocesses are already compiled?
>
> In particular, I wish to update certain parameters in `param_card.dat` with everything else remaining the same. It would seem to me that the compilation step is unnecessary and thus I would greatly speed up the generation of events if it weren't present.
>
> Regards,
>
> Josh
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Revision history for this message
Joshua Ellis (jp-ellis) said :
#2

Hi,

Thanks for the quick reply.

I did not know that the param_card was converted to a Fortran file, so it makes sense that the recompilation is necessary.

Still, why does it have to recompile each subprocess? Even though things have already been compiled, the second run of generate_events takes a comparable time to the first run. Is the parameters file recompiled for each subprocess? I thought that it would be (re)compiled once and then linked to each (already compiled) subprocess?

Regards,

Josh

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#3

Hi,

> Still, why does it have to recompile each subprocess?

We compile everything statically so this force to recompile the executable. If it was a dynamical linking this should not be necessary.

Now the fact that this takes long is likely due that you have a huge number of SubProcesses which is typically an indication that your model is not
Correctly optimised. I would advise to look at the following link:
https://answers.launchpad.net/mg5amcnlo/+faq/2312
If I'm correct this will help you quite a lot.

Cheers,

Olivier

> On 23 May 2017, at 16:34, Joshua Ellis <email address hidden> wrote:
>
> Question #633706 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/633706
>
> Status: Answered => Open
>
> Joshua Ellis is still having a problem:
> Hi,
>
> Thanks for the quick reply.
>
> I did not know that the param_card was converted to a Fortran file, so
> it makes sense that the recompilation is necessary.
>
> Still, why does it have to recompile each subprocess? Even though
> things have already been compiled, the second run of generate_events
> takes a comparable time to the first run. Is the parameters file
> recompiled for each subprocess? I thought that it would be (re)compiled
> once and then linked to each (already compiled) subprocess?
>
> Regards,
>
> Josh
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#4

Just to add.

If I did the following:

Cd Source
Touch ../Cards/param_card.dat
make

> ../bin/madevent treatcards param
> Running MG5 in debug mode
> No module named madgraph
> ************************************************************
> * *
> * W E L C O M E to *
> * M A D G R A P H 5 _ a M C @ N L O *
> * M A D E V E N T *
> * *
> * * * *
> * * * * * *
> * * * * * 5 * * * * *
> * * * * * *
> * * * *
> * *
> * VERSION 2.5.4 20xx-xx-xx *
> * *
> * The MadGraph5_aMC@NLO Development Team - Find us at *
> * https://server06.fynu.ucl.ac.be/projects/madgraph *
> * *
> * Type 'help' for in-line help. *
> * *
> ************************************************************
> INFO: load configuration from /Users/omattelaer/.mg5/mg5_configuration.txt
> INFO: load configuration from /Users/omattelaer/Documents/workspace/2.5.5/PROC_sm_32/Cards/me5_configuration.txt
> INFO: load configuration from /Users/omattelaer/Documents/workspace/2.5.5/input/mg5_configuration.txt
> INFO: load configuration from /Users/omattelaer/Documents/workspace/2.5.5/PROC_sm_32/Cards/me5_configuration.txt
> treatcards param
> quit
> INFO:
>
> INFO:
>
> cd MODEL; make
> gfortran -O -w -fbounds-check -fPIC -ffixed-line-length-132 -c -o rw_para.o rw_para.f
> ar cru ../../lib/libmodel.a couplings.o lha_read.o printout.o rw_para.o model_functions.o couplings1.o couplings2.o
> ranlib ../../lib/libmodel.a

So you can see that
1) the makefile call python to parse the "new" param_card and create the new file
2) the Model libary is recompiled (only one fortran file is recompiled)

Then if I do:
 cd ../P1_qq_ttxqq/
make

> gfortran -o madevent driver.o myamp.o genps.o unwgt.o setcuts.o get_color.o cuts.o cluster.o reweight.o initcluster.o addmothers.o setscales.o idenparts.o auto_dsig.o auto_dsig1.o auto_dsig2.o auto_dsig3.o auto_dsig4.o auto_dsig5.o auto_dsig6.o auto_dsig7.o matrix1.o matrix2.o matrix3.o matrix4.o matrix5.o matrix6.o matrix7.o -L../../lib/ -ldhelas -ldsample -lmodel -lgeneric -lpdf -lcernlib -lbias -lc++ -mmacosx-version-min=10.7

So you can see that none of the fortran file are recompile here, just the linking/final executable is recompiled.

So clearly this compilation is much faster than the original one.

Cheers,

Olivier

> On 23 May 2017, at 16:34, Joshua Ellis <email address hidden> wrote:
>
> Question #633706 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/633706
>
> Status: Answered => Open
>
> Joshua Ellis is still having a problem:
> Hi,
>
> Thanks for the quick reply.
>
> I did not know that the param_card was converted to a Fortran file, so
> it makes sense that the recompilation is necessary.
>
> Still, why does it have to recompile each subprocess? Even though
> things have already been compiled, the second run of generate_events
> takes a comparable time to the first run. Is the parameters file
> recompiled for each subprocess? I thought that it would be (re)compiled
> once and then linked to each (already compiled) subprocess?
>
> Regards,
>
> Josh
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Can you help with this problem?

Provide an answer of your own, or ask Joshua Ellis for more information if necessary.

To post a message you must log in.