doing reweightings in parallel

Hi,

This is indeed only possible to do it in single core.

> Is there a script to merge different reweightings of the same unweighted events, or would I need to write one myself?

We don’t have such script. This would be indeed a nice and safe way to go multicore.

Cheers,

Olivier

On Feb 24, 2014, at 9:21 PM, Andrew Levin <email address hidden> wrote:

> New question #244512 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/244512
>
> Is it possible to do the reweightings described here
>
> https://cp3.irmp.ucl.ac.be/projects/madgraph/wiki/Reweight
>
> in parallel? It seems that madgraph does the reweighting using one core even if you configure it to use many cores for the event generation.
>
> Is there a script to merge different reweightings of the same unweighted events, or would I need to write one myself?
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

In order to do the reweightings in parallel, I was planning to copy the original generation directory many times. However, the path name is hardcoded in various places, including the file

HTML/results.pkl

I tried simply overwriting the path name, but then the unpickling fails.

How do you recommend to do the reweightings in parallel?

Hi Andrew,

I don’t think that they are a need for multiple directory since the information are exchange via stream of data.
The way to split the efficiently work in parallel is probably to make all the reweighting in one go.
and parallel on the event. This is more efficient since like that you evaluate N+1 matrix element and not 2*N
(i.e. you evaluate only once the original matrix-element)

On line 418 of madgraph/interface/reweight_interface.py
you have the way to launch multiple estimator differing by a param_card so this can be done.
This also shows that expect for that input parameter, you can always use the same directory if you want.

Cheers,

Olivier

On Mar 2, 2014, at 6:11 PM, Andrew Levin <email address hidden> wrote:

> Question #244512 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/244512
>
> Andrew Levin posted a new comment:
> In order to do the reweightings in parallel, I was planning to copy the
> original generation directory many times. However, the path name is
> hardcoded in various places, including the file
>
> HTML/results.pkl
>
> I tried simply overwriting the path name, but then the unpickling fails.
>
> How do you recommend to do the reweightings in parallel?
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Hi Olivier,

I looked at these lines in reweight_interface.py, but I am not sure how to use this feature.

Can you give an example about how to do this?

Also, the reweighting will overwrite the unweighted events file. Is there a way to change the location of the output file?

Thanks.

Andrew

Hi Andrew,

I’m not sure either how to handle what you want to do to be honest.
So I can’t give you instructions.

> Also, the reweighting will overwrite the unweighted events file. Is
> there a way to change the location of the output file?

The output file is define at line 279:
output = open( self.lhe_input.name +'rw', 'w')

Cheers,

Olivier

On Mar 3, 2014, at 8:11 AM, Andrew Levin <email address hidden> wrote:

> Question #244512 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/244512
>
> Andrew Levin posted a new comment:
> Hi Olivier,
>
> I looked at these lines in reweight_interface.py, but I am not sure how
> to use this feature.
>
> Can you give an example about how to do this?
>
> Also, the reweighting will overwrite the unweighted events file. Is
> there a way to change the location of the output file?
>
> Thanks.
>
> Andrew
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Do you have any idea how to fix the problem with path names that happens when using a copy of the generation directory?

Doing that might be simpler (although much more inefficient).

Hi,

The only way that i know about is to go trough the unpickle file and when the data are unpicked,
then overwrite the path by the new one at all position where it is needed.

I’m actually doing that for Madspin which has a similar structure. (I don’t run them in parallel but this is needed if I move the Madspin pack after his creation).

you can see this in the file MadSpin/madspin_interface.py:

you have the following lines (line 512)
        if generate_all.path_me != self.options['ms_dir']:
            for decay in generate_all.all_ME.values():
                decay['path'] = decay['path'].replace(generate_all.path_me, self.options['ms_dir'])
                for decay2 in decay['decays']:
                    decay2['path'] = decay2['path'].replace(generate_all.path_me, self.options['ms_dir'])
            generate_all.path_me = self.options['ms_dir'] # directory can have been move
            generate_all.ms_dir = generate_all.path_me

Cheers,

olivier

On Mar 3, 2014, at 11:16 AM, Andrew Levin <email address hidden> wrote:

> Question #244512 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/244512
>
> Andrew Levin posted a new comment:
> Do you have any idea how to fix the problem with path names that happens
> when using a copy of the generation directory?
>
> Doing that might be simpler (although much more inefficient).
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.