Can not generate systematic weights for LO process when using gridpack

Asked by Evgeny Soldatov

Dear support team,

I'm trying to generate LO process with mg5 with systematics variations enabled:
    systematics = systematics_program # hidden parameter
    ['--mur=0.5,1,2', '--muf=0.5,1,2', '--dyn=-1', '--pdf=errorset,CT14lo@0,MMHT2014lo68cl@0'] = systematics_arguments # hidden parameter

It works fine.
However when I'm enabling the gridpack creation in the run_card.dat and then use the obtained gridpack for the further generation - the weights for systematic variations in the lhe file disappear.

I've tried couple of different processes:
p p > t t~
generate p p > vl vl~ a j j QCD=0 QED=5

Do you have any explanations or ideas, how to fix this?

With the best regards,
Evgeny Soldatov

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#1

Hi,

Gridpack indeed does not perform any post-processing by default.
You will need to run that post-processing code manually (CMS has option in their framework for sure, do not know about ATLAS).

Cheers,

Olivier

> On 7 Jan 2019, at 11:23, Evgeny Soldatov <email address hidden> wrote:
>
> New question #677369 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/677369
>
> Dear support team,
>
> I'm trying to generate LO process with mg5 with systematics variations enabled:
> systematics = systematics_program # hidden parameter
> ['--mur=0.5,1,2', '--muf=0.5,1,2', '--dyn=-1', '--pdf=errorset,CT14lo@0,MMHT2014lo68cl@0'] = systematics_arguments # hidden parameter
>
> It works fine.
> However when I'm enabling the gridpack creation in the run_card.dat and then use the obtained gridpack for the further generation - the weights for systematic variations in the lhe file disappear.
>
> I've tried couple of different processes:
> p p > t t~
> generate p p > vl vl~ a j j QCD=0 QED=5
>
> Do you have any explanations or ideas, how to fix this?
>
> With the best regards,
> Evgeny Soldatov
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Revision history for this message
Evgeny Soldatov (esoldato) said :
#2

Hi Olivier,

Yes, I'm speaking about ATLAS...
In this case we just need a recipe to execute the systematics code standalone that works in a shell (outside of Athena ATLAS framework), and then it should be straightforward to put into Athena.

With the best regards,
Evgeny Soldatov

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#3

Hi,

You can find the instructions here:
https://cp3.irmp.ucl.ac.be/projects/madgraph/wiki/Systematics

to get the systematics.py file,
you have it in
- madevent/bin/internal/systematics.py (except if you have remove it)
- in any standard output under bin/internal
- from the madgraph directories madgraph/various/systematics.py

so you can do something like ./madgraph/various/systematics.py input_path.lhe [output_path.lhe] options.

Note:
1) if you do not specify the output_path, we will overwrite the input file
2) input_file can be either gzip or not
As a related technical note, I would strongly suggest to use python2.7 since python2.6 does not have a nice support for zipped file.

All the options are the same as the one that you can put in the run_card (and described above)

Cheers,

Olivier

On 10 Jan 2019, at 20:28, Evgeny Soldatov <<email address hidden><mailto:<email address hidden>>> wrote:

Question #677369 on MadGraph5_aMC@NLO changed:
https://answers.launchpad.net/mg5amcnlo/+question/677369

   Status: Answered => Open

Evgeny Soldatov is still having a problem:
Hi Olivier,

Yes, I'm speaking about ATLAS...
In this case we just need a recipe to execute the systematics code standalone that works in a shell (outside of Athena ATLAS framework), and then it should be straightforward to put into Athena.

With the best regards,
Evgeny Soldatov

--
You received this question notification because you are an answer
contact for MadGraph5_aMC@NLO.

Revision history for this message
Hannes (hannes3) said :
#4

Hi Olivier,

thank you for the instructions. Do you plan to support the systematics module (and also the reweighting module) in the gridpack script in the future? This would be by far the best solution from the ATLAS standpoint. I think MadSpin is run also from gridpack, when we still used SysCalc we would prevent it from being run, then run the systematics and reweighting modules, and then run MadSpin. This is of course not very robust in case something changes on your side (as it happened with the switch to systematics.py as the default program).

Maybe even more importantly, is it possible (or planned) to steer systematics.py also in gridpack mode with the run_card? This would be much more transparent for us as this is the way we used to steer systematics (and still to do for on-the-fly generation) because this way all settings are clearly documented in the run_card and the header of the LHE file.

Kind regards,
Hannes

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#5

Hi,

> Do you plan to support the systematics
> module (and also the reweighting module) in the gridpack script in the
> future? This would be by far the best solution from the ATLAS
> standpoint.

This is something sensitive --not on the coding side but on the political side--.
CMS is also using the gridpack and they might not agree with that statement.
Maybe we should have a small meeting between CMS/ATLAS and us to discuss about subject like this.

I believe that it is important to CMS/ATLAS have the machinery to run the systematics.py (and also for the reweighting) OFFLINE. (e.g. a year after the initial event generation when more precise pdf are available). If this is the case, then your request would be less critical (I guess).

Now the technical situation for systematics.py, LO/NLO reweighting and MadSpin are quite different

- systematics.py is the easy one, it can be run inside the gridpack without any trouble
(just adding dependencies to a lhapdf version with the python support)

- reweighting is more difficult since
  1) it requires additional package (f2py)
  2) the reweight_card requires dedicated option for an efficient support of gridpack.
     (i.e. it needs a dedicated test run where we setup everything and then keep the information inside the gridpack --thanks to the option "change rwgt_dir")
     Since the generation of the gridpack does not generate any events this force to have something quite new here. But as far as i know here, we could in principle setup the reweight directory via an empty lhe file.
So this is not a real big deal.

- MadSpin has the same issue as the reweighting one (depending of the mode of madspin, you have dependencies to f2py or not), but here you need a large enough sample of events to setup the code.
(since you need to be sure that your max_weight is correct for all the cases.

If you already have a setup for running MadSpin from the gridpack, I would actually suggest to use the same one for the re-weighting since this is the same issue but with less constraint on the statistic of the test run.

> I think MadSpin is run also from gridpack, when we still
> used SysCalc we would prevent it from being run, then run the
> systematics and reweighting modules, and then run MadSpin. This is of
> course not very robust in case something changes on your side

Sorry but I do not understand that statement.

> (as it happened with the switch to systematics.py as the default program).

Actually if both program are present and that you did not specify in the run_card which one to use.
Then the default is to use SysCalc. So we are actually still 100% retro-compatible if you use the same run_card as in the past. Now indeed we have now set the choice of program explicit in the run_card and pointed to systematics since the SysCalc program is not supported anymore by their author.
(and is much less flexible compare to systematics)

> Maybe even more importantly, is it possible (or planned) to steer
> systematics.py also in gridpack mode with the run_card?

Ok I'm confuse now.
What is the difference between this request and the previous one?

Cheers,

Olivier

> On 11 Jan 2019, at 17:12, Hannes <email address hidden> wrote:
>
> Question #677369 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/677369
>
> Hannes posted a new comment:
> Hi Olivier,
>
> thank you for the instructions. Do you plan to support the systematics
> module (and also the reweighting module) in the gridpack script in the
> future? This would be by far the best solution from the ATLAS
> standpoint. I think MadSpin is run also from gridpack, when we still
> used SysCalc we would prevent it from being run, then run the
> systematics and reweighting modules, and then run MadSpin. This is of
> course not very robust in case something changes on your side (as it
> happened with the switch to systematics.py as the default program).
>
> Maybe even more importantly, is it possible (or planned) to steer
> systematics.py also in gridpack mode with the run_card? This would be
> much more transparent for us as this is the way we used to steer
> systematics (and still to do for on-the-fly generation) because this way
> all settings are clearly documented in the run_card and the header of
> the LHE file.
>
> Kind regards,
> Hannes
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Revision history for this message
Zachary Marshall (zach-marshall) said :
#6

Thanks! It sounds from the email thread like there's no problem moving to this for CMS. I think the last question (about configuration of systematics.py) is moot if it runs from inside the system. The worry is that you use the run card to configure the call to systematics.py, and we would have to read the run card and figure out how to make the same call (which isn't great), unless systematics.py reads the run card itself. But if you implement the hooks to run systematics.py automatically in the gridpack, then we should be fine.

Best,
Zach

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#7

Thanks! It sounds from the email thread like there's no problem moving
to this for CMS.

Correct. So I will do it in 2.6.5

. I think the last question (about configuration of
systematics.py) is moot if it runs from inside the system.

ok

 The worry is
that you use the run card to configure the call to systematics.py, and
we would have to read the run card and figure out how to make the same
call (which isn't great),

I do not consider that the run_card and the weight associated to each event should match.
The idea of systematics is to be able to add weights at any stage. So I would strongly advise you to
not parse the run_card to get such information. (a specific header section have been added to the lhef version 3 for that).

unless systematics.py reads the run card
itself.

If you run it with the options --from_card=internal
then we will parse the run_card and do the associate run (if only syscalc option are provide, we run according to those).

But if you implement the hooks to run systematics.py
automatically in the gridpack, then we should be fine.

Sure.

Best,

Olivier

On 13 Jan 2019, at 21:09, Zachary Marshall <<email address hidden><mailto:<email address hidden>>> wrote:

Question #677369 on MadGraph5_aMC@NLO changed:
https://answers.launchpad.net/mg5amcnlo/+question/677369

Zachary Marshall posted a new comment:
Thanks! It sounds from the email thread like there's no problem moving
to this for CMS. I think the last question (about configuration of
systematics.py) is moot if it runs from inside the system. The worry is
that you use the run card to configure the call to systematics.py, and
we would have to read the run card and figure out how to make the same
call (which isn't great), unless systematics.py reads the run card
itself. But if you implement the hooks to run systematics.py
automatically in the gridpack, then we should be fine.

Best,
Zach

--
You received this question notification because you are an answer
contact for MadGraph5_aMC@NLO.

Revision history for this message
Hannes (hannes3) said :
#8

Hi Olivier,

thanks for the detailed response and sorry for touching so many different topics with my short rant.

So first regarding systematics, I think it is great that it is planned to run this automatically from the gridpack, this way there is less room for us to screw up. I think what you describe regarding "--from_card=internal" option is what I was looking for (also as an intermediate solution until 2.6.5) -- a way to run the systematics module externally that still uses the run_card. However, maybe I am misunderstanding sth here but it does ignore the systematics_arguments of the run_card when I run "python madevent/bin/internal/systematics.py events.lhe.gz out.lhe --from_card=internal" (or specify a card). That the run_card and the actual weights are in synch as much as possible is not technically necessary for us but any discrepancies typically make debugging more confusing.

Regarding the backwards compatibility of the systematics: here the problem is that one of our workflows is that user just provides updates to the default run card (and not the run_card itself). So if we would specify sys_pdf etc and from 2.6.2 on they would still be written in the run_card but have no effect because the systematics_arguments (a new parameter not yet on our radar) would be used, which was a bit confusing.

For reweighting, can you quickly sketch what we would need to do run it efficiently with gridpacks? I am afraid we are doing this not very efficiently right now.

Finally, the MadSpin comment was indeed irrelevant, this is something that is only done in our setup and has nothing to do with MG standalone.

Cheers,
Hannes

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#9

Hi,

First, let me say that I have pushed the associated change and validate it both for the "normal" gridpack mode and for the "READONLY" mode.
You can test this with this nightly release: https://bazaar.launchpad.net/~mg5core1/mg5amcnlo/2.6.5/revision/298

> However, maybe I am misunderstanding sth here but it does ignore the systematics_arguments of the run_card when I run "python madevent/bin/internal/systematics.py events.lhe.gz out.lhe --from_card=internal"

It should not. But this might be a bug of your version. Could you send me the run_card that you are using?
(you can not attach email to this thread, so it might be better to open a bug report for that --since you can attach file in that case---)

> Regarding the backwards compatibility of the systematics: here the problem is that one of our workflows is that user just provides updates to the default run card (and not the run_card itself). So if we would specify sys_pdf etc and from 2.6.2 on they would still be written in the run_card but have no effect because the systematics_arguments (a new parameter not yet on our radar) would be used, which was a bit confusing.

I guess that in that case, you should fix your default run_card to a given version. Since otherwise this is impossible to ensure that different version of MG5aMC will return equivalent results. (sometimes default value change/...)

>For reweighting, can you quickly sketch what we would need to do run it efficiently with gridpacks? I am afraid we are doing this not very efficiently right now.

You need to avoid to re-create the matrix-element needed for the re-weighting and therefore need to set the option in the reweight_card: change rwgt_dir
rwgt_dir should not exists the first time (i.e. when you setup the directory for the test run)
and then that directory should be shipped inside the gridpack (i.e. the same strategy as for madspin(

Cheers,

Olivier

Revision history for this message
Hannes (hannes3) said :
#10

Hi Olivier,

thanks, I will create the bug report in a bit, it didn't work in 2.6.2 and 2.6.4 (running systematics --from_card).

I now also remembered which behavior I was hoping for with my initial remarks: that the systematics stuff is automatically figured out from the run card when './bin/madevent systematics run_01' is called, for example. I don't no if this makes sense but I thought this way it would be possible for us to do the post processing of gridpacks in the same way, independent of the systematics program and configuration, because this part is figured out in mg.

Thanks for clarifying the 'change rwgt_dir' option, this sounds straight forward indeed.

Cheers,
Hannes

Can you help with this problem?

Provide an answer of your own, or ask Evgeny Soldatov for more information if necessary.

To post a message you must log in.