Scale systematics not showing up

Asked by Jay Sandesara on 2021-04-02

Hello,

We are trying to generate events with the SMEFTatNLO UFO model, for different dim-6 operators. First we want to generate linear contributions to SM so something like 2Re{O_dim6*SM}. For this I use a syntax as such:

import model SMEFTatNLO-NLO
generate g g > z z NP^2==2 [noborn=QCD]

The problem is the scale systematics dont always show up in the results during the systematics computation of MadGraph - even when the corresponding weights (<weightgroup name="Central scale variation" combine="envelope">) are actually there in the events.lhe file. For example, whenever I 'turn on' the Opt operator in the param card (by setting cpt to 1.0 and setting all other coefficients to e-10), I get only PDF uncertainties displayed in the results - no scale uncertainties.

These are the settings I use in the run_card.dat:

#*********************************************************************
# Store info for systematics studies *
# WARNING: Do not use for interference type of computation *
#*********************************************************************
   True = use_syst ! Enable systematics studies
#
systematics = systematics_program
['--mur=0.5,1,2', '--muf=0.5,1,2', '--pdf=errorset'] = systematics_arguments

Now I realize that this is a 'interference' type computation which the run_card warns us not to use with the systematics module. But strangely, before we realized this and ran the systematics module for other process -Otp*SM (note that Otp , Opt difference) - then the systematics do show up during computation (Even though they are really large and very unstable depending on different dynamical scales)!

So I have two questions:

1) Why is it that MG displays the scale systematics for Otp*SM and not Opt*SM? While it displays PDF uncertainties for both? Are the weights that we do get from these systematics reweightings reliable for these processes?

2) And if this way of finding systematics is unreliable, is there a proper way to include them for this interference type linear computations in an automated way using MadGraph? It should be straightforward for quadratic contributions - Otp*Otp and Opt*Opt works well.

Thanks,
Jay Sandesara

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
2021-04-05
Last reply:
2021-04-09

> 1) Why is it that MG displays the scale systematics for Otp*SM and not Opt*SM? While it displays PDF uncertainties for both? Are the weights that we do get from these systematics reweightings reliable for these processes?

Not sure why to be honest, likely that in the case where it is not displayed it so much clear that the result would be meaningless that we do not include it. I would not trust scale uncertainty for interference out of the box. This requires (at minimum) a validation for each process to see if the printed result makes sense or not. So please do not use such number without having carefully have checked them they are likely wrong.

> 2) And if this way of finding systematics is unreliable, is there a proper way to include them for this interference type linear computations in an automated way using MadGraph? It should be straightforward for quadratic contributions - Otp*Otp and Opt*Opt works well.

You have to re-run the computation with use_syst set to False and change the value of alpsfact.
This will multiply both mur and muf.

Cheers,

Olivier

Just to add that for interference term, I strongly suggest to compare the cross-section with both sde_strategy=1 and sde_strategy=2
(which requires 2.9.x). Since integrating pure interference is quite tricky (especially for loop-induced mode)

Cheers,

Olivier

Jay Sandesara (jaysandesara) said : #3

Hi Olivier,

Thanks for the instructions! Varying alpsfact doesnt seem to change anything likely because I dont do merging. But I think varying scalefact should do the trick, as it says in the run card that it is multiplies the factor to the scales.

a) Am I correct in this assumption that setting scalefact=0.5, 1.0 and 2.0 should work the same as '--mur=0.5,1,2', '--muf=0.5,1,2' of use_syst setting? Except I guess, we wont be able to independantly vary the two scales?

Assuming it is right for now, I see that the scale variations only take effect when using the dynamic scale choice (for cpt=1 it gives a max scale uncertainty of about 30% from the central value with -1 dynamical scale choice). SMEFTatNLO authors recommend to set the reno/fact scales fixed, and to set the mu_eft scale (the scale at which the dim-6 operator coefficients are defined in the model) equal to the fixed scale. This is because MadGraph doesnt evolve these coefficients and so it is kept separate and needs to be fixed manually. I then plan to vary the mu_eft scales same as mu_R and mu_F, somehow taking into account all the running/mixing effects of the operator coefficients, to determine the uncertainties associated with that scale.

So,

b) Is there a way to vary the scales when using fixed mu_R, mu_F scale values?

In the Madgraph paper, it is mentioned that the scales work as such: we have muR_ref_fixed and muR_over_ref. muR_over_ref is set to 2 and so the muR scale is 2*muR_ref_fixed. I guess in the recent versions this terminology seems to have changed. In the run_card.dat I only see scale, dsqrt_q2fact1, dsqrt_q2fact1.

c) So would setting (note that the choice of the scales is due to the fact that we are studying off-shell Higgs boson):

 350.0 = scale ! fixed ren scale
 350.0 = dsqrt_q2fact1 ! fixed fact scale for pdf1
 350.0 = dsqrt_q2fact2 ! fixed fact scale for pdf2

be correct if I want to set muR=muF=350? Or would the correct values be 350/2 = 175? And why is there also another entry in the param_card where you can set MU_R?

And finally, even without varying the sde_strategy, for Otp*SM the xsection values do not seem to agree for multiple runs with the exact same configs. For example, following are the results for cross section from two different runs with ctp=10.0 and scalefact=1.0, with a generation of 100k events each:

Cross-section (run 1) : 0.001499 +- 9.569e-05 pb
Cross-section (run 2) : 0.001789 +- 2.409e-05 pb

d) Even taking into account the statistical uncertainties these results do not seem to agree. Does this mean that we cannot accurately calculate this process?

Thanks,
Jay Sandesara

Hi Olivier,

> Thanks for the instructions! Varying alpsfact doesnt seem to change anything likely because I dont do merging. But I think varying scalefact should do the trick, as it says in the run card that it is multiplies the factor to the scales.

Sorry my bad, I confused those two

> a) Am I correct in this assumption that setting scalefact=0.5, 1.0 and 2.0 should work the same as '--mur=0.5,1,2', '--muf=0.5,1,2' of use_syst setting? Except I guess, we wont be able to independantly vary the two scales?

yes

> b) Is there a way to vary the scales when using fixed mu_R, mu_F scale values?

I do not understand the question, you can set the value to the one you want, so you can pick the value that you want.

> In the Madgraph paper, it is mentioned that the scales work as such: we have muR_ref_fixed and muR_over_ref. muR_over_ref is set to 2 and so the muR scale is 2*muR_ref_fixed. I guess in the recent versions this terminology seems to have changed. In the run_card.dat I only see scale, dsqrt_q2fact1, dsqrt_q2fact1.

The description that you are referring too is for NLO computation, but you are running a LO computation where the run_card is different and the handling of the scale is indeed not fully identical between LO and NLO computation.

>c) So would setting (note that the choice of the scales is due to the fact that we are studying off-shell Higgs boson):
be correct if I want to set muR=muF=350? Or would the correct values be 350/2 = 175? And why is there also another entry in the param_card where you can set MU_R?

Sounds fine to me, but if you would have set 175 everywhere I would have say the same. Choosing 350 or 175 for the scale are both correct choices. The MU_R in the param_card is related to the fact that the model need to define a symbol for the renormalization scale since some of the counter-term need that value explicitly.
That input is important if you want to run in standalone (madloop) mode since in that case, you do not have any run-card.
That parameter is ignored when running LO/NLO cross-section computation (as the value for alpha-s).

>And finally, even without varying the sde_strategy, for Otp*SM the xsection values do not seem to agree for multiple runs with the exact same configs. For example, following are the results for cross section from two different runs with ctp=10.0 and scalefact=1.0, with a generation of 100k events each:

This shows a clear sign that the phase-space integration has issue.
If you are using sde_strategy=1, this is not really surprising to me (since the hyppothesis behind all optimization when using sde_strategy=1 is that we are dominated by the classical regime (in other word, that interference are a correction term and therefore small)). I hope that sde_strategy=2 would behave in a better way but I have never checked that mode on interference.
Integration of interference is really something on which I have no clue on how to do it efficiently.

> d) Even taking into account the statistical uncertainties these results do not seem to agree. Does this mean that we cannot accurately calculate this process?

Sounds like that, as said above integration of interference alone term is something that we have never optimized our code on.
We just provide a code designed for something else and found not too bad results (compare to our expectations) but indeed this is not always working. I hope that sde_strategy new mode of integration would help for that direction but again the purpose of that new mode was not targeting interference so it will fail short on some process for sure.

In top of that you do that on loop-induced process, which is in itself already very challenging for the phase-space integrator. So this is likely one of the most challenging combination to integrate.

Now we do not have any issue with the evaluation of the loop, and you have plenty of parameter (see the loop-induced paper and the helicity-recycling paper) that can change the way the phase-space parametrization will be done. So this might be "enough" to find one configuration where everything is working fine.

Can you help with this problem?

Provide an answer of your own, or ask Jay Sandesara for more information if necessary.

To post a message you must log in.