Systematic Errors and Pythia Matching

Hi,

Matched and merged sample need to be validated to be sure that the you have a smooth transition between the matrix-element regime and the parton-shower regime (and that you are not impacted by the approximation done on the Sudakov which are approximate within MLM)
So this should answer your question about if you can keep or not the default parameter.

> 1) Can I assume that the relative systematic error on the matched cross-section is the same as that on the unmatched? If not, how can I find/estimate the systematic uncertainty on the matched cross-section?

The difference between the two systematics should be small and therefore I would say that you can keep the same.
You can however recompute those error from the additional weight present in the hepmc file and recompute the error from those weights (same for all the type of uncertainty actually). This is actually how you would do it if you want to see the importance of such error for differential distribution.

> 2) To get the total systematic uncertainty on the unmatched x-section , do I add (in quadrature) the scale variation, emission scale variation, central scheme variation and pdf variation. Do I need to do anything with the dynamical scales? Is it reasonable to presume that the scale and pdf variations are uncorrelated?

I typically do not combine those and I do not think that it is reasonable to consider any of them as uncorrelated.
But please check with your experimental (or theoretical) colleages what is the way to handle those in your community.
This is more a prescription than anything else (and scale uncertainty are not an uncertainty in the statistical sense so this is not clear that you can even combine it with the PDF variation which is)

Cheers,

Olivier

> On 4 Sep 2021, at 15:15, Hannah Banks <email address hidden> wrote:
>
> Question #698625 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/698625
>
> Description changed to:
> Hello,
> I am looking to compute an estimate of the total uncertainty on a Pythia8 Matched cross-section and would be incredibly grateful for any advice.
>
> I have set use_syst = True in the run card.
>
> Madgraph then outputs:
>
> #
> # original cross-section: 4.29231243503e-05
> # scale variation: +9.53% -7.25%
> # emission scale variation: +9.53% -16.8%
> # central scheme variation: +5.33% - 0%
> # PDF variation: +27.3% -27.3%
> #
> # dynamical scheme # 1 : 4.52103e-05 +0.0207% -0.95% # \sum ET
> # dynamical scheme # 2 : 4.52103e-05 +0.0207% -0.95% # \sum\sqrt{m^2+pt^2}
> # dynamical scheme # 3 : 4.50192e-05 +0.469% -2.45% # 0.5 \sum\sqrt{m^2+pt^2}
> # dynamical scheme # 4 : 4.51387e-05 +0.579% -1.64% # \sqrt{\hat s}
>
>
> In addition to :
>
> === Results Summary for run: run_01 tag: tag_1 ===
>
> Cross-section : 4.294e-05 +- 9.316e-08 pb
> Nb of events : 10000
> Pythia8 merged cross-sections are:
>> Merging scale = 55 : 1.8171e-05 +/- 3.9e-09 [pb]
>> Merging scale = 82.5 : 1.8261e-05 +/- 3.9e-09 [pb]
>> Merging scale = 110 : 1.8059e-05 +/- 3.9e-09 [pb]
> Notice that because Systematics computation is turned on, the merging did not veto events but modified their weights instead.
> The resulting hepmc/stdhep file should therefore be use with those weights.
>
>
>
> As far as I can see, the stated uncertainty on the pythia merged x-section is the statistical uncertainty, and the relative uncertainties calculated systematic studies are relevant to the unmatched cross-section.
>
> I have the following questions:
> 1) Can I assume that the relative systematic error on the matched cross-section is the same as that on the unmatched? If not, how can I find/estimate the systematic uncertainty on the matched cross-section?
>
> 2) To get the total systematic uncertainty on the unmatched x-section ,
> do I add (in quadrature) the scale variation, emission scale variation,
> central scheme variation and pdf variation. Do I need to do anything
> with the dynamical scales? Is it reasonable to presume that the scale
> and pdf variations are uncorrelated?
>
> 3) I have not changed any of the parameters in the run card related to
> the systematics calculation - so I guess that they are running on
> default. Should I have stated any of these explicitly?
>
> I would be extremely grateful for any help with this.
>
> Thanks very much
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Thank you very much for your response I am very grateful!

I still have a couple of questions:

a) With regards to the default parameters, I meant the hidden parameters associated with the systematics calculation (not those associated with matching). Is it fine to just leave turn the systematics calculations on, or do I need to specify any of the additional parameters such as the program etc. ?

b) Is there anyway of running the systematics calculation on the reweighted pythia output? Or is the difference in the systematics from the parton calculation (unweighted files) likely to be so negligible that it is not worth considering?

c) In https://answers.launchpad.net/mg5amcnlo/+question/243995 it looks like adding in quadrature was recommended? I was just wondering why it was considered ok here?

Thanks again for all of your help

> a) With regards to the default parameters, I meant the hidden parameters
> associated with the systematics calculation (not those associated with
> matching). Is it fine to just leave turn the systematics calculations
> on, or do I need to specify any of the additional parameters such as the
> program etc. ?

The hidden parameter are for SysCalc that is not really supported anymore.
For the new systematics code, the parameter should be ok as default but you can tune those obviously.
But in order to have an idea of the error band most people use to multiply/divide the scale by two and this is what is used.

> b) Is there anyway of running the systematics calculation on the
> reweighted pythia output? Or is the difference in the systematics from
> the parton calculation (unweighted files) likely to be so negligible
> that it is not worth considering?

Maybe but I do not know a way for doing that. I know that RIVET has some support for multi-weight so they might be able to compute those error now but we do not provide tools for computing error from that format.

> c) In https://answers.launchpad.net/mg5amcnlo/+question/243995 it looks
> like adding in quadrature was recommended? I was just wondering why it
> was considered ok here?

My understanding of that thread is to combine them linearly not in quadrature.
This still assume that the meaning of the various error are all statistical.
Now with Syscalc you had less type of error quote (you did not had the central scheme variation one).
That error in particular does not make any sense to be added in quadrature (or even linearly) to the scale variation since they both compute a scale uncertainty but in different way. This error is to be sure that we realise if we have a large bias due to a wrong choice a dynamical scale or not.

Cheers,

Olivier

> On 6 Sep 2021, at 11:30, Hannah Banks <email address hidden> wrote:
>
> Question #698625 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/698625
>
> Status: Answered => Open
>
> Hannah Banks is still having a problem:
> Thank you very much for your response I am very grateful!
>
> I still have a couple of questions:
>
> a) With regards to the default parameters, I meant the hidden parameters
> associated with the systematics calculation (not those associated with
> matching). Is it fine to just leave turn the systematics calculations
> on, or do I need to specify any of the additional parameters such as the
> program etc. ?
>
> b) Is there anyway of running the systematics calculation on the
> reweighted pythia output? Or is the difference in the systematics from
> the parton calculation (unweighted files) likely to be so negligible
> that it is not worth considering?
>
> c) In https://answers.launchpad.net/mg5amcnlo/+question/243995 it looks
> like adding in quadrature was recommended? I was just wondering why it
> was considered ok here?
>
> Thanks again for all of your help
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Thank you so much for your response!

1) to get a rough estimate for the total systematic uncertainty, would it therefore be reasonable to add (linearly) the Scale variation, emission scale variation and pdf variation?

2) Is the default pdf calculation using different pdfs, or using different replicas from the same pdf?

3) Just to check: it is the parton_systematics.log file in which I should look for the results of the systematic calculations ?

Thank you very much for all of your help, I am extremely grateful!

Hi,

> 1) to get a rough estimate for the total systematic uncertainty, would
> it therefore be reasonable to add (linearly) the Scale variation,
> emission scale variation and pdf variation?

Maybe you should check the syscalc paper (or the equivalent sherpa paper) where they might discuss this.
In the syscalc paper you will get the exact definition used for the "emission scale variation" and then can see how that can be combine with the scale variation if this is not discussed in the paper.
But you can likely say that you have a theoretical error of the order of 40%.
(what surprises me a bit is why you have such large PDF uncertainty, this is typically much smaller)

> 2) Is the default pdf calculation using different pdfs, or using
> different replicas from the same pdf?

The default is to use the formula/method associated to the pdf to estimate it.
This can be replica or hessian or something else depending of the pdf selected.
The computation of the PDF error is done within lhapdf and follow the prescription of the associated PDF set.

> 3) Just to check: it is the parton_systematics.log file in which I
> should look for the results of the systematic calculations ?

I typically do not use that file but this files contains the cross-section for each variation computed, which can be usefull if you want to change the way the computation is done (in particular for the scale uncertainty since some people do not use the 9 point for the computation but only 7)

Cheers,

Olivier

> On 7 Sep 2021, at 11:05, Hannah Banks <email address hidden> wrote:
>
> Question #698625 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/698625
>
> Status: Answered => Open
>
> Hannah Banks is still having a problem:
> Thank you so much for your response!
>
> 1) to get a rough estimate for the total systematic uncertainty, would
> it therefore be reasonable to add (linearly) the Scale variation,
> emission scale variation and pdf variation?
>
> 2) Is the default pdf calculation using different pdfs, or using
> different replicas from the same pdf?
>
> 3) Just to check: it is the parton_systematics.log file in which I
> should look for the results of the systematic calculations ?
>
> Thank you very much for all of your help, I am extremely grateful!
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Thank you very much for this!

I was just looking for a file which outputs the results of the systematic calculations. It appears that this log file does this? Is there another file that it would be easier/better to look at?

Many thanks!!

No this should be the file with the relevant information.

Cheers,

Olivier