Simulation of EFT samples with amplitude decomposition

Asked by Stefanie on 2019-08-27

Dear experts,

I'm running Madgraph with an EFT model and with the decomposition method of the different contribution for the different terms in the amplitude. The model I'm using is this one:

http://feynrules.irmp.ucl.ac.be/attachment/wiki/AnomalousGaugeCoupling/SM_LS012_LM017_LT012_Ind5_UFO_June19.tar.gz

I'm mainly testing the same-charge WW leptonic final state (for this particular model). My process definitions look like these (examples):

SM case:

define l+ = e+ mu+ ta+
define vl = ve vm vt
define l- = e- mu- ta-
define vl~ = ve~ vm~ vt~
define p = g u c d s u~ c~ d~ s~ b b~
define j = g u c d s u~ c~ d~ s~ b b~
generate p p > l+ vl l+ vl j j QCD=0 S0=1 S1=1 S2=1 M0=1 M1=1 T0=1 T1=1 T2=1 @1
add process p p > l- vl~ l- vl~ j j QCD=0 S0=1 S1=1 S2=1 M0=1 M1=1 T0=1 T1=1 T2=1 @1

Interference between SM and BSM:

[same particles]
generate p p > l+ vl l+ vl j j QCD=0 T0^2==1 @1
add process p p > l- vl~ l- vl~ j j QCD=0 T0^2==1 @1

Quadratic BSM term:

[same particles]
generate p p > l+ vl l+ vl j j QCD=0 T0^2==2 @1
add process p p > l- vl~ l- vl~ j j QCD=0 T0^2==2 @1

The problem that I'm seeing is in the validation of the decomposition method where I compare the sum of the latter three samples with a fully generated BSM sample. I define it as such:

[same particles]
generate p p > l+ vl l+ vl j j QCD=0 T0=1 @1
add process p p > l- vl~ l- vl~ j j QCD=0 T0=1 @1

Looking into the total cross sections after event generation, the closure looks totally fine:
SM = 0.037113 pb
SM-BSM interference = -9.626e-05 pb
BSM quadratic = 1.898e-03
Sum of the above = 0.038915 pb
Full sample = 0.038669 pb

This makes a 0.64% difference between the sum and the full sample which looks very good. The closure in the total cross section looks very good for all tested points (The ssWW process in sensitive to 8 of the 29 parameters of this model.) Now if I look further into differential distributions (after showering with Pythia8.240) many of the parameter points look very good. However, some of the parameters show discrepancies in the tails of some observables. I have put some plots for the invariant di-boson mass at ME level (mWW), the invariant di-lepton mass here:

https://cernbox.cern.ch/index.php/s/9FE07azikDmKa3W

I'm using MG version 5.2.6.5 for GP generation and 5.2.6.2 for event generation.

We have also seen similar issues with the WZjj fully leptonic final state in this model:

https://cp3.irmp.ucl.ac.be/projects/madgraph/wiki/Models/EWdim6

I also contacted Oscar Eboli who provided the model. He could not say where this was coming from, which is why I now write to the Madgraph support.

Thanks a lot for your reply.
Best regards,
Stefanie

Question information

Language:
English Edit question
Status:
Solved
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Solved by:
Olivier Mattelaer
Solved:
2019-09-16
Last query:
2019-09-16
Last reply:
2019-09-15

Hi,

Note that we never published any paper describing the possibility to integrate the interference only term.
The reason is that we have no clue how to do that efficiently. We kept the "default" phase-space integrator: hep-ph/0208156
which is optimized on cases without interference term. Using it for pure interference term was everything but optimal but was shown to be successful in various simple case. So it might be that the phase-space integrator fails in that case.

A good test here would be to compute together the interference term + the quadratic term to see what that contributions matches.
In that case, the phase-space integration should be much more efficient.

This being said one critical issue for such comparison is the dynamical choice. Since madgraph use a Feynmam diagram based scale choice, it will depend of your process definition and the "full" will not use the same scale as the "sum" (especially since for interference by default we move back to simple HT/2 since the Feynmam diagram based one does not make sense).
So please check that you always use HT/2 (or something similar) otherwise you would need to make the comparison within theoretical uncertainties.

Cheers,

Olivier

Stefanie (stodt) said : #2

Dear Olivier,

Thanks a lot for this explanation. I understand that this decomposition approach has not been officially recommended. I'm still curious about a few more things, which I'm asking below.

I now tried to produce the same set of validation with the scale HT/2. You can find the distributions here:

https://cernbox.cern.ch/index.php/s/tXfV5gQwuOil3qF

Now the closure looks nice where the SM is dominant which is already very nice to see. Now I'm still wondering about your comment that the interference is probably not correct in our case. If I look into the mWW plot e.g. The discrepancy we see is in the very high tail, where there is almost no SM contribution left. So I'm wondering if the interference contribution can be large in this phase space at all? I also understand that we are going to really extreme regions of phase space here (especially at very high mWW beyond 3TeV an dup to 8TeV). Could this be a reason for the non-closure?

You are proposing to do the test and generate the interference and quadratic term together. How can I technically achieve this? By setting the NP^2=2, which should include NP^2==1 and NP^2==2?

I would have one more question. You are saying the interference term by default always picks the HT/2 dynamical scale. Would this also be true, if I plug in a custom scale, say mWW?

Thanks,
Stefanie

Hi Stefanie,

To be honest, I do not understand your plot.
Why the black dot disappear in the ratio plot? This is the only ratio plot that is interesting no?
Would it also be possible to have a plot of the interference? from the difference between "Full" and the amplitude squared. It seems that such interference should be negative for the large tail.
So could you plot the absolute value of the interference term (or plot it twice, while the second times it should be multiplied by -1) such that we can see that contribution.

>You are proposing to do the test and generate the interference and quadratic term together. How can I technically achieve this? By setting the NP^2=2, which should include NP^2==1 and NP^2==2?

You can use NP^2>0 for that

>I would have one more question. You are saying the interference term by default always picks the HT/2 dynamical scale. Would this also be true, if I plug in a custom scale, say mWW?

That's fine as well.

Cheers,

Olivier

Stefanie (stodt) said : #4

Hi Olivier,

Thanks for your reply.

The black dots are unfortunately hidden behind the blue ones as in the tails the only dominant term to the sum is the quadratic contribution (in blue). Regarding the interference, it it always negative. I have made plots where I plot it separately and for different scales in the link below (slide 10).

Here is a set of slides where I tested all different scale settings:

https://cernbox.cern.ch/index.php/s/dSfLUm63hjib9sz

On slide 3 you can see the generator level cross sections after event generation for all different scale choices and terms.
From your explanations we expected that the cross section for the interference term would be identical in case -1 and 3 (with identical seeds). This is not the case and we are wondering why this might be?

You can also find other comparison plots in these slides. Changing the scale unfortunately doesn't help the closure.

I'm sorry, I think I don't understand your reply to my last question. So, if I generate an interference only sample with scale setting 2. It will will take setting 2 or will change internally to setting 3 (HT/2)?

Thanks again for your help,
Stefanie

Hi,

> I'm sorry, I think I don't understand your reply to my last question.
> So, if I generate an interference only sample with scale setting 2. It
> will will take setting 2 or will change internally to setting 3 (HT/2)?

It will keep "2".
Same for "-1" if you write -1 in the run_card, it will use that definition. (but do not ask me what it is). It just that the default value written by the "output" command would not be -1 in that case but 3.

Your plot are quite interesting. I would really like to see NP^2 + INT plot.
To see if that one would be in agreement or not.
(another potentially interesting plot would be SM+INT, but it has less interest)

Cheers,

Olivier

> On 5 Sep 2019, at 14:23, Stefanie <email address hidden> wrote:
>
> Question #683400 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/683400
>
> Status: Answered => Open
>
> Stefanie is still having a problem:
> Hi Olivier,
>
> Thanks for your reply.
>
> The black dots are unfortunately hidden behind the blue ones as in the
> tails the only dominant term to the sum is the quadratic contribution
> (in blue). Regarding the interference, it it always negative. I have
> made plots where I plot it separately and for different scales in the
> link below (slide 10).
>
> Here is a set of slides where I tested all different scale settings:
>
> https://cernbox.cern.ch/index.php/s/dSfLUm63hjib9sz
>
> On slide 3 you can see the generator level cross sections after event generation for all different scale choices and terms.
>> From your explanations we expected that the cross section for the interference term would be identical in case -1 and 3 (with identical seeds). This is not the case and we are wondering why this might be?
>
> You can also find other comparison plots in these slides. Changing the
> scale unfortunately doesn't help the closure.
>
> I'm sorry, I think I don't understand your reply to my last question.
> So, if I generate an interference only sample with scale setting 2. It
> will will take setting 2 or will change internally to setting 3 (HT/2)?
>
> Thanks again for your help,
> Stefanie
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Stefanie (stodt) said : #6

Hi,

Thanks for your reply.

Then it will be consistent throughout our samples which is good. Even if it does not solve the problem yet.

I will try and make the QUAD+INT test as soon as possible and will post the outcome here. For the WZjj EW process we have tested this already. You can find the first numbers, the total cross sections at gridpack level here:

https://docs.google.com/spreadsheets/d/1XM1SSwLfmHqwjj78tnW5589EmoL1c0_DnKY3Ppgm_FM/edit?usp=sharing

It doesn't seem to be promising but I will produce numbers and plots as well.

Cheers,
Stefanie

Stefanie (stodt) said : #7

Hi Olivier,

I'm back with some more tests I made.

I have now tried to get the interference and quadratic contributions purely from full or SM samples. The thought was that the interference contribution would be half the difference between two full samples where one just swaps the sign of the parameter. In a formula as:

|INT| = 1/2 * |FULL_pos - FULL_neg|

where FULL_pos is a fully generated sample with a positive value and FULL-neg a fully generated sample with the same value put negative sign.

The same can be done with the quadratic term:

QUAD = 1/2 (FULL_pos + FULL_neg) - SM

I have tested this for the T0 and T1 operators and some resulting plots can be seen here:

https://cernbox.cern.ch/index.php/s/34nkQl6geSpKRl4

To us this really seems as if the quadratic term is not behaving properly. The differences in the interference term are also large in some bins but mostly within statistical uncertainties. And the interference contribution estimated from the full samples is also that small that we think, it cannot make up the differences of 10-15% we see between the SUM and the FULL. What do you think?

Cheers,
Stefanie

Stefanie (stodt) said : #8

Hi,

Again some more tests finished.

I have tried integrating the quadratic and interference contribution together. I have put the plots here:

https://cernbox.cern.ch/index.php/s/EkVHmGP3rBTjbtD

There is no big difference between the independent sum and the sum with the quadratic+interference integrated together.

As well, I have made the test with restricting the model with a restrict_XXX.dat card as described in this launchpad:

https://answers.launchpad.net/mg5amcnlo/+faq/2312

The plots are here:

https://cernbox.cern.ch/index.php/s/AA4FySTiujVcSVj

Comparing the plots with and without restrictions it seems that with the restriction the disagreement starts a bit later in mWW but it is still not gone.

These are all results of all tests I have made now. Does this somehow form a picture for you or do you have any idea what could go wrong?

Best,
Stefanie

My guess/understanding

1) the SM contribution is negligible in the tail
2) the intereference term is also negligible in the tail (as confirmed by the quad+int computation
3) the difference between "full" and "quad" is therefore related to how the tail is integrated.
(since we can forget the two other contributions in that part of the phase-space).

Do you generate such plot with a unique sample? or did you generate with multiple sample with cuts? Did you look at each log (for each channel of integration) and check if the cross-section for all channel was different of zero (and/or understood why they were 0)

Since you showed that you are dominated by the quadratic term, I guess that you should try to generate such sample with W onshell (and decay them with madspin).
doing it like this:
generate p p > w+ w+ j j QCD=0 ...
(and then handle the decay via MadSpin)
This will also allow you to set cut on the invariant Mass of the W pair, and have more control on the tail.

To be clear here, my worry is that the phase-space integration fails to correctly integrate the tail in the "full" case.

Cheers,

Olivier

PS: Out of curiosity, how do you justify theoritically such type of generation?
I guess that you assume that ALL dimension 6 operator are 0 (or at least extremely suppressed) but since you are not dominating by interference you are actually observing effect that are expected to be equivalent to an EFT with contribution from dimension 12. So how do you justify that you do not include the operator from dimension 10/12? Or in other word how do you justify that you can still be in a perturbative regime for the EFT?

> On 10 Sep 2019, at 09:42, Stefanie <email address hidden> wrote:
> i
> Question #683400 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/683400
>
> Stefanie gave more information on the question:
> Hi,
>
> Again some more tests finished.
>
> I have tried integrating the quadratic and interference contribution
> together. I have put the plots here:
>
> https://cernbox.cern.ch/index.php/s/EkVHmGP3rBTjbtD
>
> There is no big difference between the independent sum and the sum with
> the quadratic+interference integrated together.
>
> As well, I have made the test with restricting the model with a
> restrict_XXX.dat card as described in this launchpad:
>
> https://answers.launchpad.net/mg5amcnlo/+faq/2312
>
> The plots are here:
>
> https://cernbox.cern.ch/index.php/s/AA4FySTiujVcSVj
>
> Comparing the plots with and without restrictions it seems that with the
> restriction the disagreement starts a bit later in mWW but it is still
> not gone.
>
> These are all results of all tests I have made now. Does this somehow
> form a picture for you or do you have any idea what could go wrong?
>
> Best,
> Stefanie
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Stefanie (stodt) said : #10

Hi Olivier,

I agree with your understandings 1-3.

These plots are made with unique samples for each contribution. For all these tests I only generated the e+ mu+ v v d d final state. (But I also tested the full process in one sample which was even more severe in terms of discrepancies in the tails). So yes, I have only one gridpack/sample for FULL, INT, QUAD, and SM. None of which where zero (as can be seen in the plots, of course).

I have tried what you proposed with putting a cut on the invariant mass. I have altered the procedure slightly. I have put a cut on the variable mxx_min_pdg = {13: 500}, and mxx_only_part_antipart = False. Meaning I have generated mu+ mu+ v v d d with a 500GeV mll cut on the two muons. Here is the outcome:

https://cernbox.cern.ch/index.php/s/jOQPbypZ9dB8yhX

I guess this confirms your worry about the failing integration for the FULL process in the tails? We do conclude from this that the decomposed samples are doing the right thing, is this correct? If yes, we will change our strategy and try to rely on the decomposition for our final samples. (also because we don't want to put such a high invariant mass cut on our samples)

Concerning your last question: Of course, we have also thought about this and had long discussions. We are aware that in this regime the EFT might not be perturbative any more. However, the plots that you see in all above studies are at the current best limits for the ssWW process. This means our sensitivity is just not yet good enough to have the interference term being dominant. We know of this shortcoming and we will discuss this with our results that we will get in the end. The same argument holds for the dim6 operators. We are currently studying the impact of those in this final state but we will not be able to be sensitive enough to have any reasonable limits from this. Another point is, that I'm just not aware of any EFT model that can give us dimension 10/12 operators in VBS. And here again I would assume that the interference with the SM is really small, but that's only a guess. Another thing is that I am much in favor for clipping away these very high tails but we will also have to have inclusive limits to compare to other results that are publish. This is why we would need the closure also there.

Cheers,
Stefanie

This sounds reasonable.

Could I ask you to make a tarball of the directory that generates the events?
(the one without cuts for the FULL)
I would like to look at some advance log to understand what's going on

Cheers,

Olivier

> On 13 Sep 2019, at 19:47, Stefanie <email address hidden> wrote:
>
> Question #683400 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/683400
>
> Status: Answered => Open
>
> Stefanie is still having a problem:
> Hi Olivier,
>
> I agree with your understandings 1-3.
>
> These plots are made with unique samples for each contribution. For all
> these tests I only generated the e+ mu+ v v d d final state. (But I also
> tested the full process in one sample which was even more severe in
> terms of discrepancies in the tails). So yes, I have only one
> gridpack/sample for FULL, INT, QUAD, and SM. None of which where zero
> (as can be seen in the plots, of course).
>
> I have tried what you proposed with putting a cut on the invariant mass.
> I have altered the procedure slightly. I have put a cut on the variable
> mxx_min_pdg = {13: 500}, and mxx_only_part_antipart = False. Meaning I
> have generated mu+ mu+ v v d d with a 500GeV mll cut on the two muons.
> Here is the outcome:
>
> https://cernbox.cern.ch/index.php/s/jOQPbypZ9dB8yhX
>
> I guess this confirms your worry about the failing integration for the
> FULL process in the tails? We do conclude from this that the decomposed
> samples are doing the right thing, is this correct? If yes, we will
> change our strategy and try to rely on the decomposition for our final
> samples. (also because we don't want to put such a high invariant mass
> cut on our samples)
>
> Concerning your last question: Of course, we have also thought about
> this and had long discussions. We are aware that in this regime the EFT
> might not be perturbative any more. However, the plots that you see in
> all above studies are at the current best limits for the ssWW process.
> This means our sensitivity is just not yet good enough to have the
> interference term being dominant. We know of this shortcoming and we
> will discuss this with our results that we will get in the end. The same
> argument holds for the dim6 operators. We are currently studying the
> impact of those in this final state but we will not be able to be
> sensitive enough to have any reasonable limits from this. Another point
> is, that I'm just not aware of any EFT model that can give us dimension
> 10/12 operators in VBS. And here again I would assume that the
> interference with the SM is really small, but that's only a guess.
> Another thing is that I am much in favor for clipping away these very
> high tails but we will also have to have inclusive limits to compare to
> other results that are publish. This is why we would need the closure
> also there.
>
> Cheers,
> Stefanie
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Stefanie (stodt) said : #12

Thanks a lot for all your help and for looking into this further. I will prepare a tarball with everything that I produced for one FULL run.

Best,
Stefanie

Stefanie (stodt) said : #13

Thanks Olivier Mattelaer, that solved my question.