MadAnalysis 5

Efficiency change for each run in PADForMA5tune

Asked by pattara angkinun

Dear Madanalysis5 Team,

I am using the PADForMA5tune to calculate exclusion limit for my model but I noticed that the efficiency, which is the number of event that pass the cut / total number of event, change when I rerun it (with the same signal file). As a result the exclusion limit change for each run. Is this an expected results? and why? It seem to me that applying the same cut to the same events should give the same result every time.

Best regards,
Pattara Angkinun

Question information

Language:
English Edit question
Status:
Solved
For:
MadAnalysis 5 Edit question
Assignee:
No assignee Edit question
Solved by:
Benjamin Fuks
Solved:
Last query:
Last reply:
Revision history for this message
Best Benjamin Fuks (fuks) said : 		#1

Hi,

The simulation of the detector involves random numbers so that the results change at each run. The difference should however be in the error bars. if not, yo may need to generate more events.

Cheers,

Benjamin

> On 2 Apr 2019, at 14:52 , pattara angkinun <email address hidden> wrote:
>
> New question #679721 on MadAnalysis 5:
> https://answers.launchpad.net/madanalysis5/+question/679721
>
> Dear Madanalysis5 Team,
>
> I am using the PADForMA5tune to calculate exclusion limit for my model but I noticed that the efficiency, which is the number of event that pass the cut / total number of event, change when I rerun it (with the same signal file). As a result the exclusion limit change for each run. Is this an expected results? and why? It seem to me that applying the same cut to the same events should give the same result every time.
>
> Best regards,
> Pattara Angkinun
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

Revision history for this message
pattara angkinun (pattara.angkinun) said : 		#2

Thanks Benjamin Fuks, that solved my question.

Revision history for this message
pattara angkinun (pattara.angkinun) said : 		#3

Dear Benjamin,

I have a few more question related to this topic. Firstly, I check the cutflow file and notice that the efficiency reported in the CLs_output.dat is not the same as counter of the last cut / initial counter, so how do you get the efficiency in the CLs_output.dat. Secondly, what is the meaning of statistical uncertainty reported in the CLs_output.dat.

Best regards,
Pattara Angkinun

Revision history for this message
Benjamin Fuks (fuks) said : 		#4

HI Pattara,

> Firstly, I check the
> cutflow file and notice that the efficiency reported in the
> CLs_output.dat is not the same as counter of the last cut / initial
> counter, so how do you get the efficiency in the CLs_output.dat.
This can't be. Can you please show me both files?

> Secondly, what is the meaning of statistical uncertainty reported in the
> CLs_output.dat.
That is the statistical uncertainty computed from the number of inputted events, their weights and a binomial law for the uncertainties associated with each cut.

Cheers,

Benjamin

Revision history for this message
pattara angkinun (pattara.angkinun) said : 		#5

Dear Benjamin,
here is one of an output folder: https://www.dropbox.com/sh/5w9lljns3i9gryi/AACB_pslCC8l1s3ZLQ1I5sGSa?dl=0

You can see for example the file "Stop_to_T_neutralino__LowDeltaM__MET_greater_than_150.saf", the initial counter is 19,998 and the last counter is 999. So, the efficiency should be 0.04995, but it is shown as 0.0375362 in the CLs_output.dat.

Best regards,
Pattara Angkinun

Revision history for this message
Benjamin Fuks (fuks) said : 		#6

Hi Pattara,

The results you sent me are and consistent. This analysis changes the event weights to account for trigger efficiencies. The good numbers to use are thus not the number of entries, but the sum of the weights. In this case, you get 0.037 as the efficiency.

Cheers,

Benjamin

> On 28 May 2019, at 24:47 , pattara angkinun <email address hidden> wrote:
>
> Question #679721 on MadAnalysis 5 changed:
> https://answers.launchpad.net/madanalysis5/+question/679721
>
> pattara angkinun posted a new comment:
> Dear Benjamin,
> here is one of an output folder: https://www.dropbox.com/sh/5w9lljns3i9gryi/AACB_pslCC8l1s3ZLQ1I5sGSa?dl=0
>
> You can see for example the file
> "Stop_to_T_neutralino__LowDeltaM__MET_greater_than_150.saf", the initial
> counter is 19,998 and the last counter is 999. So, the efficiency should
> be 0.04995, but it is shown as 0.0375362 in the CLs_output.dat.
>
> Best regards,
> Pattara Angkinun
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

Revision history for this message
pattara angkinun (pattara.angkinun) said : 		#7

Dear Benjamin,

thanks a lot for the answer.

Best regards,
Pattara Angkinun

Revision history for this message
pattara angkinun (pattara.angkinun) said : 		#8

Dear Benjamin,

sorry to bother you again but I still couldn't understand about the trigger in CMS_SUS_13_011. I look in the c++ code for it and I saw that the trigger cut is just "Manager()->ApplyCut(true, "trigger");", so doesn't it mean that every event will pass this cut?
If so, why do I have a change in entries after the trigger cut since the reweighted part should not change it. The example can be seen in the dropbox that I attach above.

Best regards,
Pattara Angkinun

Revision history for this message
Benjamin Fuks (fuks) said : 		#9

Hi Pattara,

The event weight is changed right before that cut, so that whlist all events pass the cut, their weight is changed. As a result, the entry in the cutflow has changed. This is explained in more detail in the comment preceeding the trigger cut. I hope this helps.

Regards,

Benjamin

> On 24 Jun 2019, at 18:46 , pattara angkinun <email address hidden> wrote:
>
> Question #679721 on MadAnalysis 5 changed:
> https://answers.launchpad.net/madanalysis5/+question/679721
>
> pattara angkinun posted a new comment:
> Dear Benjamin,
>
> sorry to bother you again but I still couldn't understand about the trigger in CMS_SUS_13_011. I look in the c++ code for it and I saw that the trigger cut is just "Manager()->ApplyCut(true, "trigger");", so doesn't it mean that every event will pass this cut?
> If so, why do I have a change in entries after the trigger cut since the reweighted part should not change it. The example can be seen in the dropbox that I attach above.
>
> Best regards,
> Pattara Angkinun
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.