Output from MadAnalysis Fastjet .lhco and .lhe

Asked by Jasper on 2017-03-16

Hi there,

I am using Madanalysis to cluster pythia hep files produced by madgraph with fastjet in reconstruction mode. I need to root the output to create a datafile with the kinematic properties of leptons per event (for machine learning purposes). I normally process hep files from MadGraph to create data where each line contains the properties of leptons (Px,Py,Pz) from one event.

I would prefer to use lhe or hep data types to do this, because then particle mother information is preserved. I have found that I can get output.lhe and output.lhco files. However, those .lhe files are "simplified" and can therefor not be rooted with ExRootAnalysis (I get: ERROR: invalid event format) . Do you know how I can root these "simplified" files?

I could also use the .lhco output, as that can be rooted with ExRootAnalysis. However, I am less familiar with this file type. I only recognize is at delphes output, so after detector simulation. In my Madanalysis, delphes is disabled, so I am not sure how I should interpret this output. Is it what a detector could measure if it were perfect?

Finally, I wonder about the ROOT output that should be created according to (https://arxiv.org/pdf/1206.1599.pdf (p.59)). I cannot find I anywhere, and wonder if it was replaced with the .saf file I do find. Is there anyway to get this file and would it be useful for what I am trying to do? I also found something about a TheMouth.root file that could be created as output, but I only found it in context of delphes simulation using MadAnalysis, and I rather not use delphes, because it discards to much information I might need.

Thanks a lot!

Jasper

Question information

Language:
English Edit question
Status:
Answered
For:
MadAnalysis 5 Edit question
Assignee:
No assignee Edit question
Last query:
2017-04-05
Last reply:
2017-04-05

This question was reopened

Benjamin Fuks (fuks) said : #1

Hi Jasper,

The LHE format outputted by MA5 is its own LHE-like format, so that it is not surprizing other programs cannot read it. You could instead directly output an LHCO format that should be readable by ExRootAnalysis. If you really want to use the LHE output of MA5, you need to design your own root translator.

The root output of ma5 is the one connected to delphes 3. You need to internally install delphes within ma5 to enable it. Note that in principle, you have more information in the dephes output than in an LHE or LHCO output. I therefore do not understand your statement.

Cheers,

Benjamin

Jasper (jvdk) said : #3

Hi Benjamin,

Thank you for your help! I think I will use the LHCO format output. So is it true that the LHCO file outputted by MA5 is the sort of a perfect detector output? So Delphes would add all kinds of efficiencies and regions where it cannot measure etc? This was what I meant with that Delphes gives less information. It adds a level of complexity to the event which I don't want (yet) for training my machine learning algorithm.

Thanks again,
Jasper

Benjamin Fuks (fuks) said : #4

Hi Jasper,

> Thank you for your help! I think I will use the LHCO format output. So
> is it true that the LHCO file outputted by MA5 is the sort of a perfect
> detector output?
It depends if you turn on b-tagging efficiencies / mistagging rates, etc. in the ma5 setup at runtime. But assuming you don't, it is.

> So Delphes would add all kinds of efficiencies and
> regions where it cannot measure etc? This was what I meant with that
> Delphes gives less information. It adds a level of complexity to the
> event which I don't want (yet) for training my machine learning
> algorithm.
In principle, you can tune the delphes cards and set all of these two 100%.

Cheers,

Benjamin

Jasper (jvdk) said : #5

Thanks Benjamin Fuks, that solved my question.

Jasper (jvdk) said : #6

Hi,

So I have know been using Delphes in MadAnalysis5 to create TheMouth.root files from my pythia.hep files. These files are created with MadGraph for several different background processes. The total number of generated events is for one of these processes for example is 199492, and normalized to luminocity there are 8740023 events. Now I want to use the root file to extract momentum information for events where three leptons were produced. So, I want a data file where each row is and event and each column is a property of that events. So for example row 1 column 1 could be the PT of the first lepton from the first event. I do this by analyzing the root file. When applying the three lepton cut (N(l)=3) in MA5 I get 17086 events. Now, I believe that I can only get event specific information from the 199492 generated events. This would mean that I should be able to get event specific information from (199492/8740023)*17086=390 three lepton events right? However, this is not what I observe when I analyze the root file. I get 0 events where three leptons are produced, all events have N(l) < 3. I'm really curious to find out what I am doing wrong. Assuming I am doing the root analysis correctly, where do I make a mistake or wrong assumption?

I hope it is clear what I am trying to do, if not I am more than happy to elaborate.

Cheers,
Jasper

Benjamin Fuks (fuks) said : #7

Hi Jasper,

I am not sure to follow. Please correct me if I am wrong. You first apply a N(l)==3 cut in ma5 and get 390 events passing this cut (let's remove the normalization for simplicity). However, when you analyse the root file otherwise (do you mind defining this 'otherwise'), you are getting 0 event pasing the cut. Is that correct?

Cheers,

Benjamin

> On 24 Mar 2017, at 11:48 , Jasper <email address hidden> wrote:
>
> Question #566049 on MadAnalysis 5 changed:
> https://answers.launchpad.net/madanalysis5/+question/566049
>
> Status: Solved => Open
>
> Jasper is still having a problem:
> Hi,
>
> So I have know been using Delphes in MadAnalysis5 to create
> TheMouth.root files from my pythia.hep files. These files are created
> with MadGraph for several different background processes. The total
> number of generated events is for one of these processes for example is
> 199492, and normalized to luminocity there are 8740023 events. Now I
> want to use the root file to extract momentum information for events
> where three leptons were produced. So, I want a data file where each row
> is and event and each column is a property of that events. So for
> example row 1 column 1 could be the PT of the first lepton from the
> first event. I do this by analyzing the root file. When applying the
> three lepton cut (N(l)=3) in MA5 I get 17086 events. Now, I believe that
> I can only get event specific information from the 199492 generated
> events. This would mean that I should be able to get event specific
> information from (199492/8740023)*17086=390 three lepton events right?
> However, this is not what I observe when I analyze the root file. I get
> 0 events where three leptons are produced, all events have N(l) < 3. I'm
> really curious to find out what I am doing wrong. Assuming I am doing
> the root analysis correctly, where do I make a mistake or wrong
> assumption?
>
> I hope it is clear what I am trying to do, if not I am more than happy
> to elaborate.
>
> Cheers,
> Jasper
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

Jasper (jvdk) said : #8

Hi Benjamin,

Yes that is true. I use a c++ script to analyze it. The general structure of the script looks like:

[
A bunch c++ related stuff

TFile *pythia_file = TFile::Open("/vol/astro2/users/jkolk/MG5/HEPTools/madanalysis5/madanalysis5/bin/ttbar/Output/_defaultset/TheMouth.root");

Then a bunch of definitions from Delphes.h

Then I loop over all events and ask if (Electron_Size + Muon_Size ==3). This is False in all events.
]

I hope this clarifies what I am doing. The exact program is not really important I guess, the biggest question is I think if I can analyze the outputted root file in such a way.

Cheers,
Jasper

Benjamin Fuks (fuks) said : #9

Hi Jasper,

The output delphes format from ma5 is slightly different from the one of the standard delphes. The only thing that comes to my mind is that oyu are looking to the wrong collections. Try the collections with 'MA5' in their names.

Regards,

Benjamin

> On 24 Mar 2017, at 12:58 , Jasper <email address hidden> wrote:
>
> Question #566049 on MadAnalysis 5 changed:
> https://answers.launchpad.net/madanalysis5/+question/566049
>
> Status: Answered => Open
>
> Jasper is still having a problem:
> Hi Benjamin,
>
> Yes that is true. I use a c++ script to analyze it. The general
> structure of the script looks like:
>
> [
> A bunch c++ related stuff
>
> TFile *pythia_file =
> TFile::Open("/vol/astro2/users/jkolk/MG5/HEPTools/madanalysis5/madanalysis5/bin/ttbar/Output/_defaultset/TheMouth.root");
>
> Then a bunch of definitions from Delphes.h
>
> Then I loop over all events and ask if (Electron_Size + Muon_Size ==3). This is False in all events.
> ]
>
> I hope this clarifies what I am doing. The exact program is not really
> important I guess, the biggest question is I think if I can analyze the
> outputted root file in such a way.
>
> Cheers,
> Jasper
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

Jasper (jvdk) said : #10

Hi there,

So, we solved the problems above but now something strange is happening. If my supervisor and I try the same analysis:

set main.fastsim.package = delphes
set main.fastsim.detector = atlas
set main.fastsim.pileup = none
set main.fastsim.output = true
import <hep file>
define l = mu+ mu- e+ e-
select N(l) ==3
submit <dirname>

We get different amount of 3lep events (111618 vs 32905) in the PDF output. We are sure we are using the same input file and that we inputted the same command. We are both using MA5 in reconstruction mode. If we root the outputted TheMouth file, we get different Delphes.h files as well.

Do you have any idea where these discrepancy could come from?

Thanks!
Jasper

Benjamin Fuks (fuks) said : #11

Hi Jasper,

That's surprizing and not understandable. Can both you and supervisor update the delphes installation. Maybe is one of you using an older verison. Another test would be to try ma5 v1.6beta (to bet obtained from launchpad).

Regards,

Benjamin

> On 5 Apr 2017, at 15:28 , Jasper <email address hidden> wrote:
>
> Question #566049 on MadAnalysis 5 changed:
> https://answers.launchpad.net/madanalysis5/+question/566049
>
> Status: Answered => Open
>
> Jasper is still having a problem:
> Hi there,
>
> So, we solved the problems above but now something strange is happening.
> If my supervisor and I try the same analysis:
>
> set main.fastsim.package = delphes
> set main.fastsim.detector = atlas
> set main.fastsim.pileup = none
> set main.fastsim.output = true
> import <hep file>
> define l = mu+ mu- e+ e-
> select N(l) ==3
> submit <dirname>
>
> We get different amount of 3lep events (111618 vs 32905) in the PDF output. We are sure we are using the same input file and that we inputted the same command. We are both using MA5 in reconstruction mode. If we root the outputted TheMouth file, we get different Delphes.h files as well.
>
> Do you have any idea where these discrepancy could come from?
>
> Thanks!
> Jasper
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

Jasper (jvdk) said : #12

Hi Benjamin,

I installed MA5 in MG5 using mg5 > install MadAnalysis. Now I see my version of MA5 is 1.6.6. Do I understand correctly that this is then the beta version? Could it be that the problem stems from this?

My supervisor does have older versions of MA5 (1.2) and Delphes (3.2.0). My Delphes version is 3.3.3. Do you think it likely that this can be the cause? Because we actually think that physically her answer is more likely.

Thanks!
Jasper

Benjamin Fuks (fuks) said : #13

Hi Jasper,

The delphes cards may be different, and there were also bug fixes in delphes. I would recommend that you make sure you are comparing apples to apples. I would also recommend your supervisor to upgrade to at least v1.5. We also fixed several issues in ma5.

Regards,

Benjamin

> On 5 Apr 2017, at 17:13 , Jasper <email address hidden> wrote:
>
> Question #566049 on MadAnalysis 5 changed:
> https://answers.launchpad.net/madanalysis5/+question/566049
>
> Status: Answered => Open
>
> Jasper is still having a problem:
> Hi Benjamin,
>
> I installed MA5 in MG5 using mg5 > install MadAnalysis. Now I see my
> version of MA5 is 1.6.6. Do I understand correctly that this is then the
> beta version? Could it be that the problem stems from this?
>
> My supervisor does have older versions of MA5 (1.2) and Delphes (3.2.0).
> My Delphes version is 3.3.3. Do you think it likely that this can be the
> cause? Because we actually think that physically her answer is more
> likely.
>
> Thanks!
> Jasper
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

Can you help with this problem?

Provide an answer of your own, or ask Jasper for more information if necessary.

To post a message you must log in.