Wrong number of events when loading hepmc file

Asked by CG on 2019-07-03

Dear MadAnalysis Team,

I have a problem when trying to analyse events at particle level using a hepmc.gz file for input.
I used MadGraph5_aMC to generate 10000 events and then showered them using Pythia8.

When I use the LHE files from MadGraph output (so parton level), MA5 seems to correctly import these samples and
shows the correct number of 10000 events. When applying cuts, the sum of events kept plus the number of rejected events (after the first cut) equals the number of total events times the cross section, due to MA5 normalizing.

But if I try to use the particle level Pythia8 output (hepmc.gz), MA5 says that there are 10023 events in the file
and the number of kept and rejected events after the first cut sum to something strange, like 27.5 events.
For a cross check, I converted the .hepmc to .root (using Delphes converter card) and
inspected that file with ROOT and checked that it contains the right number of 10000 events.

However, when running MA5 with the hepmc.gz input, I get some warnings like "HEPMC linecode unknown", please see the log file below.

It would be very nice if you could help me with this issue.

Cheers,
Chris

~/Software/madanalysis5$ python bin/ma5 -R analyse3.txt
MA5:
MA5: *************************************************************
MA5: * *
MA5: * W E L C O M E to M A D A N A L Y S I S 5 *
MA5: * ______ ______ *
MA5: * /'\_/`\/\ __ \/\ ___\ *
MA5: * /\ \ \ \_\ \ \ \__/ *
MA5: * \ \ \__\ \ \ __ \ \___``\ *
MA5: * \ \ \_/\ \ \ \/\ \/\ \_\ \ *
MA5: * \ \_\\ \_\ \_\ \_\ \____/ *
MA5: * \/_/ \/_/\/_/\/_/\/___/ *
MA5: * *
MA5: * MA5 release : 1.7.21 2019/02/07 *
MA5: * *
MA5: * Comput. Phys. Commun. 184 (2013) 222-256 *
MA5: * Eur. Phys. J. C74 (2014) 3103 *
MA5: * *
MA5: * The MadAnalysis Development Team - Please visit us at *
MA5: * https://launchpad.net/madanalysis5 *
MA5: * *
MA5: * Type 'help' for in-line help. *
MA5: * *
MA5: *************************************************************
MA5: Platform: Linux 4.4.0-143-generic [Linux mode]
MA5: Reading user settings ...
MA5: Checking mandatory packages:
MA5: - Python [OK]
MA5: - GNU GCC g++ [OK]
MA5: - GNU Make [OK]
MA5: Checking optional packages devoted to data processing:
MA5: - Zlib [OK]
MA5: - FastJet [OK]
MA5: - Root [OK]
MA5: - Delphes [DISABLED]
MA5: - Delphes-MA5tune [DISABLED]
MA5: Checking the MadAnalysis 5 core library:
MA5: => MadAnalysis libraries found.
MA5: => MadAnalysis test program works.
MA5: Checking optional packages devoted to reinterpretation:
MA5: - SciPy [OK]
MA5: - PAD [DISABLED]
MA5: - PADForMA5tune [DISABLED]
MA5: Checking optional packages devoted to histogramming:
MA5: - Root [OK]
MA5: - Matplotlib [OK]
MA5: - pdflatex [OK]
MA5: - latex [OK]
MA5: Package used for graphical rendering: Root
MA5: *************************************************************
MA5: Particle labels from input/reco_default.txt
MA5: => 14 particles successfully exported.
MA5: Multiparticle labels from madanalysis/input/multiparticles_reco_default.txt
MA5: => 9 multiparticles successfully exported.
MA5: => 9 multiparticles successfully exported.
MA5: Executing the commands from the script
MA5: /home/c/Software/madanalysis5/analyse3.txt...
MA5: ma5>set main.fastsim.package=fastjet
MA5: ma5>set main.fastsim.algorithm=antikt
MA5: ma5>set main.fastsim.radius=0.4
MA5: ma5>import /home/c/storage/events/tag_1_pythia8_events.hepmc.gz
MA5: -> Storing the file 'tag_1_pythia8_events.hepmc.gz' in the dataset 'defaultset'.
MA5: ma5>plot N(a)
MA5: ma5>submit
MA5: Creating folder 'ANALYSIS_30'...
MA5: Copying 'SampleAnalyzer' source files...
MA5: Inserting your selection into 'SampleAnalyzer'...
MA5: Writing the list of datasets...
MA5: Writing the command line history...
MA5: Creating Makefiles...
MA5: Compiling 'SampleAnalyzer'...
MA5: Linking 'SampleAnalyzer'...
MA5: Running 'SampleAnalyzer' over dataset 'defaultset'...
MA5: *******************************************************
    * SampleAnalyzer for MadAnalysis 5 - Welcome.
    * Initializing all components
      - version: 1.7.21 (2019/02/07)
      - general: everything is default.
      - extracting the list of event samples...
      - analyzer 'MadAnalysis5job'
      - jet clusterer 'anti kt'
        with algo: R=0.4 ; p=-1 ; PTmin=5
        with bjet: dR=0.5 ; Exclusive ; IDeff=1 ; MisID(c)=0 ; MisID(q)=0
        with tau: IDeff=1 ; MisID(q)=0
    * Running over files ...
    * 1/1 /home/c/storage/events/tag_1_pythia8_events.hepmc.gz
        => file size: 842.84 Mo
        => sample format: HEPMC file produced by an unknown generator (cross section assumed in pb).
        => progress: [> ]
#--------------------------------------------------------------------------
# FastJet release 3.3.2
# M. Cacciari, G.P. Salam and G. Soyez
# A software package for jet finding and analysis at colliders
# http://fastjet.fr
#
# Please cite EPJC72(2012)1896 [arXiv:1111.6097] if you use this package
# for scientific work and optionally PLB641(2006)57 [hep-ph/0512210].
#
# FastJet is provided without warranty under the terms of the GNU GPLv2.
# It uses T. Chan's closest pair algorithm, S. Fortune's Voronoi code
# and 3rd party plugin jet algorithms. See COPYING file for details.
#--------------------------------------------------------------------------

        => progress: [=> ]
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------

        => progress: [===> ]
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------

        => progress: [====> ]
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------

        => progress: [======> ]
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------

        => progress: [=======> ]
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------
WARNING: --------------------------------------------------------------------------------
WARNING: Msg | HEPMC linecode unknown
WARNING: Details |
WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
WARNING: --------------------------------------------------------------------------------

        => progress: [===================================]
        => total number of events: 10023 ( analyzed: 10023 ; skipped: 0 )
    * Finalizing all components ...
    * Total number of processed events: 10023.
+----------------------------------------------------------------------------------------------------------------------+
| LogReport-Warning |
+----------------------------------------------------------------------------------------------------------------------+
| Message NIterations @ File Line |
|----------------------------------------------------------------------------------------------------------------------|
| HEPMC linecode unknown 44 Reader/HEPMCReader.cpp 331 |
+----------------------------------------------------------------------------------------------------------------------+
    * Goodbye.
MA5: *******************************************************
MA5: Checking SampleAnalyzer output...
MA5: Extracting data from the output files...
MA5: Preparing data for the reports ...
MA5: Generating all plots ...
MA5: Generating the HMTL report ...
MA5: -> To open this HTML report, please type 'open'.
MA5: Generating the PDF report ...
MA5: -> To open this PDF report, please type 'open ANALYSIS_30/Output/PDF/MadAnalysis5job_0'.
MA5: Generating the DVI report ...
MA5: Well done! Elapsed time = 128 seconds
MA5: ma5>open

Question information

Language:
English Edit question
Status:
Solved
For:
MadAnalysis 5 Edit question
Assignee:
No assignee Edit question
Solved by:
CG
Solved:
2019-07-08
Last query:
2019-07-08
Last reply:
2019-07-05
Benjamin Fuks (fuks) said : #1

Hi Chris,

This sounds like a bug. Can you please generate and share an hepmc sample of a reasonable size (maybe 500 events) in which the problem arises, so that I could have a look and provide a fix (or at leat to understand what is going on). Thanks in advance.

Cheers,

Benjamin

> On 3 Jul 2019, at 15:27 , CG <email address hidden> wrote:
>
> New question #681774 on MadAnalysis 5:
> https://answers.launchpad.net/madanalysis5/+question/681774
>
> Dear MadAnalysis Team,
>
> I have a problem when trying to analyse events at particle level using a hepmc.gz file for input.
> I used MadGraph5_aMC to generate 10000 events and then showered them using Pythia8.
>
> When I use the LHE files from MadGraph output (so parton level), MA5 seems to correctly import these samples and
> shows the correct number of 10000 events. When applying cuts, the sum of events kept plus the number of rejected events (after the first cut) equals the number of total events times the cross section, due to MA5 normalizing.
>
> But if I try to use the particle level Pythia8 output (hepmc.gz), MA5 says that there are 10023 events in the file
> and the number of kept and rejected events after the first cut sum to something strange, like 27.5 events.
> For a cross check, I converted the .hepmc to .root (using Delphes converter card) and
> inspected that file with ROOT and checked that it contains the right number of 10000 events.
>
> However, when running MA5 with the hepmc.gz input, I get some warnings like "HEPMC linecode unknown", please see the log file below.
>
> It would be very nice if you could help me with this issue.
>
> Cheers,
> Chris
>
>
>
>
>
> ~/Software/madanalysis5$ python bin/ma5 -R analyse3.txt
> MA5:
> MA5: *************************************************************
> MA5: * *
> MA5: * W E L C O M E to M A D A N A L Y S I S 5 *
> MA5: * ______ ______ *
> MA5: * /'\_/`\/\ __ \/\ ___\ *
> MA5: * /\ \ \ \_\ \ \ \__/ *
> MA5: * \ \ \__\ \ \ __ \ \___``\ *
> MA5: * \ \ \_/\ \ \ \/\ \/\ \_\ \ *
> MA5: * \ \_\\ \_\ \_\ \_\ \____/ *
> MA5: * \/_/ \/_/\/_/\/_/\/___/ *
> MA5: * *
> MA5: * MA5 release : 1.7.21 2019/02/07 *
> MA5: * *
> MA5: * Comput. Phys. Commun. 184 (2013) 222-256 *
> MA5: * Eur. Phys. J. C74 (2014) 3103 *
> MA5: * *
> MA5: * The MadAnalysis Development Team - Please visit us at *
> MA5: * https://launchpad.net/madanalysis5 *
> MA5: * *
> MA5: * Type 'help' for in-line help. *
> MA5: * *
> MA5: *************************************************************
> MA5: Platform: Linux 4.4.0-143-generic [Linux mode]
> MA5: Reading user settings ...
> MA5: Checking mandatory packages:
> MA5: - Python [OK]
> MA5: - GNU GCC g++ [OK]
> MA5: - GNU Make [OK]
> MA5: Checking optional packages devoted to data processing:
> MA5: - Zlib [OK]
> MA5: - FastJet [OK]
> MA5: - Root [OK]
> MA5: - Delphes [DISABLED]
> MA5: - Delphes-MA5tune [DISABLED]
> MA5: Checking the MadAnalysis 5 core library:
> MA5: => MadAnalysis libraries found.
> MA5: => MadAnalysis test program works.
> MA5: Checking optional packages devoted to reinterpretation:
> MA5: - SciPy [OK]
> MA5: - PAD [DISABLED]
> MA5: - PADForMA5tune [DISABLED]
> MA5: Checking optional packages devoted to histogramming:
> MA5: - Root [OK]
> MA5: - Matplotlib [OK]
> MA5: - pdflatex [OK]
> MA5: - latex [OK]
> MA5: Package used for graphical rendering: Root
> MA5: *************************************************************
> MA5: Particle labels from input/reco_default.txt
> MA5: => 14 particles successfully exported.
> MA5: Multiparticle labels from madanalysis/input/multiparticles_reco_default.txt
> MA5: => 9 multiparticles successfully exported.
> MA5: => 9 multiparticles successfully exported.
> MA5: Executing the commands from the script
> MA5: /home/c/Software/madanalysis5/analyse3.txt...
> MA5: ma5>set main.fastsim.package=fastjet
> MA5: ma5>set main.fastsim.algorithm=antikt
> MA5: ma5>set main.fastsim.radius=0.4
> MA5: ma5>import /home/c/storage/events/tag_1_pythia8_events.hepmc.gz
> MA5: -> Storing the file 'tag_1_pythia8_events.hepmc.gz' in the dataset 'defaultset'.
> MA5: ma5>plot N(a)
> MA5: ma5>submit
> MA5: Creating folder 'ANALYSIS_30'...
> MA5: Copying 'SampleAnalyzer' source files...
> MA5: Inserting your selection into 'SampleAnalyzer'...
> MA5: Writing the list of datasets...
> MA5: Writing the command line history...
> MA5: Creating Makefiles...
> MA5: Compiling 'SampleAnalyzer'...
> MA5: Linking 'SampleAnalyzer'...
> MA5: Running 'SampleAnalyzer' over dataset 'defaultset'...
> MA5: *******************************************************
> * SampleAnalyzer for MadAnalysis 5 - Welcome.
> * Initializing all components
> - version: 1.7.21 (2019/02/07)
> - general: everything is default.
> - extracting the list of event samples...
> - analyzer 'MadAnalysis5job'
> - jet clusterer 'anti kt'
> with algo: R=0.4 ; p=-1 ; PTmin=5
> with bjet: dR=0.5 ; Exclusive ; IDeff=1 ; MisID(c)=0 ; MisID(q)=0
> with tau: IDeff=1 ; MisID(q)=0
> * Running over files ...
> * 1/1 /home/c/storage/events/tag_1_pythia8_events.hepmc.gz
> => file size: 842.84 Mo
> => sample format: HEPMC file produced by an unknown generator (cross section assumed in pb).
> => progress: [> ]
> #--------------------------------------------------------------------------
> # FastJet release 3.3.2
> # M. Cacciari, G.P. Salam and G. Soyez
> # A software package for jet finding and analysis at colliders
> # http://fastjet.fr
> #
> # Please cite EPJC72(2012)1896 [arXiv:1111.6097] if you use this package
> # for scientific work and optionally PLB641(2006)57 [hep-ph/0512210].
> #
> # FastJet is provided without warranty under the terms of the GNU GPLv2.
> # It uses T. Chan's closest pair algorithm, S. Fortune's Voronoi code
> # and 3rd party plugin jet algorithms. See COPYING file for details.
> #--------------------------------------------------------------------------
>
> => progress: [=> ]
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
>
> => progress: [===> ]
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
>
> => progress: [====> ]
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
>
> => progress: [======> ]
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
>
> => progress: [=======> ]
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
> WARNING: --------------------------------------------------------------------------------
> WARNING: Msg | HEPMC linecode unknown
> WARNING: Details |
> WARNING: Where | Function = FillEvent ; File=Reader/HEPMCReader.cpp ; Line=331
> WARNING: --------------------------------------------------------------------------------
>
> => progress: [===================================]
> => total number of events: 10023 ( analyzed: 10023 ; skipped: 0 )
> * Finalizing all components ...
> * Total number of processed events: 10023.
> +----------------------------------------------------------------------------------------------------------------------+
> | LogReport-Warning |
> +----------------------------------------------------------------------------------------------------------------------+
> | Message NIterations @ File Line |
> |----------------------------------------------------------------------------------------------------------------------|
> | HEPMC linecode unknown 44 Reader/HEPMCReader.cpp 331 |
> +----------------------------------------------------------------------------------------------------------------------+
> * Goodbye.
> MA5: *******************************************************
> MA5: Checking SampleAnalyzer output...
> MA5: Extracting data from the output files...
> MA5: Preparing data for the reports ...
> MA5: Generating all plots ...
> MA5: Generating the HMTL report ...
> MA5: -> To open this HTML report, please type 'open'.
> MA5: Generating the PDF report ...
> MA5: -> To open this PDF report, please type 'open ANALYSIS_30/Output/PDF/MadAnalysis5job_0'.
> MA5: Generating the DVI report ...
> MA5: Well done! Elapsed time = 128 seconds
> MA5: ma5>open
>
>
>
> --
> You received this question notification because you are an answer
> contact for MadAnalysis 5.

CG (cgwd) said : #2

Hello Benjamin,

I just re-run the showering with pythia8 on a different machine and now it seems that MA5 reads the events correctly.
So it seems there was a problem with my pythia8 installation.

But thanks anyway for your help.