multi_run mode cann't work

Asked by karo

Hello,

       When I worked in multi_run mode(bin/madevent), it cann't work well. Delphes.lhco file will have 0 events. Pythia seems to work well.
       But if i work in single run mode (bin/generate_events), MG5/Pythia/Delphes work very well.
       I don't know why.

Cheers,
Karo

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
Pavel Demin Edit question
Last query:
Last reply:
Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#1

Hi Karo,

Weird, those are the same routines which are used in both case.
Did you look at the pythia/pgs log to see why they fail to run correctly?

Cheers,

Olivier

On Nov 27, 2013, at 4:21 PM, karo <email address hidden> wrote:

> New question #239917 on MadGraph5:
> https://answers.launchpad.net/madgraph5/+question/239917
>
> Hello,
>
> When I worked in multi_run mode(bin/madevent), it cann't work well. Delphes.lhco file will have 0 events. Pythia seems to work well.
> But if i work in single run mode (bin/generate_events), MG5/Pythia/Delphes work very well.
> I don't know why.
>
>
> Cheers,
> Karo
>
> --
> You received this question notification because you are a member of
> MadTeam, which is an answer contact for MadGraph5.

Revision history for this message
karo (karo-t) said :
#2

Hi Olivier,

    The pythia log is correct. But there are error in delphes log.

   ** Reading pythia_events.hep
** [----------------------------------------------------------------] (0.00%)
** ERROR: Inconsistent size of arrays. File is probably corrupted.
** Reading delphes.root
Error in <TChain::LoadTree>: Cannot find tree with name Delphes in file delphes.root
** Input file contains 0 events
** Exiting...

   The screen showed that

Running Pythia
Creating Pythia LHE File
Generating pythia lhe events
Warning! Never use this pythia lhe file for detector studies!
Creating Pythia LHE Root File
** Calculating number of events to process. Please wait...
** Input file contains 0 events
** Exiting...
finish
No pgs_card detected, so not run pgs
prepare delphes run
Running Delphes
delphes done

Cheers,
Karo

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#3

Ok then this is linked to a problem with the pythia output which can't be generated if the size of the file is larger than 4Gb.
How many events did you have in the lhe event file?

Cheers,

Olivier

On Nov 27, 2013, at 5:21 PM, karo <email address hidden> wrote:

> Question #239917 on MadGraph5 changed:
> https://answers.launchpad.net/madgraph5/+question/239917
>
> Status: Answered => Open
>
> karo is still having a problem:
> Hi Olivier,
>
> The pythia log is correct. But there are error in delphes log.
>
> ** Reading pythia_events.hep
> ** [----------------------------------------------------------------] (0.00%)
> ** ERROR: Inconsistent size of arrays. File is probably corrupted.
> ** Reading delphes.root
> Error in <TChain::LoadTree>: Cannot find tree with name Delphes in file delphes.root
> ** Input file contains 0 events
> ** Exiting...
>
> The screen showed that
>
> Running Pythia
> Creating Pythia LHE File
> Generating pythia lhe events
> Warning! Never use this pythia lhe file for detector studies!
> Creating Pythia LHE Root File
> ** Calculating number of events to process. Please wait...
> ** Input file contains 0 events
> ** Exiting...
> finish
> No pgs_card detected, so not run pgs
> prepare delphes run
> Running Delphes
> delphes done
>
> Cheers,
> Karo
>
> --
> You received this question notification because you are a member of
> MadTeam, which is an answer contact for MadGraph5.

Revision history for this message
karo (karo-t) said :
#4

multi_run 2
every run is 50000.

I think the all events should be 10,0000

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#5

That number of events usually go through.

Pavel do you have an idea about that?
Karo what is the size of the hep file produce by pyhtia?

Cheers,

Olivier

Revision history for this message
karo (karo-t) said :
#6

The size of the hep file is about 4.6 Gb.

The same number of events for other physical process not ttbar(lvqqbb,lvlvbb) could work well.

Best,
Karo

Revision history for this message
Pavel Demin (pavel-demin) said :
#7

Hi,

There is already a bug open for a similar problem:

STDHEP files larger than 4GB are corrupted

https://bugs.launchpad.net/madgraph5/+bug/1218842

This problem has been known since many years and it was always suggested to produce less than 40-50k events per STDHEP file.

Cheers,

Pavel

Revision history for this message
Pavel Demin (pavel-demin) said :
#8

I've just looked at the the mcfio/stdhep code from the pythia-pgs package.

My conclusion is that the mcfio/stdhep library used by the pythia-pgs/src/pythia.f program can't work with files larger than 4GB.

For more details, please see

https://bugs.launchpad.net/madgraph5/+bug/1218842

Revision history for this message
karo (karo-t) said :
#9

Hi,

    I have read that bug report.

    I hope it would be solved in the future. Because in many studies it really needs more than 40-50k events.

Cheers,
Karo

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#10

Hi Karo,

Yes but in many studies, you can import more than one input files. So this is not a big trouble.

Cheers,

Olivier

On Nov 28, 2013, at 9:21 AM, karo <email address hidden> wrote:

> Question #239917 on MadGraph5 changed:
> https://answers.launchpad.net/madgraph5/+question/239917
>
> karo posted a new comment:
> Hi,
>
> I have read that bug report.
>
> I hope it would be solved in the future. Because in many studies it
> really needs more than 40-50k events.
>
>
> Cheers,
> Karo
>
> --
> You received this question notification because you are a member of
> MadTeam, which is an answer contact for MadGraph5.

Revision history for this message
Pavel Demin (pavel-demin) said :
#11

Hi Karo,

I've replaced xdr_getpos/xdr_setpos with ftello/fseeko in some of the mcfio files and set the type of the position variables to off_t and now I can generate large STDHEP files without any problem.

Hopefully, this fix will be available in on of the future MadGraph releases.

Meanwhile, here is how the fixed mcfio files can be installed

# install pythia-pgs if it's not already installed
./bin/mg5
install pythia-pgs
quit

# install new mcfio files
cd pythia-pgs/libraries/PGS4/src/stdhep-dir/mcfio/src
wget http://cp3.irmp.ucl.ac.be/downloads/pythia-pgs-mcfio-files.tgz
tar -vzxf pythia-pgs-mcfio-files.tgz
cd -

# rebuild pythia-pgs
cd pythia-pgs
make clean
cd libraries/pylib
make
cd ../..
make
cd ..

# back to MG5
./bin/mg5

Revision history for this message
karo (karo-t) said :
#12

Hello,

         Pavel, I will try what you said. Thank you very much.

         Olivier, If I wanna more than one hundred thousand events, I must run MG5/Pythia/Delphes many times. That's what I do, again and again. Is there other automatic way to do that ?

Cheers,
Karo

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#13

Yes they are the multi_run options:

If you run the generation with the mg5 interface
launch [PATH]
you can instead do
launch [PATH] -i
then type:
multi_run X

where X is the number of run that you want

If instead you do
./bin/generate_events [OPTIONS]
you can do
./bin/madevent
multi_run X [OPTIONS]
or even
./bin/madevent multi_run X [OPTIONS]

Cheers,

Olivier

Revision history for this message
karo (karo-t) said :
#14

Hi Olivier,

    The events number is also limited even though it works in multi_run mode. Because this process would produce large pythia.hep file. If this file is more than 4Gb, the above problem will appear.

    If there is no better way, maybe we can try the method from Pavel.

    What do you think?

Cheers,
Karo

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#15

Hi Karo,

The multi_run will produce
1) one lhe file for the full sample (and X separate files as well for each run)
2) and produce X files for pythia/ reconstruction level output (i.e. not full sample)

So this should be fine.

>
> If there is no better way, maybe we can try the method from Pavel.

This is done.
just do "install pythia-pgs "
and you should have the version patched by Pavel.

Cheers,

Olivier

On Nov 29, 2013, at 3:16 PM, karo <email address hidden> wrote:

> Question #239917 on MadGraph5 changed:
> https://answers.launchpad.net/madgraph5/+question/239917
>
> Status: Answered => Open
>
> karo is still having a problem:
> Hi Olivier,
>
> The events number is also limited even though it works in multi_run
> mode. Because this process would produce large pythia.hep file. If
> this file is more than 4Gb, the above problem will appear.
>
> If there is no better way, maybe we can try the method from Pavel.
>
> What do you think?
>
> Cheers,
> Karo
>
> --
> You received this question notification because you are a member of
> MadTeam, which is an answer contact for MadGraph5.

Revision history for this message
Arian Abrahantes (arian-abrahantes) said :
#16

sorry to come again on this, I just installed a fresh MG framework and the issue appears again but not in every 50k run. So something is still failing with the patch. I'll reduce my events per run it seems safe until some solution is available. cheers

Revision history for this message
Arian Abrahantes (arian-abrahantes) said :
#17

it happens again yet I have simulated 20k events. It depends on my SUSY
parameterization actualy two of them (very different) because I had tested
at least 3 of them w/o any problem.

the generation process is a p p > t1 t1~ going to dileptons and n1 final
states. It seems to be a pythia problem this is awkward

cheers,

arian

On Tue, Feb 18, 2014 at 9:22 AM, Arian Abrahantes <
<email address hidden>> wrote:

> Question #239917 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/239917
>
> Arian Abrahantes posted a new comment:
> sorry to come again on this, I just installed a fresh MG framework and
> the issue appears again but not in every 50k run. So something is still
> failing with the patch. I'll reduce my events per run it seems safe
> until some solution is available. cheers
>
> --
> You received this question notification because you are a direct
> subscriber of the question.
>

Can you help with this problem?

Provide an answer of your own, or ask karo for more information if necessary.

To post a message you must log in.