long "improve" time for VBS event generation from gridpack

Asked by Hannes-a-mildner

Hi,

we are generating, ing MG 3.5.7, events from a gridpack for the process
generate p p > e+ e- e- ve~ j j QCD=0 @0
add process p p > mu+ mu- mu- vm~ j j QCD=0 @0
add process p p > ta+ ta- ta- vt~ j j QCD=0 @0
(in the SM, with massive taus).

We are aware this is a tricky process and phase space and are already using
2 = sde_strategy.

Still, we are suprised to see that, when we want to generate more than a few events from the gridpack, a lot of time is needed to "improve" and the job gets stuck:
16:19:59 generate 14000 events
16:19:59 P0_qq_taptamtamvlqq
16:30:45 P0_qq_lllvlqq
16:38:47 Using random number seed offset = 44
16:38:47 INFO: Generating 14000.0 unweighted events.
16:38:48 sum of cpu time of last step: 145 days,10h00m37s
16:38:48 INFO: need to improve 723 channels

This (or failure to generate enough events) is something we often struggle with in ATLAS when studying VBS.

Is there something we could do to improve the behaviour, in gridpack generation or when generating events from the gridpack?
I paste the full process and run cards below for completeness.

Cheers,
Hannes

---

16:19:58 Py:MadGraphUtils INFO proc_card:
16:19:58 #************************************************************
16:19:58 #* MadGraph5_aMC@NLO *
16:19:58 #* *
16:19:58 #* * * *
16:19:58 #* * * * * *
16:19:58 #* * * * * 5 * * * * *
16:19:58 #* * * * * *
16:19:58 #* * * *
16:19:58 #* *
16:19:58 #* *
16:19:58 #* VERSION 3.5.7 2024-11-29 *
16:19:58 #* *
16:19:58 #* The MadGraph5_aMC@NLO Development Team - Find us at *
16:19:58 #* https://server06.fynu.ucl.ac.be/projects/madgraph *
16:19:58 #* *
16:19:58 #************************************************************
16:19:58 #* *
16:19:58 #* Command File for MadGraph5_aMC@NLO *
16:19:58 #* *
16:19:58 #* run as ./bin/mg5_aMC filename *
16:19:58 #* *
16:19:58 #************************************************************
16:19:58 set group_subprocesses Auto
16:19:58 set ignore_six_quark_processes False
16:19:58 set low_mem_multicore_nlo_generation False
16:19:58 set complex_mass_scheme False
16:19:58 set include_lepton_initiated_processes False
16:19:58 set gauge unitary
16:19:58 set loop_optimized_output True
16:19:58 set loop_color_flows False
16:19:58 set max_npoint_for_channel 0
16:19:58 set default_unset_couplings 99
16:19:58 set max_t_for_channel 99
16:19:58 set zerowidth_tchannel True
16:19:58 set nlo_mixed_expansion True
16:19:58 import model sm
16:19:58 define l+ = e+ mu+
16:19:58 define l- = e- mu-
16:19:58 define wp = w+
16:19:58 define wm = w-
16:19:58 define p = g u c d s u~ c~ d~ s~
16:19:58 define j = g u c d s u~ c~ d~ s~
16:19:58 define l+ = e+ mu+ ta+
16:19:58 define l- = e- mu- ta-
16:19:58 define vl = ve vm vt
16:19:58 define vl~ = ve~ vm~ vt~
16:19:58 define wpm = w+ w-
16:19:58 generate p p > e+ e- e- ve~ j j QCD=0 @0
16:19:58 add process p p > mu+ mu- mu- vm~ j j QCD=0 @0
16:19:58 add process p p > ta+ ta- ta- vt~ j j QCD=0 @0
16:19:58 output -f -nojpeg
16:19:58 Py:MadGraphUtils INFO run_card:
16:19:58 #*********************************************************************
16:19:58 # MadGraph5_aMC@NLO *
16:19:58 # *
16:19:58 # run_card.dat MadEvent *
16:19:58 # *
16:19:58 # This file is used to set the parameters of the run. *
16:19:58 # *
16:19:58 # Some notation/conventions: *
16:19:58 # *
16:19:58 # Lines starting with a '# ' are info or comments *
16:19:58 # *
16:19:58 # mind the format: value = variable ! comment *
16:19:58 # *
16:19:58 # To display more options, you can type the command: *
16:19:58 # update to_full *
16:19:58 #*********************************************************************
16:19:58 #
16:19:58 #*********************************************************************
16:19:58 # Tag name for the run (one word) *
16:19:58 #*********************************************************************
16:19:58 tag_1 = run_tag ! name of the run
16:19:58 #*********************************************************************
16:19:58 # Number of events and rnd seed *
16:19:58 # Warning: Do not generate more than 1M events in a single run *
16:19:58 #*********************************************************************
16:19:58 14000 = nevents ! Number of unweighted events requested
16:19:58 44 = iseed ! rnd seed (0=assigned automatically=default))
16:19:58 #*********************************************************************
16:19:58 # Collider type and energy *
16:19:58 # lpp: 0=No PDF, 1=proton, -1=antiproton, *
16:19:58 # 2=elastic photon of proton/ion beam *
16:19:58 # +/-3=PDF of electron/positron beam *
16:19:58 # +/-4=PDF of muon/antimuon beam *
16:19:58 #*********************************************************************
16:19:58 1 = lpp1 ! beam 1 type
16:19:58 1 = lpp2 ! beam 2 type
16:19:58 6800.0 = ebeam1 ! beam 1 total energy in GeV
16:19:58 6800.0 = ebeam2 ! beam 2 total energy in GeV
16:19:58 # To see polarised beam options: type "update beam_pol"
16:19:58
16:19:58 #*********************************************************************
16:19:58 # PDF CHOICE: this automatically fixes alpha_s and its evol. *
16:19:58 # pdlabel: lhapdf=LHAPDF (installation needed) [1412.7420] *
16:19:58 # iww=Improved Weizsaecker-Williams Approx.[hep-ph/9310350] *
16:19:58 # eva=Effective W/Z/A Approx. [2111.02442] *
16:19:58 # edff=EDFF in gamma-UPC [eq.(11) in 2207.03012] *
16:19:58 # chff=ChFF in gamma-UPC [eq.(13) in 2207.03012] *
16:19:58 # none=No PDF, same as lhapdf with lppx=0 *
16:19:58 #*********************************************************************
16:19:58 lhapdf = pdlabel ! PDF set
16:19:58 260400 = lhaid ! if pdlabel=lhapdf, this is the lhapdf number
16:19:58 # To see heavy ion options: type "update ion_pdf"
16:19:58 #*********************************************************************
16:19:58 # Renormalization and factorization scales *
16:19:58 #*********************************************************************
16:19:58 False = fixed_ren_scale ! if .true. use fixed ren scale
16:19:58 False = fixed_fac_scale ! if .true. use fixed fac scale
16:19:58 91.188 = scale ! fixed ren scale
16:19:58 91.188 = dsqrt_q2fact1 ! fixed fact scale for pdf1
16:19:58 91.188 = dsqrt_q2fact2 ! fixed fact scale for pdf2
16:19:58 3 = dynamical_scale_choice ! Choose one of the preselected dynamical choices
16:19:58 1.0 = scalefact ! scale factor for event-by-event scales
16:19:58
16:19:58
16:19:58 #*********************************************************************
16:19:58 # Type and output format
16:19:58 #*********************************************************************
16:19:58 True = gridpack !True = setting up the grid pack
16:19:58 -1.0 = time_of_flight ! threshold (in mm) below which the invariant livetime is not written (-1 means not written)
16:19:58 average = event_norm ! average/sum. Normalization of the weight in the LHEF
16:19:58 #*********************************************************************
16:19:58 # Matching parameter (MLM only)
16:19:58 #*********************************************************************
16:19:58 1.0 = alpsfact ! scale factor for QCD emission vx
16:19:58 False = chcluster ! cluster only according to channel diag
16:19:58 ! (turn off for VBF and single top processes)
16:19:58
16:19:58
16:19:58 #*********************************************************************
16:19:58 #
16:19:58 #*********************************************************************
16:19:58 # Phase-Space Optimization strategy (basic options)
16:19:58 #*********************************************************************
16:19:58 0 = nhel ! using helicities importance sampling or not.
16:19:58 ! 0: sum over helicity, 1: importance sampling
16:19:58 2 = sde_strategy ! default integration strategy (hep-ph/2021.00773)
16:19:58 ! 1 is old strategy (using amp square)
16:19:58 ! 2 is new strategy (using only the denominator)
16:19:58 # To see advanced option for Phase-Space optimization: type "update psoptim"
16:19:58 #*********************************************************************
16:19:58 # Customization (custom cuts/scale/bias/...) *
16:19:58 # list of files containing fortran function that overwrite default *
16:19:58 #*********************************************************************
16:19:58 = custom_fcts ! List of files containing user hook function
16:19:58 #*******************************
16:19:58 # Parton level cuts definition *
16:19:58 #*******************************
16:19:58 0.0 = dsqrt_shat ! minimal shat for full process
16:19:58 #
16:19:58 #
16:19:58 #*********************************************************************
16:19:58 # BW cutoff (M+/-bwcutoff*Gamma) ! Define on/off-shell for "$" and decay
16:19:58 #*********************************************************************
16:19:58 15.0 = bwcutoff ! (M+/-bwcutoff*Gamma)
16:19:58 #*********************************************************************
16:19:58 # Standard Cuts *
16:19:58 #*********************************************************************
16:19:58 # Minimum and maximum pt's (for max, -1 means no cut) *
16:19:58 #*********************************************************************
16:19:58 15.0 = ptj ! minimum pt for the jets
16:19:58 4.0 = ptl ! minimum pt for the charged leptons
16:19:58 0.0 = misset ! minimum missing Et (sum of neutrino's momenta)
16:19:58 -1.0 = ptjmax ! maximum pt for the jets
16:19:58 -1.0 = ptlmax ! maximum pt for the charged leptons
16:19:58 -1.0 = missetmax ! maximum missing Et (sum of neutrino's momenta)
16:19:58 {} = pt_min_pdg ! pt cut for other particles (use pdg code). Applied on particle and anti-particle
16:19:58 {} = pt_max_pdg ! pt cut for other particles (syntax e.g. {6: 100, 25: 50})
16:19:58 #
16:19:58 # For display option for energy cut in the partonic center of mass frame type 'update ecut'
16:19:58 #
16:19:58 #*********************************************************************
16:19:58 # Maximum and minimum absolute rapidity (for max, -1 means no cut) *
16:19:58 #*********************************************************************
16:19:58 5.0 = etaj ! max rap for the jets
16:19:58 5.0 = etal ! max rap for the charged leptons
16:19:58 0.0 = etalmin ! min rap for the charged leptons
16:19:58 {} = eta_min_pdg ! rap cut for other particles (use pdg code). Applied on particle and anti-particle
16:19:58 {} = eta_max_pdg ! rap cut for other particles (syntax e.g. {6: 2.5, 23: 5})
16:19:58 #*********************************************************************
16:19:58 # Minimum and maximum DeltaR distance *
16:19:58 #*********************************************************************
16:19:58 0.4 = drjj ! min distance between jets
16:19:58 0.2 = drll ! min distance between leptons
16:19:58 0.2 = drjl ! min distance between jet and lepton
16:19:58 -1.0 = drjjmax ! max distance between jets
16:19:58 -1.0 = drllmax ! max distance between leptons
16:19:58 -1.0 = drjlmax ! max distance between jet and lepton
16:19:58 #*********************************************************************
16:19:58 # Minimum and maximum invariant mass for pairs *
16:19:58 #*********************************************************************
16:19:58 110.0 = mmjj ! min invariant mass of a jet pair
16:19:58 0 = mmll ! min invariant mass of l+l- (same flavour) lepton pair
16:19:58 -1.0 = mmjjmax ! max invariant mass of a jet pair
16:19:58 -1.0 = mmllmax ! max invariant mass of l+l- (same flavour) lepton pair
16:19:58 {} = mxx_min_pdg ! min invariant mass of a pair of particles X/X~ (e.g. {6:250})
16:19:58 {'default': False} = mxx_only_part_antipart ! if True the invariant mass is applied only
16:19:58 ! to pairs of particle/antiparticle and not to pairs of the same pdg codes.
16:19:58 #*********************************************************************
16:19:58 # Minimum and maximum invariant mass for all letpons *
16:19:58 #*********************************************************************
16:19:58 0.0 = mmnl ! min invariant mass for all letpons (l+- and vl)
16:19:58 -1.0 = mmnlmax ! max invariant mass for all letpons (l+- and vl)
16:19:58 #*********************************************************************
16:19:58 # Minimum and maximum pt for 4-momenta sum of leptons / neutrino *
16:19:58 # for pair of lepton includes only same flavor, opposite charge
16:19:58 #*********************************************************************
16:19:58 0.0 = ptllmin ! Minimum pt for 4-momenta sum of leptons(l and vl)
16:19:58 -1.0 = ptllmax ! Maximum pt for 4-momenta sum of leptons(l and vl)
16:19:58 #*********************************************************************
16:19:58 # Inclusive cuts *
16:19:58 #*********************************************************************
16:19:58 0.0 = xptj ! minimum pt for at least one jet
16:19:58 0.0 = xptl ! minimum pt for at least one charged lepton
16:19:58 #*********************************************************************
16:19:58 # Control the pt's of the jets sorted by pt *
16:19:58 #*********************************************************************
16:19:58 0.0 = ptj1min ! minimum pt for the leading jet in pt
16:19:58 0.0 = ptj2min ! minimum pt for the second jet in pt
16:19:58 -1.0 = ptj1max ! maximum pt for the leading jet in pt
16:19:58 -1.0 = ptj2max ! maximum pt for the second jet in pt
16:19:58 0 = cutuse ! reject event if fails any (0) / all (1) jet pt cuts
16:19:58 #*********************************************************************
16:19:58 # Control the pt's of leptons sorted by pt *
16:19:58 #*********************************************************************
16:19:58 0.0 = ptl1min ! minimum pt for the leading lepton in pt
16:19:58 0.0 = ptl2min ! minimum pt for the second lepton in pt
16:19:58 0.0 = ptl3min ! minimum pt for the third lepton in pt
16:19:58 -1.0 = ptl1max ! maximum pt for the leading lepton in pt
16:19:58 -1.0 = ptl2max ! maximum pt for the second lepton in pt
16:19:58 -1.0 = ptl3max ! maximum pt for the third lepton in pt
16:19:58 #*********************************************************************
16:19:58 # Control the Ht(k)=Sum of k leading jets *
16:19:58 #*********************************************************************
16:19:58 0.0 = htjmin ! minimum jet HT=Sum(jet pt)
16:19:58 -1.0 = htjmax ! maximum jet HT=Sum(jet pt)
16:19:58 0.0 = ihtmin !inclusive Ht for all partons (including b)
16:19:58 -1.0 = ihtmax !inclusive Ht for all partons (including b)
16:19:58 #*********************************************************************
16:19:58 # WBF cuts *
16:19:58 #*********************************************************************
16:19:58 0.0 = xetamin ! minimum rapidity for two jets in the WBF case
16:19:58 0.0 = deltaeta ! minimum rapidity for two jets in the WBF case
16:19:58 #*********************************************************************
16:19:58 # maximal pdg code for quark to be considered as a light jet *
16:19:58 # (otherwise b cuts are applied) *
16:19:58 #*********************************************************************
16:19:58 4 = maxjetflavor ! Maximum jet pdg code
16:19:58 #*********************************************************************
16:19:58 #
16:19:58 #*********************************************************************
16:19:58 # Store info for systematics studies *
16:19:58 # WARNING: Do not use for interference type of computation *
16:19:58 #*********************************************************************
16:19:58 True = use_syst ! Enable systematics studies
16:19:58 #
16:19:58 systematics = systematics_program ! none, systematics [python], SysCalc [depreceted, C++]
16:19:58 ['--weight_info=MUR%(mur).1f_MUF%(muf).1f_DYNSCALE%(dyn)i_PDF%(pdf)i','--remove_wgts=".*MUR0.5_MUF2.0.*|.*MUR2.0_MUF0.5.*"','--pdf=NNPDF30_nlo_as_0118_nf_4,PDF4LHC21_40_pdfas_nf4,NNPDF31_nlo_as_0118_nf_4,NNPDF40_nlo_as_01180_nf_4,NNPDF30_nlo_as_0119_nf_4@0,NNPDF30_nlo_as_0117_nf_4@0,NNPDF30_nnlo_as_0118_nf_4@0,CT14nlo_NF4@0,MMHT2014nlo68clas118_nf4@0,MSHT20nnlo_nf4@0,MSHT20nlo_nf4@0,NNPDF31_nnlo_as_0118_nf_4@0,NNPDF40_nnlo_as_01180_nf_4@0,PDF4LHC15_nlo_nf4_30@0','--muf=0.5,1.0,2.0','--mur=0.5,1.0,2.0','--dyn=-1,1,2,3,4'] = systematics_arguments ! see: https://cp3.irmp.ucl.ac.be/projects/madgraph/wiki/Systematics#Systematicspythonmodule

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#1

Hi,

I thought that I was done to have to focus on such type of processes... Looks like I was wrong...
Given that any investigation for a new design of a phase-space integrator can take quite a long time.
Either they are big progress (which is actually) with the next big release (which deeply change the phase-space strategy --see the MadNis and other related paper--)

But in the shorter term, my suggestion would be to
1) check using the complex mass scheme (which is perfectly suited for your process) is not a solution
2) check if the FD-gauge (with sde=1) is not a valid solution for your process.

Cheers,

Olivier

Revision history for this message
Hannes-a-mildner (hannes-a-mildner) said :
#2

Thanks Olivier,

so there is nothing obviously wrong with the setup right now but your suggestion might yield a more optimal integration performance?

And it is not completely expected to find such a message and the job being stuck afterwards (presumably because "improve" is complicated for some of these channels?)

16:19:59 generate 14000 events
16:19:59 P0_qq_taptamtamvlqq
16:30:45 P0_qq_lllvlqq
16:38:47 Using random number seed offset = 44
16:38:47 INFO: Generating 14000.0 unweighted events.
16:38:48 sum of cpu time of last step: 145 days,10h00m37s
16:38:48 INFO: need to improve 723 channels

Cheers,
Hannes

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#3

> so there is nothing obviously wrong with the setup right now but your suggestion might yield a more optimal integration performance?

Since the reason of unstability is gauge invariance related, having better handling of gauge invariance and of the associated cancellation can lead to more optimal integration performance indeed (see 2102.00773 [hep-ph])

> And it is not completely expected to find such a message and the job being stuck afterwards (presumably because "improve" is complicated for some of these channels?)

16:19:59 generate 14000 events
16:19:59 P0_qq_taptamtamvlqq
16:30:45 P0_qq_lllvlqq
16:38:47 Using random number seed offset = 44
16:38:47 INFO: Generating 14000.0 unweighted events.
16:38:48 sum of cpu time of last step: 145 days,10h00m37s
16:38:48 INFO: need to improve 723 channels

Given that the survey/gridpack generateion took 145 days and the number of channel to generate events from.
I would not expect that step to go super fast. Recent gridpack can now be run on multiple core, this can help to make this faster obviously (but the ressource usage does not really improve).
But except the red flag that integration is struggling nothing conceptually wrong.

Cheers,

Olivier

Can you help with this problem?

Provide an answer of your own, or ask Hannes-a-mildner for more information if necessary.

To post a message you must log in.