Question about madgraph "job" description in ps_optim?

Asked by Radha Mastandrea on 2021-02-15

I am currently looking at some of the hidden parameters in the ps_optim block, specifically "refine_evt_by_job", which has the description: "control the maximal number of events for the first iteration of the refine (larger means less jobs)". In this case, can you define what a "job" is? My thinking is that a "job" is a few calculations of the matrix element / integration at a given phase space point, so more jobs means looking at more phase space points? I am just a bit confused as in other documentation, a "job" is simply everything that happens after entering the "launch" command.

Thank you very much!

Question information

English Edit question
MadGraph5_aMC@NLO Edit question
No assignee Edit question
Last query:
Last reply:

What job means is the following: When you run the code, you will see line like for the various cpu heavy part of the computation:
(example for the refine stage of g g > t t~ g g)
INFO: Idle: 9, Running: 8, Completed: 0 [ current time: 21h14 ]
INFO: Idle: 8, Running: 8, Completed: 1 [ 22.6s ]
INFO: Idle: 6, Running: 8, Completed: 3 [ 29.9s ]
INFO: Idle: 4, Running: 8, Completed: 5 [ 36.9s ]
INFO: Idle: 2, Running: 8, Completed: 7 [ 40.9s ]
INFO: Idle: 1, Running: 8, Completed: 8 [ 50.2s ]
INFO: Idle: 0, Running: 6, Completed: 11 [ 57.2s ]
INFO: Idle: 0, Running: 4, Completed: 13 [ 1m 6s ]
INFO: Idle: 0, Running: 1, Completed: 16 [ 1m 11s ]
INFO: Idle: 0, Running: 0, Completed: 17 [ 1m 13s ]

So in the above example, I have 17 jobs (and they take 1m13s on my laptop)

If you set the above parameter to a very large number (like 100k when you request only 10k events) then you will not have any splitting and we will run to minimal number of submission:
Which in this case is 11 jobs (taking 2m 22s). In this case I'm saturing my cpu (8 core) for only 22s (compare to 50s in the above case)

Setting it to very low number (like 10), then I'm probably hitting some security since I only submitting 108 jobs
taking 2m23s, I'm saturating my cpu for a full 2m6s here, and the code is slower since I pay the price of the overheard and to the fact that each of those computation have some minimum task to perform.

So this will allow to saturate Rome/Milan cpu.



Can you help with this problem?

Provide an answer of your own, or ask Radha Mastandrea for more information if necessary.

To post a message you must log in.