Avoid LSF emails when running large gridpacks

Asked by Roberto Covarelli

Hello,

I am trying to edit the cluster.py code, as explained by Olivier in this post,

https://answers.launchpad.net/mg5amcnlo/+question/251681

in order to avoid that the CERN LSF cluster floods me with hundreds of emails when generating complicated processes.
However, as opposite to that user, I am running in gridpack creation mode.

In this mode, the solution proposed by Olivier

"Change:
       if stdout and isinstance(stdout, str):
           command.extend([‘-o', stdout])
by
       if stdout and isinstance(stdout, str):
           command.extend([‘-o', stdout])
       else:
           command.extend([‘-o’, ‘/dev/null’])
and
       if stderr and isinstance(stdout, str):
           command.extend(['-e', stderr])
       elif stderr == -2: # -2 is subprocess.STDOUT
           pass
by
       if stderr and isinstance(stdout, str):
           command.extend(['-e', stderr])
       elif stderr == -2: # -2 is subprocess.STDOUT
           pass
       else:
           command.extend([‘-e’, ‘/dev/null’])
"

does not work for me. The error is:

Working on SubProcesses
    P0_qq_wp_wp_lvl
    P0_qq_wp_wp_tapvl
    P1_qq_wm_wm_lvl
    P1_qq_wm_wm_tamvl
INFO: Idle: 1, Running: 3, Completed: 0 [ 0.18s ]
INFO: Idle: 0, Running: 3, Completed: 1 [ 30.4s ]
INFO: Idle: 0, Running: 3, Completed: 1 [ 1m 0s ]
INFO: Idle: 0, Running: 2, Completed: 2 [ 1m 30s ]
INFO: All jobs finished
Start waiting for update. (more info in debug mode)
Command "generate_events pilotrun" interrupted with error:
IOError : [Fail 5 times]
  [Errno 2] No such file or directory: '/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM
/wellnu01234j_5f_LO_MLM_gridpack/work/processtmp/SubProcesses/P0_qq_wp_wp_lvl/G1/results.dat'
Please report this bug on https://bugs.launchpad.net/madgraph5
More information is found in '/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_
LO_MLM_gridpack/work/processtmp/pilotrun_tag_1_debug.log'.
Please attach this file to your report.

I report here the file processtmp/pilotrun_tag_1_debug.log as suggested in the error

generate_events pilotrun
Traceback (most recent call last):
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/extended_cmd.py", line 879, in onecmd
    return self.onecmd_orig(line, **opt)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/extended_cmd.py", line 872, in onecmd_orig
    return func(arg, **opt)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/madevent_interface.py", line 1912, in do_generate_events
    postcmd=False)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/extended_cmd.py", line 919, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/extended_cmd.py", line 872, in onecmd_orig
    return func(arg, **opt)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/madevent_interface.py", line 2374, in do_survey
    cross, error = sum_html.make_all_html_results(self)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/sum_html.py", line 410, in make_all_html_results
    P_comb.add_results(name, pjoin(P_path,name,'results.dat'), mfactor)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/sum_html.py", line 131, in add_results
    oneresult.read_results(filepath)
  File "/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu01234j_5f_LO_MLM_gridpack/work/p
rocesstmp/bin/internal/misc.py", line 242, in deco_f_retry
    raise error.__class__, '[Fail %i times] \n %s ' % (i+1, error)
IOError: [Fail 5 times]
 [Errno 2] No such file or directory: '/afs/cern.ch/work/c/covarell/mg5_amcatnlo/zz1jet-offshell-8TeV/genproductions/bin/MadGraph5_aMCatNLO/wellnu01234j_5f_LO_MLM/wellnu0
1234j_5f_LO_MLM_gridpack/work/processtmp/SubProcesses/P0_qq_wp_wp_lvl/G1/results.dat'

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Alexis Kalogeropoulos (alkaloge) said :
#1

Hi Roberto

I also had the same problem - Can you try putting in your executable this line

 export LSB_JOB_REPORT_MAIL="N"

and submit with the following options

bsub -q ${queue} -J ${name} -i /dev/null -o my_job -e /dev/null ....

at least that greatly helped me .

Best

Alexis

Revision history for this message
Roberto Covarelli (roberto-covarelli) said :
#2

Hi Alexis,

Sorry for the naive question but what does it mean

"Can you try putting in your executable this line
export LSB_JOB_REPORT_MAIL="N" ?

in .bin/mg5_aMC only commands of the type set <madgraph option> <value> seem to be accepted. How do I issue a shell command?

Roberto

Revision history for this message
Alexis Kalogeropoulos (alkaloge) said :
#3

Hi Roberto

you can always run the .bin/mg5_aMC ran from a bash wrapper script (like the gridpack_generation.sh we internally use)- this is where you should put it.

Best

Alexis

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#4

Hi,

You can also put this line in your ~/.bashrc file.

Cheers,

Olivier

On 21 May 2015, at 05:01, Alexis Kalogeropoulos <email address hidden> wrote:

> Question #267171 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/267171
>
> Status: Open => Answered
>
> Alexis Kalogeropoulos proposed the following answer:
> Hi Roberto
>
> you can always run the .bin/mg5_aMC ran from a bash wrapper script
> (like the gridpack_generation.sh we internally use)- this is where you
> should put it.
>
> Best
>
> Alexis
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Can you help with this problem?

Provide an answer of your own, or ask Roberto Covarelli for more information if necessary.

To post a message you must log in.