Getting binary/corrupted output when I use pythia

Asked by Koay Yong Sheng

Everything works fine if I dont use pythia to shower/hadronize. But since updating to 2.6.7, it always give me corrupted/binary output during pythia8 jobs. This occurs regardless of the type of model I'm using. I tried using different models and it still occurs. I'm not sure whether there is some problems with my installation. I installed pythia via the command "install pythia8" and the same goes for lhapdf6, mg&P8 interface, zlib, etc. This weird ouutput occurs both in my laptop and my work computer. I tried reinstalling for both machine but the problem remain. I have no problems before the update, everything works find in the previous version.

Below is an example of the output I get at my terminal, I have used the default sm model and
p p > ta- ta+ is my process. I cant even show the whole output (or particularly the point it breaks) since the binary/corrupted output is so much that it goes beyond my terminal limit. Note: I manually stop the program as you can see from the "^C"s.

Do inform me whether this is a bug or I have installed it wrong somehow. Also do inform me what kind of files that I would need to upload in order for you to check this matter properly. Thanks in advance and hope to hear from you soon.

" d
 |j�n|jd
             r�tjtjj
��t|jd d
          <n|j
dd�kr�t
j
d�dS|jd
                                         s�tj t
|jdd��
            r�|jd��n>t
|jdd�}t j!|j�}|r+t
j"|�n|jj#dd�|j$dd�|j%tj&t
|j
  dd��} | j't
|j
  dd�dd�d
}
d
d
g}
       t(|j
d�dkr�d
}
nHt(|j
d�d ks
|j
d!d"ks
|j
d#d"krd
}
n|j)| |
�}
   t
|j
t*j*�}|%j+|t
|j
  dd�d&t
           �d'|jkr�tj,|jd'd(�}nIt-r�tj,t
|jdtj.�d(�}ntj,t
�f|j2��t1|
|j
  d$|jd,�}t
|j
�g�f}|j+|�|j6�tj7|�}tj8||j9t7j:B�| d0�r�d1|}n7tj5|�}|s�|jd2d3|d4|��nd5|d6j|tj1|
                                                     satn*tj |
                                                                 �o�t7j;tj7|
                                                                             �j9�}t<j<��idd76dd86dd96id:6�|r=t
j=d;|�t0|d*�}tj>|gd<|d=|d>t
|j
  d$|j��}|s9t
j=d?d@|
                          dA�ndS|jddBkr�|j?|jd�|jddkr�t@|jdCd�}n+|jdd kr�t@|jAjBd�}ntj1| jCdBdD�}tDjEt
|j
  d$|j| jCdBdD��}
t|
�}
| dEdkr,|
}
nY| dE}
|
|
kr�|jdF|
dG|
dHt
|j
  d$|j| jCdBdD��n|jdd kr�dI}n
|jddkr�dJ}n|
|} t@tF| |�d�}n|jddBks|jdd kr
 |jdKdkr
 d|_At
j=dLdM|�|jdd kr|jdKdkr|jAjG|dNgd<|d=tHjId>t
|j
  d$|j��}!nOt0|d*�}"tjJ|d<|"d=tHjId>t
|j
  d$|j��}!|"j6�|!dBkr=|jdOdP|!dQ|�q=n|j
ddRkrZ t
jKdS�t
jKdT�tdU��nt
|j
  d$|jdV�}#tjL|#�r� tMjN|#�ntjO|#�tMjP||#�tPjP| �}$|
                                                                     r� dW|$dX<n
dY|$dX<|$jCdBjQdDdZ�|$j+t
|#d[�t
|j
  dd�d&t
           �t
|#d\�}t0|d*�}|jd]dr_
d^}nd_}||tj1|�f}|j+|�|j6�tj7|�}tj8||j9t7j:B�|
|g|}%x(tR|
|�D]}&|%|&cd7<q�
W|
|g|}'x(tR|
|�D]}&|'|&cd7<q
                                          Wt
j=d`�|
jSda|%d>|#dbt
                                                                   �}(|(t|%�kr�
                                                                                   tTdct|%�|(f��ng})g}*xptR|(�D]b}+|)jUdd|+�|*jUt
|#de|+��tMjVt
|#|
jWdf|+�t
|#|)d��q�
              Wt
j=dg�xtX|)�D]\}&},tj&t
|#d[��}-|-jQdE|'|&�|-jQdh|-dhtY|'|&�tY|
��|-j+t
|#di|&�t
|#d[�djt�t
|#tj1|��t
|#di|&�t
g}/tjOt}.|jd]dr�
|#de|&��t
|#de|&�}0x�|.t
Wg}.tj1|�}ndm|&g}/|#}0|jAj\|dNt]|&�gd>|0dn|.do|/dp|/�q.
t W�fdq�}2|jAj^|#|2�t
j=dr�|jd]r�xWtX|)�D]F\}&},tjJdsdtdm|&gd>|#�tj
|#dm|&��q}Wnt0|d*�}3dB}4xY|*D]Q}5t
|5du�}6|3j+dvdwdx�|3j+dy|&dx�|3j+dvdwdx�|3j+t0|6dz�j'�dx�|
|
 kr�|j_|6�\}7}8}9t`d{�|7|8|9gD��r�q�n|4d7}4�d7dr�|7�d7<n�d7c|77<�d8dr�|8�d8<n�d8c|87<�d9dr$|9�d9<q7�d9c|97<q�q�W|4dBkr`�d7ctY|4�
<n|3j6�d}:dB}4xo|*D]g}5t
|5d|�};tja|;�s�q}n|jb|;�}<t|<�dBkr�|4d7}4t�d:�dBkr/|<�d:<x��d:D](}=�d:|=dd �d:|=d<qWq�xY|<jc�D]H\}=}>�d:|=dBc|>dB7<�d:|=dc|>dd 7<q<Wn|jd|;d}dB�}?|:dr�|?}:q}x,tX|:�D]
\}&}@|@|?|&|:|&<q�Wq}W|:dk r]|:jet
|j
  d$|jd~�dd��tMjVt
|j
  d$|jd��t
|j
  d$|jd���n|4dBkr�xF�d:D]7}=tfjg�d:|=d�tY|4��d:|=d<qtWnd}Ax�|*D]�}5t
|5d��}Btja|B�s�q�n|jd|Bd}dB�}C|Adr|C}Aq�x,tX|A�D]
\}&}@|@|C|&|A|&<q#Wq�W|Adk r�|Ajet
|j
  d$|jd��dd��tMjVt
|j
  d$|jd��t
|j
  d$|jd���ng}DxB|*D]:}5t
|5dW�}Etja|E�s�q�n|DjU|E�q�Wt|D�dBkrt
|j
  d$|j|
       �}Ftjh���}Gt0t
|Gd��d*�}HdB}IxEt0|DdBdz�D]0}|j[d��s�|Id7}I|Hj+|�qwPqwW|Hj6�t0t
|Gd��d*�}JdB}KxEtji|Dd�D]0}|j[d��r|Kd7}K|Jj+|�q�Pq�W|Jj6�|Kdkr=tTd��nx�|DD]�}Etjjd�tk�}L|LdBkr�tjjd�|E|Itjl|E�f�tjJd�d�tj1|E�gd>tjl|E��qDtmjnd�krtjjd6jd�d�d�d�d�jd��tR|I�D��|Eg��qDtjjd��tjjd6jd�d�d�|Ed�d�d�d�t]|I�d�d�g
��tjjd6jd�d�|Eg��tjjd6jd�d�ggtR|I�D]}&d�|&d^q�d�|Eg��qDWtjjd6jd�t
|Gd��g|Dt
|Gd��d�|Fg��WdQXntjL|#�r=tMjN|#�nt
|j
  d$|jd|�}Mtja|M�r�tMjV|Mt
|j
  d$|jd���nt
|j
  d$|jd��}Ntja|N�r�tMjV|Nt
|j
  d$|jd���ntja|�
                      s*d�dxjt0|dz�jo�d��kr?t
j"d�|�dS|jpd��}O|Osdt
j"d��n|jqjUd�|
|
 kr�
tr�fd��d7d8d9gD��r2|jddBks�|jdd kr
|jdKdkr
|j_t
|j
  d$|jd,��\�d7<�d8<�d9<q2t
j"d�d��nt`�fd��d7d8d9gD��s*|jj#d��d7�|jj#d��d8��d8}8�d9}9�d7}7|j|jjs|j%�d�}Py7tfjg|P|8|9d |7d d|8|9|8�}QWnttk
rd�}QnX|jj#d�|Q�n|j
d�r`|jj#d�d�|jj#d�dB�nt
|j
  d$|jd��}Mtja|M�rt�d:�dBkrjddBks�|jdd kr�|jdKdkr�|jb|M��d:<qt
j"d�d��i�d:<n�d:}R|Rr�d�}Stujvd�|Stujw��tx�fd��|Rjc�D��}Rt(|j
d�dkr� d�n| d�}T|T|Rkr�jj#d�|R|TdB�|jj#d�|R|Td�q�t0t
|j
�t(|j
d�r�
d�|jkr�
|jd�cd�|jj|d�7<q�
d�|jj|d�|jd�<ntdf�q,
Wn|Uj+d�d��|Uj6�n|jj{|
|j
  d$|jd�|jf�}W|jj+|W�|j$d�tj}t<j<���d�d�|jd�r�
|j~d�d�td�t�n|j|jj|�dS(�slaunch pythia8i����Ns
                                                                                                                     --no_defaultROspythia8_card.datR5iR9tpythia_versionR[tautomatic_html_openings
                      crossx.htmlRGtunitR`sFPythia8 does not support normalization to the sum. Not running Pythia8R~tMG5aMC_PY8_interfaces�The MG5aMC_PY8_interface tool cannot be found, so that MadEvent cannot steer Pythia8 shower.
Please install this tool with the following MG5_aMC command:
  MG5_aMC> install mg5amc_py8_interface_pathR1R�s*Running Pythia8 [arXiv:1410.3012]RUspythia8_card_default.dattsettertuserR�R�R�RkiRlgRmRXs%s_pythia8.cmdtdirect_pythia_inputtheptools_install_dirtlibHEPToolsRhs@!
! It is possible to run this card manually with:
! %s %s
!
run_shower.shtbashttcshs
                          /usr/bin/envs/usr/bin/env %ss.No s hell could be found in your environment.
s6Make sure that either '%s' is in your path or that thes; command '/usr/bin/env %s' exists and returns a valid path.s#!%s
%sRtsigma_mtNacctNtryRzs�Pythia8 is set to output HEPMC events to to a fifo file.
You can follow PY8 run with the following command (in a separate terminal):
    tail -f %sRMR�R�s:You can now run a tool that reads the following fifo file:s:
   %s
where PY8 outputs HEPMC events (e.g. MadAnalysis5).s$MG:color:GREENit
                                                                     cluster_sizes
Beams:LHEFsMain:numberOfEventss3You specified more events (%d) in the PY8 parametersD'Main:numberOfEvents' than the total number of events available (%d)s in the event file:
 %sidi�tnb_cores%Follow Pythia8 shower by running the s:following command (in a separate terminal):
    tail -f %sRis&Pythia8 shower interrupted with returns
 code %d.
s2You can find more information in this log file:
%sRtRs<Either run in single core or change event_norm to 'average'.s�Pythia8 parallelization with event_norm set to 'sum' is not supported.Either run in single core or change event_norm to 'average'.tPY8_parallelizations
events.lhe.gzs events.hepmcsHEPMCoutput:files /dev/nulls
              PY8Card.dats
run_PY8.shtcluster_temp_paths&#!%s
./%s PY8Card.dat >& PY8_log.txt
sz#!%s
ln -s ./events_$1.lhe.gz ./events.lhe.gz
./%s PY8Card_$1.dat >& PY8_log.txt
mkdir split_$1
if [ -f ./events.hepmc ];
then
   mv ./events.hepmc ./split_$1/
fi
if [ -f ./pts.dat ];
then
   mv ./pts.dat ./split_$1/
fi
if [ -f ./djrs.dat ];
then
   mv ./djrs.dat ./split_$1/
fi
if [ -f ./PY8_log.txt ];
then
   mv ./PY8_log.txt ./split_$1/
fi
tar -czf split_$1.tar.gz split_$1
s4Splitting .lhe event file for PY8 parallelization...t partitiontzipsCError during lhe file splitting. Expected %d files but obtained %d.sevents_%s.lhe.gzsplit_%ds
_%d.lhe.gzsSubmitting Pythia8 jobs...sHEPMCoutput:scalingsPY8Card_%d.datt
                                                                           add_missingRxtPY8Cardssplit_%d.tar.gzt
                                                                                                                 input_filest
                                                                                                                             output_filestrequired_outputc sL|||dkrdStjstartPY8timer(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pytwait_monitoring�tDone(s*Merging results from the split PY8 runs...ttars-xzfs
        PY8_log.txtR$i#s
s" -> Pythia8 log file for run %d <-R�cs s]}|dVqdS(N(R6(R�telem((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>�sdjrs.dattrun_idtdjrsRtHwUdjrs.HwUs
                                           %s_djrs.datspts.dattptsspts.HwUs
%s_pts.dats
           header.hepmctEs
tail.hepmcsHepMC::s2HEPMC files should only have one trailing command.shead -n -1 %s &> /dev/nulls(head -n -1 %s | tail -n +%d > %s/tmpfileR�R4tdarwintseds-is''s'%s;$d't;cs s
|]}d|dVqdS(s%idiN((R�R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>@ss
head -n -1theads-ns-1R�ttailt+t>ttmpfile-e '%id's-e '$d'tcatsInclusive cross section:i����s7Fail to produce a pythia8 output. More info in
     %sR�s(Failed to produce Pythia8 merging plots.c3 s|]}�|dVqdS(N(R6(R�Rg(tPY8_extracted_information(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>lss.Pythia8 cross-section could not be retreived.
sITry turning parallelization off by setting the option nb_core to 1. YYYYYc3 s|]}�|dVqdS(N(R6(R�Rg(R�(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>vsRhRiR2g��RjRcs6Pythia8 merged cross-sections could not be retreived.
sHTry turning parallelization off by setting the option nb_core to 1.XXXXXs%[\+|-]?\d+(\.\d*)?([EeDd][\+|-]?\d+)?s,^\s*Weight_MERGING\s*=\s*(?P<merging>%s)\s*$c3 sN|]D\}}�j|�dk rt�j|�jd��|fVqdS(tmergingN(R*R6R>tgroup(R�R?R�(tcentral_merging_re(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>�s sJetMatching:qCuts
error_pythia8s%s_merged_xsecs.txts%-20s%-20s%-20s Merging:TMSt
Merging scalesCross-section [pb]sMC uncertainty [pb]s%-20.4g%-20.6e%-20.2e
s)Cross-sections could not be read from thes9XML node 'xsection' of the .dat file produced by Pythia8.tMGGenerationInfos)# Matched Integrated weight (pb) : %s
Merging scalesCross-section [pb]sMC uncertainty [pb](�R�telphes --no_defaultR?R
(R�R`(R�N(s
                                                            ImportErrortinternal.histogramst
histogramstadgraph.various.histogramsR�RzR_R`RaRR%R7R4RRbRWR�R[RR\RKR/t
ask_pythia_run_configurationR&Rt open_fileR�R,R R<RR�R�RZR'R5ReR�R�R%R�StringIORRt
get_HEPTools_location_setterR�tpardirRR�getvaluetget_shell_typeR6twhichRSR�tchmodR�tS_IEXECR�R�R!R�tconfigure_run_modeR�R.R�R�R�R�tmintlaunch_and_waitR�tSTDOUTR�R2RcR�trmtreeRdtcopyR�R�RBRR�tmoveRxR
R>RRFtsubmit2RERtparse_PY8_log_fileR�R
textract_cross_sections_from_DJRR�tHwUListR�R�R�RNBackReadR�__file__RR�platformt readlinesR@RXR.t
return_tagtZeroDivisionErrorR�Rt
IGNORECASER5RwRR^RARvR�R>(XR$RJR�R�R�Rlt
                                                 pythia_mainwarningsR�R�Rtmerged_run_typesR�tpythia_cmd_cardcmd_cardpreamblet
pythia_logt
lhe_file_namelhe_filetn_available_eventsn_eventstmin_n_events_per_jobtoutput_fifotpy8_logtpy8_bkgrd_proctn_corest
min_n_coreret_codeRMtparallelization_dirtParallelPY8CardR�R�tpartition_for_PY8n_splitst
                                                                                            split_filest
split_dirssplit_idt
split_filetsplit_PY8_Cardin_filest out_filest
                                                  selected_cwdtin_fileR�tpythia_log_filetn_addedt split_dirlog_fileR�R�R�tdjr_HwUdjr_filetxsecsR�R�t
                                                                                                                                                             new_djr_HwUthisttpts_HwUpts_filet
              new_pts_HwUtall_hepmc_filest
hepmc_filet
           hepmc_outputRQtheadertn_headR�tn_tailR�t
djr_outputt pt_outputt
successfulR2terror_mRzt
central_scalet
xsecs_fileR{t
              banner_path((R�R�R�sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt
hia8EsD
# 
   
                           
  & 
"+
      
 


    
         

     
         


                                         

 .  
9  
                 &


 

 



 
          

   


)
'
              
                     (
#
 8
         
 (
#




 

>:"&(
                                                %9: 
                                                                 %


 7
(9 

        
           -

c$
(* !  ( 
\}}}xtj|�D]�}d
}|j|�s�|j|�} | ssq=qGt| jd��d}q=q=y�|d
                                                                     r�tjd��d}n|d
                                                                                        r�jd��}n|d
                                                                                                     rjd��}n|dkr$|j d �n|||fSWq=t
k
rFdSXq=W|j d
d
 |�d
        S(s@ Parse a log file to extract number of event and cross-section. s�Les Houches User Process\(es\)\s*\d+\s*\|\s*(?P<tried>\d+)\s*(?P<selected>\d+)\s*(?P<generated>\d+)\s*\|\s*(?P<xsec>[\d\.e\-\+]+)\s*(?P<xsec_error>[\d\.e\-\+]+)sSInclusive cross section\s*:\s*(?P<xsec>[\d\.e\-\+]+)\s*(?P<xsec_error>[\d\.e\-\+]+)R?ge��At generatedttriedis
Pythia8 shower failed since its5 did not accept any event from the MG5aMC event file.s:Could not find cross-section and event number information sin Pythia8 log
  '%s'.N(NNN(NNN(
                   R�RR6RR�R�R>R�R%RR&(
log_file_pathpythiaretpythia_xsec_reR�R�R�RJR!tfinal_PY8_xsec((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s4
                                                                                                                                                                                

  
     
         
  cC s|ddljj}|j|�jd�}td�|D��}y|d}WiSX|jd�}td�|D��Ss-Extract cross-sections from a djr XML output.i����NR�cs s*|] }t|jd��|fVqdS(RN(R%t
                    getAttribute(R�tnode((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>�sixsectioncs s^|]T}|jd�t|jdjj�d�t|jdjj�d�gfVqdS(RxiiN(RKR>t
childNodesRwRB(R�R?((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>�s(txml.dom.minidomtdomtminidomtparsetgetElementsByTagNameR5(R$R<RQt run_nodest all_nodestselected_run_nodet xsections((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s cC s|j|�}d|krVtjjt|jdd��s@dSt}|jd�n}|j s�|j
|�|j
       d�n
|j
               d�|j
|�|j
j�dS|s�|j|d �n|jd
r
tjtjj|jd
                 ��|jd
<n|j
     s;t|j�d
�y tjt|jdd��Wntk|jdd
r�nXt
j
d|t
t|jdd��j�t
j �s6t
t|jdd�d�}|j!d|�|j"�n|j#}t|jd|j d|�}dg|j
                                                                                                              dr�j$d�n|j
                                                                                                                              d
d
�t,|jdS|j+j$ddd�t|d �gd$d%|d&t(j)d't|jd��tjt|jdd#��tjjt|jdd��s�t kr�d
d
dg7nt|d �tj%d!<|j&j't|d
       d
�r^tj-t|jd|j d|��}t
j.d)�} xi|D]a}| j
|�}
|
q�nySt/|
j0d*�j1d+d,��d-}
                    t,|
j0d.��}
         t,|
WnJt2k}
r�|jjd0d1�|jjd2d1�|jjd3d1�n�X|jjd0|
                                                     �|jjd2|
                                                                �|j|j j3|j#�d4}|
                                                                                             r/t4j5||
d5| |
     d5d
         |
| |
    �}n
d6|
   }|jjd3|�Pq�W|j"�nt|jdd�}|jd7}|jd8}|jd9}|jj6t|jdd��t,|j
                                                                                     d
�r
d:|jkr|jd:cd;|jj7d07<q
d;|jj7d0|jd:<nt|jd|j d<|j |f�}|jj!|�|j8|�t,|j
                      d
�r�tj9t|jdd
�d%t|jd|j |d=��n|j
                                                                      d|j:kr�y|j;d>�Wn&t<k
jt=|��q�Xtjjt|jdd��r�tj9t|jdd�d%t|jd|j |d?��tjjt|jdd@��r�t|jd|j dA|j#�}tj9t|jdd@�d%dB|�q�q�n|j>d>�tjjt|jddC��rtj9tlaunch pythiasJ�|jdKr|j?dLdIdJ�n|j@|jj7�dS(Ms
              --no_defaultROspythia_card.datNR5RGRts>pythia-pgs require event_norm to be on sum. Do not run pythia6i����R�s
                                                                                                                               crossx.htmliR9spythia-pgs_pathRKR1R�sRunning PythiaRXs
         pythia.dones^\s*LHAPATH=%s/PDFsetsRNs
%s_pythia.logspythia_events.hepRcsyst.datRksbeforeveto.trees
xsecs.trees
           events.treesmass_width_2004.mct
                                           PDG_MASS_TBLR�sunweighted_events.lheR�RMR�R�s4Fail to produce pythia output. More info in
     %ssl\s*I\s+0 All included subprocesses\s+I\s+(?P<generated>\d+)\s+(?P<tried>\d+)\s+I\s+(?P<xsec>[\d\.D\-+]+)\s+IR?tDR�ge��ARERFRhiRiRjR2ii'R�R�R[R�s)# Matched Integrated weight (pb) : %s
s%s_%s_banner.txts_pythia_beforeveto.tree.gztPythias_pythia_syst.dat.gzs
                                                                            syscalc.dats%s_syscalc.dats%s.gzspythia_events.lhes%s_pythia_events.lhe.gztfinishR0R1spgs --no_defaultR?R
R�sdelphes --no_default(ARzR_R`RaRR%R7R4RRbRTR�R,R R2R�R&RR�R�R[RR\RKR/R'R5RGR�R�RR�R�RRRSReR�R�R.R�R�R�RZRXR%R�RR>R�R
R&RR�R�R^RAt
                                                                                                                                                                                run_hep2lheRVReR?R
                    RER@R�R>(R$RJR�Rlt
pythia_srcR�R�R
                R�RHR!R�R�R�R2R?tpydirR�R^R_RCfilename((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt do_pythias�
                                                                                                                                                                      !
 
 6 
!

cC s|j|�}|j|�\}}}d|krC|jd�n|dkr<tjjt|jdd��r�j d�q<x�t
j
 tdd�t|jd��D]�}|j
d�}|jr�|jd �ny(|jd tjj
|d
   j|�f�Wq�|jk
r0}j|�q�Xq�Wd
|��ny+|jj|�|jd|ddjd|��ss|jd
                             �Wn
tk
r�t
jd�nXt
j
 dt|jd|��t
j
 dt|jd|��7gD]$} d| kr
                                  tjj| �^q
                                                     |r�gD]} || krC| ^qCd|ksyd|kr�y+|j|dd|kr�td�nWntk
r�q�X�}
tjjt|jd|d��r�jd�ntjjt|jd|d��r,jd�ntjjt|jd|d��rrjt|jd|d��n|
�kr�j d�q�q�nd|kr�n�d|kr�gD]} d| kr�| ^q�ndkrgD]} d kr�| ^q�nd
|krEgD]} d
| kr$| ^q$nd|kr�gD]0} d
| ks�d ks�d| krX| ^qXn|j
                                                r��r�d
d
j�}
                                                                  |j
|
                                                                      dd dd!g�}
                                                                                    nd}
                                                                                         |
��Wq�tkt|jd|| dkr�x�D]�}
��Wq�tktj
t|jd||
��q�Xq�Wnd|ks�d"|kr9y1|r
|j|dd|kr
td�nWntk
r0q9Xt
j
 d#|t|jd$��t
j
 tdd#|�t|jd$��7t
j
 tddd#|�t|jd$��7|js��dkr�d}
                                        n1d
d
j�}
                                                    |j
|
                                                        dd dd!g�}
                                                                      |
�q
Wq9nd|krt dkr9x
D]}
j
 dt|jd|��|r�y-tj
t|jd|d%||f��Wn
tk
r�j d&�nX||jkr|jj ||�d
                                     Sqt!gD]
}dtjj|�k^q��rrt"d'|�qqt
jt|jd|��||jkr|jj |�d
                                                                                                                 Sn'jd(||rd)|pd*f�|jj#|||�|jd*dd�d
                                                                                                                                                                     S(+s%Remove one/all run or only part of itR[R.RXsAA run with name all exists. So we will not supress all processes.Rws
                                                                                                            *_banner.txtiis-fs
                                                                                                                                 remove %s %sRNsNo run '%s' detecteds
                                                                                                                                                                        Cleaning events.lhe.gzsunweighted_events.lhe.gzsplots_parton.htmlsEBe carefull that partonic information are on the point to be removed.R9R:R;s2Do you want to delete the following files?
     %ss
    RtchoicesR�R�s%s*RSs%s_%s_banner.txtsfail to remove the banners�Some output still exists for this run.
                Please remove those output first. Do for example:
                remove %s all banner
                sQThe banner is not removed. In order to remove it run:
    remove %s all banner %ss --tag=%s R($RzRZR�R_R`RaRR%R RZRR�R{R~RR�R�RR!tresulsR�R5R6RGRHR�R/RR�R4R�R�t
delete_runR�RR(R$RJR�R�R�R�R*R2t to_deleteR�tnb_rmR�Rtfile2rmR((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt do_remove�s�
                                                                                                                                                                                
%1%(
   $$$"
                    
                       (
                          (
                             (
                                
                                   !
"/2 
 
 
1
!cC s�|j�|j|�}|j|�tjd|j�|js_|jdg|d�nt g|D]}|dk^qi�r�t
|j
  d|jd�}t
j|�r�tjdj |d
�nt
j|�r�tj|t
|j
  d��|jd�t
j|�sItjt j
|j
  dd�d
        d |�q�y-t
                             jt
|j
  dd��t
           j|�Wq�tk
r�q�Xq�tjd
                  �nt g|D]}|d
k^q��r�t
|j
  d|jd|j�}t
j|�rtjd |�nt
j|�rutj|t j
|j
  dd��|jd�tjt
|j
  dd�d
        d |�q�tjd�nt g|D]}|d
k^q��r5t
|j
  d|jd|j�}t
j|�r�tjd |�nt
j|�r%|jd�tj|�q5tjd�nt g|D]}|d
k^q?�r�t
|j
  d|jd|j�}t
j|�r�tjd |�nt
plot_card.datR\R.R�RXsunweighted_events.lhes.gzs%s.gzRLRMs No valid files for partonic plotR9s%s_pythia_events.lhespythia_events.lheRZs
No valid files for pythia plotR:s%s_pgs_events.lhcoR�so valid files for pgs plotR;s%s_delphes_events.lhcoR�sNo valid files for delphes plotN(R.R�(R.R9(R.R:(R.R;(RCRzR`R R!RbRR[R7R�RR%R_R`RaRRORRR@RVR4RGReR�R�(R$RJR�R'R_((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pytdo_plotNsf
cC s�|j�|j|�}|j|�|jdkrFtjd�dStjd|j�|dg|dt �t
j
 t
dd��|_tg|D]}|dk^q��rut
d |j
|jd
   �}tjj|d
|�ntjj|�retj|t
d |j
d
 ��|jd �tjt
d |j
d
|�qutjd�ntg|D]}|dk^q�r�t
d |j
|jd|j�}tjj|d
|�ntjj|�r�tj|ttjd
d |j
d��y|jd�Wn'tk
rQ}tjt
|��dSXtjt
d |j
|�t
d |j
|jd|j�}tjt
d |j
d�d|�q�tjd�ndS(s6Evaluate systematics variation weights for a given runis(SysCalc can't be run for decay processesNs"Calculating systematics for run %ss
                                                                                                                                                                       run_card.datR\ROR.R�RXsunweighted_events.lhes.gzs%s.gzRMs0No valid files for parton level systematics run.R9s%s_pythia_syst.datsyst.datRZs%s_syscalc.dats
                                                                                                                                                       syscalc.datsNo valid files for pythia level(R.R�(R.R9(
RCRzRgR|R R2R!RbR[RR\R�RR%R,R�R_R`RaRROR�R�R?RVReR
                                                                                                    RZRE(R$RJR�R'R_R2((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt
do_syscalc�sJ
%#
dS|jsdS|jd}|jddddt�d|jkrtjt |j
d|j��
       r�tjt |j
d|jd ��r�t
               j
                d
jtt |j
d|jd ��ntjt |j
dd
  ��rtjt |j
dd
  ��qnd
dd |jkr�|jd
   dt�t |j
d�}|j}|}|jjd
jt |d�dt |t|�d|��nd|jkrZt |j
d�}|j}|}t ||d|�}|jjd�tjj|�rZd|jkr(tj|�qW|jddd
j|d|�qZn|jddd dt�t
                    dtdt�|jj�g|_dS(su tar the pythia results. This is done when we are quite sure that
        the pythia output will not be use anymore NRes
storing files of previous runR0R2R�RXsunweighted_events.lhe.gzsunweighted_events.lhes+gzipping output file: unweighted_events.lhes
           reweight.lheR9s$Storing Pythia files of previous runspythia_events.hepRMs%s_pythia_events.hepRUs%s_pythia8_events.hepmcR�s%Storing Pythia8 files of previous runR�R1(RbRXR,R5R6R7R_R`RaRR%R R!RRVR4RER
RR/tsave(R$R�RR�R�t file_path((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRC%!"   
R
K sg|D]} t| �^q}|d'r7|j}ntjj|�rstj|tj�
                                                               rstd|�n]|r�tjjt ||��r�tjt ||�tj�
                                                                                                                   r�tdt ||��n|dkrw|j
|d|j
     |d|fdd'dt
�} �t
tj|g|d|d|�}
�| tj|t
f�|
        rtd |�qn�|d(krtjj|�}
                                          d
                                           |
                                            kr�d
j|j��tjd�}tjd�}yt
|�}Wn&t
k d
d
|d
|�qd
|�n|jj%|d|d|dr�|j#|�j$�d}|d�}|j|�}dj |�j!�}x�|D](}|jd|�|jd|�q�WnMxJ|D]B}tjj"t ||��r!|
             kr�d
j|j��g}g}dt&t'|d��}|d kr�d!}ng|dD]}d"||f^q�}x|D]}|jd|�q�Wx�|D]�}d#|dkr8t&t|d�j!d#�d�}d}dksf|t&t'|d��kr�tjjt |d
|d
|�qd&|t||d$��r�y
tj(trV|||d$��Wq�q�Xq�q�W|jj+|d|d|d%|d�q�q�|
              krd
d
|d
|�q|jj,|d%|d|d|�nd'S()Rs
input_app.txts chmod +x %s iiR0RR�RMs%s run in %f ss-%s didn't stop properly. Stop all computationiRR�s
              symfact.dats iproc.datdname.mgRSrandinitsMadLoop5_resources.tar.gzs\s*j=(G[\d\.\w]+)sgrid_directory=(G[\d\.\w]+)sfor i in ([\d\.\s]*) ; dosG%ssG%s/results.dats%s/results.dattftn26R�R�R�R4s_%st_0RsG%s%sRyRnRisrefine_splitted.shN(ii(-RER6RR_R`RataccesstX_OKR�RR5RRR�RR�R R!RR�R%R.RR&R�tget_pdf_input_filenameR�RRRGR�R"R�RBRcR�tgroupsR�R%R>R4R�tlexiststcluster_submittsubmit(
R$texeR�RMRiRRR�R(R'RRDtexenameR�R�R�R/torigreR}R�tIreRwtnbR6tG_gridtsuffixR�toffset((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR#�s�
                                                                                                              
                                                                                                               (!

                                                                                                                          
                                                                                                                           $
                                                                                                                              %!
                                                                                                                                       
 


          





*(
           !
 
 
!
cC sRt|jdd�}t|�j�}tjd|tjtj�rJdSdSdS(s(Find if Madevent is in Group mode or notR)srun_config.incs
s*parameter\s+\(ChanPerJob=2\)R�tv4NRR%RR�R�R�R�R�(R$RkR�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pytfind_madevent_mode�s
R$ s[tj��|dr$�j}n|dkrW|rc���fd�}���fd�}nd�}d}y �jj�j|d|�WqWtk
r1}t|��j s��j
ddd d
d
 g�}n}|kr�jj
                     �n(|d
                           kr+�j
                                   d
kd|�S�qWt �d
rS}�jj
            ��qWXndS(s% monitor the progress of running job ic s+�j|||�fdddtd��S(NR0Rt starttime(R5R6R(tidleR�R[(RR$R�(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR>�s
c s+�j|||�fdddtd��S(NR0RR�(R5R6R7(R�R�R[(RR$R�(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR>�s
cS sdS(N(R6(R�R�R[((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR>�Rt
                                                                                                                                                                                update_firstsUCluster Error detected. Do you want to clean the queue? ("c"=continue the run anyway)R�RRaR�R�RR�RN(R�R6RR.RR%RGR R!RR�R4R$R9R$RR�RR5R�R2R((RR$R�sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR$�s2
                                                                                                   
                                                                                                    
                                                                                                     
    
         
              
c C s}i|_ttjjt|jdd��tjjt|jdd��g�}|j|kr�|d�r�|d�r�|j r�|j j
|�ndS||_|j
�|jd ddd t
r|rtjtjj|jd
                      ��t|jd
<ntj�|_|j�}|d
�r�t|jdd�}t|jddd�}tj||�tj|�n|j�tjdgdt|jd��|j
ddkrd|jd<|j
t|jd��|j
�}t|j
d�g}|j ||�n|j
ddkrKd|_!d|jd<n|j
ddkr�t|j
d�|_"d|j
d<|j
j#t|jdd��ttjjt|jdd��tjjt|jdd��g�}||_n�tjj$t|jdd
��rax^t%t|jdd
��D])j&d
�} t| d
�|_"Pq1Wn|
j'd
d�|_"|j
d d!kr�d"dl"}
|
j(|j"�n6|j
d dkr�d"dl"}
|
j(|j
d �n|j
d#d$kr
                       t)j*d%�|j+�n|j,�d|_-|j.d�t)j*d&�x<d'gD]1}
                                                                            |jd(|
                                                                                  gdtjj|jd��qBWtjj/|j
d)�}
                                                                                                                        |
                                                                                                                         j0�d*kr�d+}
                                                                                                                                      ntjj$t|jdd,��r�tj1t|jdd,��ntjj$t|jdd-|
           d,��rFt2j3t|jdd-|
                              d,�t|jd��n|j4d)|
                                                    kr�tjj5t|jdd.��r�tj1t|jdd.��n|j
d)d+krLt)j6d/|
                                                                                                                  �tj7d0d1gdtjj|jdd-|
j;�krLt<d6| �d2t8j9d3t8j9�j:�d}
f��qLn|jd(gdtjj|jdd-|
                                ��|
                                      |j4d)<|j4j#t|jdd7��gt%t|jdd8��D]}|j=�^q�}xKt>|�D]=\}}t|jdj=��}|jd9gd|�q�Wttjjt|jdd��tjjt|jdd��g�}||_dS(:s+ All action require before any type of run ROs
                                                               run_card.datsparam_card.datR�R,Nscompile directoryR0tupdate_resultsR�s
MG5_param.datt clean4pdfR�tpdlabeltlhapdfR7R�tlhaidRtiseediRSRlR$ii�ut crossx.htmlR�smssm-R)R�s
                                                                               python_seedi����i����RkisRunning with CKKW matchingscompile Source DirectoryR.R'R�R�R�tbias_dependenciesR�s libbias.as
Compiling the bias module '%s'tmaket
                                                                        requirementsRMR�tVALIDtINVALIDs2The bias module '%s' cannot be used because of:
%sR9s
launching_dirR�RFR�R�R�tcheck_nb_eventsRR,tR�Rtgetcwdt
                                                  link_lhapdftget_lhapdf_pdfsetsdirR%tcopy_lhapdf_settpdffileR�RRRaRRBtrandinttseedR R!ttreat_ckkw_matchingtupdate_make_optsR,R�R�R�R4RRR9R
R\R�R�R�R�R�RR
R
(R$R5time_modR�R+R�t
pdfsetsdirt
lhaid_listRJRwR�Rxt bias_nametbias_module_validR
R
  R(R)R((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s� !'-  
                                                                                                                                          

!' 
       !"
                   
 
/"!'cC s
tjj|�r|S|SdS(sScheck if the directory exists. if so return the path otherwise the
        defaultN(R_R`Rc(R`R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt check_dirXsc
C s�t|d�r�|jr�|j|jdkr�|r�|s`ttj|jdj���Sttj|jdj���|jdfSq�|s�|jd|S|jd||jdfSq�n|�}ig|j6}i}x�|D]�}g||<x�t t
|jdt
      j
|�d��D]b}|j�\} t| �dkr+||jt
|d��| |t
|d�<q+q+Wq�W||f|_|j|d|�S( s,get the list of Gdirectory if not yet saved.R,iiRSs
                                                                                                symfact.datsG%stsymfactN(R3R,R%R6Ryt itertoolstchainRuget_PdirRRR_R`R�RBR%R�RP(
R$RR�tPdirsR,mfactorstPRJR�tmfactor((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRPes*
                                                                                                                                          0"
 
1
"c
C s�d�}i ddddddgd6ddddgd6dddg6dgd6dgd6dgd6dgd6gd 6gd
6}||jkry|r�t|jd
                     d
<||_|j|j|j�nqxn||D]b}t|_|j|jd|�r|j
<||_|j|j|j�PqqW|||�S|jr�|j
                                      �n||_t
|j||�|_d|jkr�|jjd�|_n't|jd } tj
                                   d
<t} n�|j|jkrD|dkrDn�|j|jkr�tjd�|j||j�|jjd|ddt
                                                                     �n�xR||D]F}t |j|jd|�r�|j
<n|rN||jkrN|| s
|j|jdd}||jd
|_|||�S(s=define the run name, the run_tag, the banner and the results.cS s�|dkrdS|d
kr2|j|jddSxftt|j|j�
                              d �D]=}|j|j|}|js�|js�|jrZ|dSqZWdS(
                                                                               NR�R9RUtmadanalysis5_partontmadanalysis5_hadroniR�i����i(R9RUR�R�(R/RbR�RR9R�RU(R$R0R�ttagRun((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt
                                                                                                  get_last_tag�s
                                                                                                                   
                                                                                                                      .R�R9R:R;R�R�RUR\RbROs
                                                                                                                                                            run_card.datRei����RVR,s"Trying to run data on unknown run.s
add run %sR.R1R�(RbRR%R\R�R,ReR/tadd_runR�tget_available_tagRCRRKR[t
                                                                             charge_cardR7R RZtupdateR�(
allow_new_tagR�t
                 upgrade_tagR,tnew_tag((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��st








 



cC sst|jd�}|dkrotjd�tjd�t|jdd�}tjd|�d|jd<nS( sWFind the number of event in the run_card, and check that this is not
        too largeR3i@Bs*Attempting to generate more than 1M eventss;Limiting number to 1M. Use multi_run for larger statistics.ROs
                                                                                                                                   run_card.dats=perl -p -i.bak -e "s/\d+\s*=\s*nevents/1000000 = nevents/" %sNR%R,R RZRR%R_R�(R$RHR`((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s
 cC sq|jd7_|jd kr;tdt|j�d�n|jddkrmdl}|j|j�nS( 
s change random numberii�usRandom seed too large s > 30081*30081R�i����i����Ni+�5(R�RRER,R�(R$R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�s
                            cC s6tt|jdd�d�}|jd|j�dS(s&save random number in appropirate fileRSRlRhsr=%s
N(RRR%t
writelinesR�(R$R}((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR s
cO stjj|||�S(N(t
common_runt
C s�|jd}|jd}|jd}|jd}|jd}|jd}|jd}i|6|dme/s6yo6|dy/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�sc
6|d
    6|d
�r�|d }n|dks�|dkr�d}n|dkr�d} nd} |jr&t|j| �} nt|j| �} td| �t j
j
 | �r�t|jdd�}
t
|j dtd|
�q�d}
          t|
            �|jddd
�xctd
d
�D]R}
                                         |jjd|jd d!|
                                                         gd"|jdtt|jd#|
                                                                             �d$��q�W|j�x\td
d
�D]H}
                                                                                                               t|jdd�}
t jd%|j|
f�t
       j|
d| �q6Wnd&S('scheck for ckkwR�R�R�R�R�R�R�te1te2tpdtlhatxqR�ii����iR�s8lib/issudgrid-%(e1)s-%(e2)s-%(pd)s-%(lha)s-%(xq)s.dat.gzs0lib/issudgrid-%(e1)s-%(issudgrid.datRLRMsXNo sudakov grid file for parameter choice. Start to generate it. This might take a whilesGENERATE SUDAKOV GRIDR0R�i����is%s/gensudgrid Ris%dR�sgensudgR$R�R�R�R�R�R�R�tsudgrid%s.log >> %sN(R,RFR}R�RtwebbinR%R R!R_R`RaRROR7R5R�R.RuR�RR$R�RV(
 translationt issudfileR`tmsgR�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyttreat_CKKW_matchingsN
  
 $  

sunweighted_events.lhesunweighted_events.rootcC s�|jddd�|jd}t}|jd�r[tj|dt�t}|d }ny0tj|||gd |j d
��Wn
t
k
r�t
      j
       d
jjd �nX|r�t
S(screate the LHE root file sCreating root filesR0R�R�s.gzRLi����s%s/ExRootLHEFConverterR�RXs9fail to produce Root output [problem with ExRootAnalysis]s%s.gzN(R5R&RR}RROR7R�RR%RGR RZR_R`RaR4RV(R$R
totar((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRBMs(
cC s�|jd|jkrdS|jd}|
                                             sAtjj|�
                                                        rEdS|jddkritjd�dStd|�|jd kr�d
|j<n|jj �g}d
             |jd
j krd
|jd
   �}|j
           d
            �}|g|D]j
                      �d^q�7}nI|g|jd
                                      j
                                        �D]+j
                                             �
�dkr(^q(7}y|j�} Wn4tk sMt
r�}
tjt|
��tjd�dSXg|D]
                      |j|
                          g| �^q�i|jdd6|jdd6|j6|jdd6|jd
                                                                        d
                                                                          6}
t|jddd�}tt|jddd��j�}d |jd}
                                    |
                                     d
                                      kr�|
                                             d
                                              j
                                                dd�djd
                                                           d�|
                                                                d
                                                                 <n�|
                                                                     d
                                                                      j
                                                                        �}g}x`|D]X}|j
|�dkr�|j|�q�|d
cd
|7<q�Wd
j �s�|j|�q�t
|�|
     d
      <|
        d
         j�dAkrHd|
                       d
                        <n|
                           dj�dBkrkd|
                                           d<nd|jkr�d
|jd<nt|d$�j|j�tjj|�s�t
St|jd%�}|s�|d&krdt||j
d'�}tjj|�ptjj|d(�s0t|d'�}nt|d)�}t|f�d$�}d-|j
krgtj d.tj!tj"�}|j#|j
d-�}|rgt$|j%d��}t&|jd/�}xZ|jj
                                                                                          �D]B}t$|�|kr?t'd0�nt$|�|krt'd1�qqWqgnt|d2�}t|d3�}q�|j(d4|�ntjj|�s�tjj|d(�r�t)j*|d(�q�t'd5|�n|j+d6|d7|j��yPt)j,tjj
|d8�|||gd9|d:t-j.d;|�}t/j0d<�Wn$t1k
rz}
tj2d=|
�nBXtjj|�s�tjd>�n|d&kr�t3j4||�n|j+d?|d7|j�d@t
�t5S(Cscreate the syscalc outputRcNRaRGRts/SysCalc works only when event_norm is on 'sum'.srunningsys_scalefacttde %sR�R8s30 50s&&tsys_pdfRii�s=Systematic computation requires lhapdf to run. Bypass SysCalct
              sys_alpsfacttsys_scalecorrelationReRR�ssyscalc_card.datssyscalc_template.datR�is
 i����s %ss
RR�tfalseR�s.false.RhRXR�sunweighted_events.lhes.gzs
                                                         syscalc.lhes%s_systematics.logRZs%s_%s_syscalc.logt
                                                                                                               mgpythiacards
^\s*qcut\s*=\s*([\+\-\d.e]*)R�sLqcut value for sys_matchscale lower than qcut in pythia_card. Bypass syscalcsJqcut value for sys_matchscale lower than xqcut in run_card. Bypass syscalcsyst.dats
                                                                                                                                              syscalc.datsInvalid mode %ss
Events file %s does not exitss$Calculating systematics for %s levelR0R�RMR�R�isJfail to run syscalc: %s. Please check that SysCalc is correctly installed.sgSysCalc Failed. Please get_lhapdf_idR�RBRR%R�RGR\RERZR�RR%RR�R
R�R�RRRRbR[R�RR�R�R�R>R�R�Ryscalc for %s levelR1(RR�R�R�s.false.(RR�R�R�s.false.(6R,ReR&R_R`RaR R<R!t
                                                                                       RRROR5R�R�R�R�tsleeptOSErrorR2RR�R7(R$R�R@R�RfR�RJR�R
t
  pdfsets_dirR2tonelhat
$-*Itdt event_dirRMtpatR�R�R�tproc((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR?js�


              (%
                        (

                                          
  
                                                                                          
c C sg}d|ks
d|kr.|jd�nd|ksFd|krV|jd�n|jdgd |jd
|d
   |d
|dt�\}}||_d|kr�d}nddg}|ddkr�|jd�n|ddkr|jd�n|dd/kr%|jd�n|dd0kr�|jd
�t}tjt |j
d
d��r�t
             }|jd �q�n|d!d"kr�|jd#�n|d$d"kr�|jd%�n|d&j
                                                                          �d1kr�|jd(�n|d&j
                                                                                                 �d2kr |dd"k
                                                                                                                  r |jd)�n|d&j
|�|j�tjjt |jd,d-��r�|jd-�n|jr�|jt |jd,d��|Sd|kr�|dr�|j|d.t �d3krF|jd+�n|j
                                                                                              d
�n|j|d.t
�|S(4s9Ask the question when launching generate_events/multi_runs-Rs
--reweights
           reweight=ONs-Ms --madspins
madspin=ONRt0t ask_classR�R�Rt first_cmdtreturn_instancet dynamicalR8sparam_card.dats
                                                                                                            run_card.datR�R�spythia_card.datR�spythia8_card.datR�R�s
                                                                                                                                                                     DELPHES+PGSs
                                                                                                                                                                                pgs_card.datR�sdelphes_card.datR�Rwsdelphes_trigger.datR�R�smadspin_card.datR�sreweight_card.datR�t
                                                                                                      MADANALYSIS5s
madanalysis5_parton_card.dats
madanalysis5_hadron_card.datt
                                                                                                                                                                               MAplot_card.datROsMadLoopParams.datR\(R�s
                                          DELPHES+PGS(R�s
                                                          DELPHES+PGS(R�(R�(R�(R�R�taction_switcherRR7R�R_R`RaRR&RR�t
keep_cardst
           get_cardcmdR
R%R�R[( R$R�R�t
                                               passing_cmdR�t
cmd_switchtcardsdelphes3R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�sZ 
                                                                                                                                                 
"' 
 ! 
"ic
C s|dkrdnd|}ddg}|dkrD|jd�n|jdra|j�nid d6d
d6d
    6}d
} |dkr�||D]}|d7}q�d| d7} n
| d7} |kr�| d|7} n|js%|s4|j| d�}q4n|s4d }n|j�rM||}nt}
|d kr�t}
|dkr�tj t
|j
  dd��r�d
}q�tj t
|j
  dd��r�d
              }q�d
                    |}nt
d|�d|g} j
                |d
kr|dkr|
               jd�n|d
                         krr|
                               jd�t}
                                        tj t
|jdd��rrt}
                |
                 jd�qrn|j|
                              dddd
g�|jr�|S|s�|j}n|
r�|j|
       d
d
dgd d d!|dkd"|�n+|j|
                                              d
d
dgd!|dkd"|�|S(#s&Ask the question when launching pythiaiRs%dR�R�t2R�t3R8R:R;pythia%ssLWhich programs do you want to run?
    0 / auto : running existing cards
s 1. pythia : Pythia
s
    2. pgs : Pythia + PGS
s 1. pythia8 : Pythia8
s& 3. delphes : Pythia%s + Delphes.
ROs
plot_card.datttsdelphes_card.datsWill run in mode %sspythia%s_card.datRwsdelphes_trigger.dattignores
madanalysis5_parton_card.dats
madanalysis5_hadron_card.dats
pythia_suffixtavailable_modeRxRGR&R�R8R�R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�4sl

                                                                                                                                             
! 
          


   
 

 ! 
"
 
             
(ZR
R
 RjR7ReR�R�R�R�RR�RdRXRqR
                                   t_initMadLoop_optsRRR�RtqueueR6R�RkR�tasking_for_runR#R�R�RNRRR�RMR
                                                                                                                      RRR+R/R�tscanparamcardhandlingR-RARbRcR>R~R�R�t
     staticmethodR�R�R�RRR3R<RJRWRuRvR{R}R�R�RDR�R�R`RgRhRiRCR#RR$R�R�RPR�R�RRR�R�RBR?R�RR�(((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�s�
                                                         
                                                           






6 7
@,
                         3 q +
 H* * T S�" G
 z  V A `   H � ��� $  � � D1 3
' �
  �
       
         e 
                           8

�?tMadEventCmdShellcB seZdZRS(s&The command line processor of MadGraph(R
R
i|_dS(N(tfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��sRScB sYeZiZed��Zed��Zed��Zed��ed��Z RS(cC s
           name_to_pdg(tcls((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�scC sg}xrtt|dd��D]X}|s1qn|j�}tjjt|d|��r|jt|d|��qqW|S(sreturn the list of SubprocessesRSs
subproc.mg(RRR
R_R`RaR�(R%R
                              RJRx((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s
                                                                                                                                        
 cC sd}i}d}tjjtjj|d��s@tj|�Sx�ttjj|d��D]�}|d j�}|dkr�|}n|dj�dkr�q\n|�}|}|d}||j d�7}||kr||c|j
d�g7<q\|j
processes.dattmirrorR�R�i(turn the list of processes with their nameiRs
                             R_R`RaR�RStget_subP_info_v4RR
R%tcountRB(R`tnb_subtnamesold_mainRJtmaint
                                                                                                           sub_proccess((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/inte"ubP_info�s&!yt
      
             

   cC s�d}iggd6}tjj|d�}d}xYt|�D]K}|jd�ry|d7}|ddj|d�q>|dkr>Pq>q>W|S(sG return the list of processes with their name in case without grouping iRs
        auto_dsig.fsC Process:ii(R_R`R�RRFR�(R`R�R�tfoundRJ((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s


leshouche.inctIDUPsd��D]b}d|kr4q
ntjd|�}|jg|jd�jd�D]}t|�^qe�q
W|S(sAreturn the pdg codes of the particles present in the Subprocessess
                    /([\d,-]+)/iR�RRR�R�R�R�RBR%(R`tall_idsRJR�R((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��s

                                                                                                                                                                           <(
R
R
 R�t
     classmethodRR�R�R�R�R�(((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRS�s
                                                                                                                             
RIcB s}eZdZddddd�Zd�Zd�Zd�Zd��Zd�Z
d
�Z
  d
   �Z
     d
�ZRS(sNThe command for the gridpack --Those are not suppose to be use interactively--ii����cO s�t|_tj||||�d|_||_|j|_||_t|d<t |�|_
|j
  |||�|j
||�n'tdt|�t|�t|��dS(s'Initialize the command and directly runiR�sGridpack run failed: N(RreadonlyR�R#R1R�R�t
                                                                                                                                    granularityR&R%RHtwrite_gridcardtprepare_local_dirRRRE(R$R%RHR�tgranR�R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR#�s
   
  
cO sdS(N((R$R�R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR5�scC s=|j�}|j}tj|||j�|_d|_dS(sload the current results statusRN(R�tprocesstgen_crossxhtmlt
AllResultsR%R/t last_mode(R$R�R((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR��
                                                                                                                              cC sQ|js*tt|jdd�d�}ntdd�}|jd|j�dS(s&save random number in appropirate fileRSRlRhsr=%s
N(R�RRR%R�R�(R$R}((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR !cC s;ytt|�j|�Wntk
r6t|_nXdS(N(R"RIt
cC s�dG|jGH|jrItjjd�s7tjd�ntdd�}n
tt|jdd�d�}j/�}te/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR
|d<||d<||d<||<|j
                    |�d S(
grid_card.datRhtGridRuntgeventstgseedtngranN(locationsWRITE GRIDCARDROs
                                              R%R�R_R`RaRdRRR\t
                                                                  GridpackCardR7RR(R$RHR�RR}t
                                                                                                  gridpackcard((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�s
                           

                                         



cC s�t|d�r/|j|jdkr/|jSn|js�gtt|jdd��D]!}t|jd|j��^qT|_n7gtt|jdd��D]}|j�^q�|_|jS(s,get the list of Pdirectory if not yet saved.R�iRSs
subproc.mg(R3R%R�R�RRR
(R$R
((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�(s
 F4cC sY|js
tj|j�n9x6tt|jdd��D]}tj|j��q8WdS(s7create the P directory structure in the local directoryRSs
subproc.mgNR�R_tchdirR%RRRdR
(R$RJ((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR8 "cC sAtjd|�|jd|�|jso|jddd�tj|j ddd�gd |j �n|jd
dd�|j
|�|j
       d
        �|js|j
                   d
                    �|j
dkr�|j |j
        d|jdtdt�n|j
                            ddt�n,|j
                                          dd|jd�dtdt�dS(s$ launch the generation for the grid sgenerate %s eventss
GridRun_%ssrestoring default dataR0RR�t
                                          restore_dataR�R�sGenerating EventsR�R2R4R5ssystematics %s --from_cardR?R
sdecay_events -from_cardsRXsunweighted_events.lhe.gzN(R R!R�R�R5R6RR�RR%t
 cC s�|jd7_|}tg|jj�D]
\}}||df^q(�|_d|_|j�|jd|ddMG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRBs& 
�t j
d|j
   �i|d6t
6|jd 6}tj||�}|j�|j|_d
Stj|�}|j|j} |jjd
                              �|jjd
                                       | �d
��D]<}jt
|j
d
|
|
d�rStjj t
|j
d
��rS|
^qS}
      t!tj"d�}
                x t#|
|�}t
tjj%|j&|��}tj]\}
d|�xOt'j(d|�D];}tjj)|�d d*kr#tj*t
||��q#q#Wt
|d�}t'j+t
|d�gdt,j-dt!|d�d�tjj.t
|d
��r�t'j(d
|�}t/|�}|j|7_x�t#|�D]�\}}tjj)dt/||j0d
|dd||dd d!|j||
      �f�tjj.t
|j
d
                           ��r�|j1d"t2�t3d#|t!t
|j
d
                                                         ��j4�f�q�q�Wq�q�W|j1d d$|j�|jd%dd&�ytj*t
|d'��Wnt5k
���}t6j7|j
�|j8�\}d |jjd
                                      �|jjd
                                               | �|jd(d&d)t
                                                                    �|
                                                                         j9�d
S(+s Special refine for gridpack run.iisRefine results to %sR0s$Using random number seed offset = %sRRR
                                                                                                             R�NRR2RSR�Rhs %s s*ajob*iRRsrun.Rsgen_ximprove.lofinish refineR1(RRsrun.R(:R�R5RR�R�RRR5R6R R!R�RR�R�R tgen_ximprove_gridpackRtAll job submitted for refine number %ssCombining runsR�scombine_runs.logs
gscalefactR%tcollect_resultR?R@R/R'RR_R�RR%RFR`RcRR�R
R
R!R�RR�R�R4R�R�R�RaRR#R$R7R R�RGR$R%R RS(R$RHR'R�R�R&R+PresultsRR2R�R
                                                                                                                                                                R�R(R)RR-R*tlogfileR.R2R�R0((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRcs�5 


  "<
                         


 
                                                       $!
3
                                                                             
C s�|jr4d}tjj|�sFtj|�qFnt|jd�}|j|�}||�|j }|j 
d}|j
      s�t
|jd�|_ j
        n|j
           j|j�|j
                    j|jjd|j
d�t|d�s�d|_n|j
                     j|j�tjjt||j��s;tjt||j��n|j
                                                                  jt||j|j|f��d �}tj�}|j
                                                                                                   |_
                                                                                                     ddgd} }
}
 |j�}
      |
       j�x�|
djjt| D]�}
��r�tj
d
d �}|j
t|
d��|j
t|
7}||�|jd
�|�||
7} |jd�||
    t!|�dkrk|j"t|jd|jd�|ddddd|j#�tj�}|j
                                                           |_
                                                             |j
t|jd|jd�| t$j%t&d�|
D���|
        �d7qkq�q�Wt|d�s�|j'�|_(n|j
                                            j| |j#�|j"t||jd�|ddd|j#dt)j*dj
d
d
|j(�}rmxot+�D]^}y$tj,t||jd|��Wt-k
retj,t||jd
|��XWn|jj.d|�|j
                                           j| |�|j
d j/�d$kr�|j0�nd#S(%s(Advanced commands: Launch combine eventsRXReR�RR3R�is%s_%s_banner.txtcS s|jS(N(R=(R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR>�Rs
events.lheRs
             results.datR?R@RAiPspartials%s.lhe.gzRBiRCg{�G�z�?RDcs s|]}|dVqdS(iN((R�R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>siREsunweighted_events.lhe.gzRFRGRHspartials%s.lheRHR�R�R�N(R�R�(1R�R_R`RaRdRR%RzRJRR,R[R\RKR/RLRMRAR3R�RNRbRRR�RORPRQR%RRRSR^R�R�RRTRHR�R�RtRURERVRR�R4RGR'R�RW(R$RJtoutdirR�RR�RYRZR[R\R]R^R,R1R�RHR�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interfce/made$terface.pyRJ�s� 
               

 
                                 
                                  
                                       

"

                          
                           "

+N(R $
R
 RjR6R#R5R�RRR�R�RRRRJ(((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRI�s   
 ! eR^cB s�eZdZeed��Zeeddddd��Zedddddgd gd
��Z ed
           d
            ��Z
edd
��Zdd
     RS(s� A container class for the various methods for initializing MadLoop. It is
    placed in MadEventInterface because it is used by Madevent for loop-induced
    simulations. c
                        C s�tjjt|d��r`tjt|d��tjt|d��tjt|d��nttjd�}tj�}j dddgd|d �}tj�|}|d
kr�t
j
 d
jd|�dS|stj�}t
|d|�}tj�|}d}n�jd
|d|dt�z.j�x
j�r{tjd�q_WWdj�Xj}jj}jj}|j�|d
kr�t
jd|�dS|||fS(sg Compile the check program in the directory dir_name.
        Return the compilation and running time. Rs
loop_matrix.oRhR's-j1R�R�iis Error while executing make in %ss./checkRMR�Rt close_fdsg�������?Ns#Error while executing ./check in %s(NNN(NNN(
R_R`R
RR4RR�R�RRRVR!R6R�R�t
                 ProcessTimerRR7texecutetpollR�RStmax_rss_memorytt1tt0Rt
returncodeRZ( dir_namecheckRamR�Rtretcodetcompilation_timerun_timet ram_usagetptimer((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt
                                                                                                                                                                               make_and_run$s@
                        $
                             
                                    
                                      

     
           

 ii����gc
                C sG|}tjj|�
                                 s2tjj|�dk
                                               r�t|d�}tjj|�s�gtjd|�D]0}td|�rftjj |�rf|^qft
�dkr�tdd�}q�q�ntjj|�s�t
                                    ddt
|d�} | j�} |���nt
|�} tjd d
|rEd
      nd
         |
�}
d||
�}
|dkr�tjdd||
�}
ntjdd|
�}
|dkr�tjddd|jdd�|
�}
n$|dkrtjdd|
�}
n|dkr,tjdd|
�}
n| j|
�| j�d
S(
s{Set check_sa.f to be reading PS.input assuming a working dir dir_name.
        if hel_config is different than -1 then check_sa.f is configured so to
        evaluate only the specified helicity.
        If mu_r > 0.0, then the renormalization constant value will be hardcoded
        directly in check_sa.f, if is is 0 it will be set to Sqrt(s) and if it
        is < 0.0 the value in the param_card.dat is used.
        If the split_orders target (i.e. the target squared coupling orders for
        the computation) is != -1, it will be changed in check_sa.f via the
        subroutine CALL SET_COUPLINGORDERS_TARGET(split_orders).s
check_sa.fsP*_*s
                 .*P\d+_\w*$is)Could not find the location of check_sa.fs
 from the specified path %s.R�RhsREADPS = \S+\)s
                                                                                                                            READPS = %s)s.TRUE.s.FALSE.sNPSPOINTS = \d+sNPSPOINTS = %di����s
SLOOPMATRIX\S+\(\S+,MATELEM,s"SLOOPMATRIXHEL_THRES(P,%d,MATELEM,s
SLOOPMATRIX_THRES(P,MATELEM,gs
MU_R=SQRTSsMU_R=%ss%.17eteR�Rs"SET_COUPLINGORDERS_TARGET\(-?\d+\)s
SET_COUPLINGORDERS_TARGET(%d)N(R_R`R
R�RRR�R�R�RcRRRERR�RStsubR
RR(
                                                                                                                                                    dir_pathtread_pstnpointst
hel_configtmu_rt
                 split_ordersRkR�t
0
 directoriestfilecheck_sa((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pytfix_PSPoint_in_check[sD
                        

     
                 
                  
                    
                      
HelFilter.datsLoopFilter.datiic s(|d!ri}n|d!r=�d!r=tdd�n|d!rjtjjt�tj��}n�d!r�gtd|�D]
}tjj |�r�|^q�}|r�|d�q�tddt
|��ntjj
              t�d��sVt
                          |�kr$|dd kr$d |d<nt
                                                               |�d
krV|d
        krVd
|��t|dt |<qVnt
         �tjj
�n
tj�} tj|td�}
| ds�y�jd�Wq�tk
r�q�Xn| ddkr1y�jd�Wq1tk
r-q1Xn��fd�}
                    tjjt�d��}
| td�|D��s�g|D]
^q{|}n}}| jdt�xA|gkr�|
                                    �r�|j�}|dkr
d}| jdd�| jdd�n | jd�| jdd�t|�}| jd�| jt|d
                                                                                              ��tj�d
t
d
|�tj
��\}}}|d!kr�t
jd
��d!}d!Sd|j �ks�|dd!kr�||d<n||d <q�W|
jt|d
      ��|
            �rd!S||Sd!S("sa Run the initialization of the process in 'run_dir' with success
        characterized by the creation of the files req_files in this directory.
        The directory containing the driving source code 'check_sa.f'.
        The list attempt gives the successive number of PS points the
        initialization should be tried with before calling it failed.
        Returns the number of PS points which were necessary for the init.
        Notice at least run_dir or SubProc_dir must be provided.
        A negative attempt number given in input means that quadprec will be
HelFilter.datco s�tt�d�d�}|j�}|j�tg�D])}tjjt�d||��at %s.t be provided in run_initialization.sP[0-9]*is(Could not find a valid running directorys in %s.s
                                                                         ^q8�p�tjt�d��
                                                                                              p�tj t�d�tj
�
  S(s True if init not done yet.sproc_prefix.txtR�RTR(
                                                          RRR�RSR�R_R`RaR
RoRp(tproc_prefix_filet
                                                                                                          proc_prefixtfname(t
                                                                                                                              my_req_filestrun_dir(sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyt need_init�s
                                                                       
6
cs s]}|dkVqdS(iN((R�tattempt((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>�sR�i����t
CTModeInitit ZeroThresg�dy���=g��&�
                                      .>t
                                          MaxAttemptsR*R+s$Failed at running the process in %s.tProcess_compilationtInitializationN(!R6RR_R`tabspathRR�RR�RcRER
RRyR�R\R]R4R&RaR�R�R7R6R�RRR^R2RR&RVR2R(R8t
                                             SubProc_dirtinfost req_filesattemptstdirR/t
to_attemptt
MLCardPathR`t
use_quad_prectMLCard_origR9tis_loop_inducedR:t
              curr_attemptt
                           compile_timeR#R$((R7R8sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pytrun_initialization�s�
                                                                                                                                                               

 !
""
     

 


   !
                
                 
 RXc C sud�}t|dd�}tjj|�s=td|�ntj|�}ddg}|ds�y|jd�Wq�k
r�q�Xn|d kr�y|jd�Wq�k
r�q�Xnx�t j
t|dd
|��D]�}tjj
                 |�
                     s�tjjt|d
                                 ��
                                     r
q�nt
                                            t|d
�|j�|t|dd�|�r�tSq�WtS(s8Checks whether the necessary filters are present or not.cS s7tg|D]&}tjjt|||��
                                                                                                                       ^q
�S(s5 Returns true if not all required files are present. (R�R_R`RaR(tML_resources_pathR5tr_filesR6((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madeventloop_matrix.fsproc_prefix.txtR�RT(RR_R`R
RR\R]R4R&R�tiglobRcRR�RSR7R(aproc_dirRWR9RGR`RCv_folderR4R5((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR�s: 

  


              

   c
|jd�tjd�t|d��r�tjddt|d��gdt|d��ntj t|dd
��r�tjdd
             gdt|d��ntjdd
gdt|d��tj gdt|d��tjdd
dd�|rBt
            j
             |�}n t
}d�}d�}igtjt|dd|��D]9} tj| �s�tjt| d��r�| ^q�}
tjdt|
�t|
�dkr�dndf�tjdjd�|
D���x�|
D]�}
    i|
      <d
        |d,rf|j|t|
                      �|
                         d,g�q
|j|t|
                                           �|
                                              gtd|
^q�g�q
W|jd|d
|�x�| d�D]}
D]�}
    |
     }|d
d,r.td
dtj|
                          �|d,rd n
                                      d!|
                                         |f�n|d
d"krbtjd#tj|
                                                                        �d$�q�tjd%tj|
                                                                                             �d&t|d
�|d
d"kr�d'nd(|d)|d*f�q�Wtj
d+�d,S(-sgAdvanced commands: Compiles and run MadLoop on RAMBO random PS points to initilize the
        filters.s1Compiling Source materials necessary for MadLoop sinitialization.R'ttreatCardsLoopNoInitR�R)sall --no_MadLoopInitCutToolst
                                                                                                                                                libcuttoolstIREGIlibiregilibmodelt libdhelass2Initializing MadLoop loop-induced matrix elements s
(this can take some time)...cS sS|dr'tjd|d|�}n
tjd|d|d|�}||d<dS(NR8RBRDR#i(R6R^RM(R8RBRDRV((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pytrun_initialization_wrapperds
                                                                                                                                   
cS s6|||dkrdStjd|||f�dS(Nis9MadLoop initialization jobs: %d Idle, %d Running, %d Done(R R\(R�R�R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/iloop_matrix.fs7Now Initializing MadLoop matrix element in %d folder%s:iR*Rs, cs s%|]dtjj|�VqdS(s'%s'N(R_R`R�(R�RR((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pys <genexpr>{siR�R#s
Failed the initialization ofs$ loop-induced matrix element '%s'%s.s
 (using default n_PS points)s( (trying with a maximum of %d PS points)is0Nothing to be done in '%s', all filters already s:present (use the '-r' option to force their recomputation)s'%s' finished using s7%d PS points (%s), in %.3g(compil.) + %.3g(init.) secs.tDPtQPR>R?s MadLoop initialization finished.N(R R\R6RRRR�R_R`RaR!R.t MultiCoretonecoreR�RPRcR
RR�RuR�RRR�R�(RQRVRWRYRZtmcoreRZR�t init_infoR�tVirtualFoldersRRmax_multt
multiplierRj((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR_Bsj 

%
%""   %
 &' ,

  
2
 
                   &N(
                              R
R
 RjR�RR&R7R2R6RMR�R_(((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR^s
63  �* iiisNMadGraph/MadEvent 5 works only with python 2.6 or later (but not python 3.X).
s&Please upgrate your version of python.t
InvalidOptioncB eZRS((R MyOptParsercB s*eZdefd��YZdd�ZRS(t
R
 (((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRe�sRcC stj|��dS(N(RdRe(R$R�((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyR2�s(R
R
 RGReR2(((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.pyRd�ss
usage: %prog [options] [FILE] tusages-ls --loggingR�Rthelps<logging level (DEBUG|INFO|WARNING|ERROR|CRITICAL) [%default]Rs--webtactiont
store_truetdestR�sforce toce to be in secure modes--debugR\sforce to launch debug modeRlogging.R�sme5_logging.confR�t force_runsdo_%ss2and %s can not be interpreted as a valid command.s,ERROR: %s not a valid command. Please retryRR=squit on KeyboardInterrupt(�Rjt
GNU_SPLITTINGR7R`RBRR�R�R
RR�R�R�R�R�RR�R�R�R
t
R�Rt getLoggerR R;R�R�R2R�tinternal.extended_cmdt
internal.miscRR�RRRtinternal.filesRtinternal.gen_crossxhtmlRtinternal.gen_ximproveR tinternal.save_load_objecttsave_load_objecttinternal.clusterR.tinternal.check_param_cardR�tinternal.sum_htmlR%tinternal.combine_runsR$R�R�RRtmadgraph.interface.extended_cmdRZt'madgraph.interface.common_run_interfacetmadgraph.iolibs.filestiolibst madgraph.iolibs.save_load_objectt madgraph.madevent.gen_crossxhtmlR�t
madgraph.madevent.gen_ximprovetmadgraph.madevent.sum_htmltmadgraph.various.bannerR�tmadgraph.various.clustertmadgraph.various.misct
madgraph.madevent.combine_runsR�tmodels.check_param_cardRGR RYR
ControlSwitchR�R�RvR�RLR�RSRIR^tAskforEditCardt tMadEventAlreadyRunningR�R
                                                      version_infoR<optparset
                                                                             OptionParserRdRftparsert
add_optiont
           parser_errorRR�RtargvR�t
parse_argsR&R�Retlogging.configtinternal.coloring_loggingR
RR%R0RMtconfigt
fileConfigtrootsetLevelR"R6cmd_lineR3R@trun_cmdR9(((sQ/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.py<module>sV
                                                                                                                                                          
                                                                                                                                                           
                                                                                                                                                            
                                                                                                                                                             
                                                                                                                                                              
                                                                                                                                                               
                                                                                                                                                                
                                                                                                                                                                 
                                                                                                                                                                  
                                                                                                                                                                   
                                                                                                                                                                    
                                                                                                                                                                     
                                                                                                                                                                      
                                                                                                                                                                       
                                                                                                                                                                        
                                                                                                                                                                         
                                                                                                                                                                            
.
     %
        
"  ���}��� �"�����������������U�B�� 
                                                                                                                                 
                                                                                                                                  ' 
                                                                                                                                         
                                                                                                                                          
                                                                                                                                           *!! ,
^[[?62;c^C^C^C^C^C^Crm jobs on queue
INFO: Stop by the user
stopping all operation
            in order to quit MadGraph5_aMC@NLO please enter exit
INFO: generate_events Stop by the user
^C^C^C^C^C^C^C^C^C^Cstopping all operation
            in order to quit mg5 please enter exit
MG5_aMC>writting history and quit on KeyboardInterrupt
INFO: storing files of previous run
INFO: Done
^CTraceback (most recent call last):
  File "./mg5_aMC", line 191, in <module>
    cmd_line.exec_cmd('quit all', printcmd=False)
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/extended_cmd.py", line 1544, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/extended_cmd.py", line 1464, in onecmd_orig
    return func(arg, **opt)
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.py", line 5905, in do_quit
    return common_run.CommonRunCmd.do_quit(self, *args, **opts)
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/common_run_interface.py", line 3433, in do_quit
    self.store_result()
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/madevent_interface.py", line 5361, in store_result
    self.update_status('Done', level='pythia',makehtml=False,error=True)
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/interface/common_run_interface.py", line 3491, in update_status
    self.results.update(status, level, makehtml=makehtml, error=error)
  File "/gpfs/share/home/s6yokoay/MG5_aMC_v2_6_7/madgraph/madevent/gen_crossxhtml.py", line 294, in update
^C self.current.update_status(level)
"

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:

This question was reopened

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#1

Hi,

The version 2.7.1 should fix this printing issue. (Note that the result is not impacted at all)
2.7.1 is not yet released but should be very soon (probably tonight)

Cheers,

Olivier

Revision history for this message
Koay Yong Sheng (kysheng95) said :
#2

Thanks Olivier Mattelaer, that solved my question.

Revision history for this message
Koay Yong Sheng (kysheng95) said :
#3

Hi,
Thanks for the quick updates whenever there is a problem but unfortunately this still occurs for 2.7.1.2.

Cheers,
Yong Sheng

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#4

Hi,

The problem is that does not show up on all machines.
The problematic line should be in the file madgraph/interface/madevent_interface.py
around line 4666

                            pid = os.system('head -n -1 %s &> /dev/null' % __file__)
                            if pid == 0:
                                os.system('head -n -1 %s | tail -n +%d > %s/tmpfile' %
                                          (hepmc_file, n_head, os.path.dirname(hepmc_file)))
                                misc.call(['mv', 'tmp', os.path.basename(hepmc_file)], cwd=os.path.dirname(hepmc_file))
                            elif sys.platform == 'darwin':
                                # sed on MAC has slightly different synthax than on
                                os.system(' '.join(['sed','-i',"''","'%s;$d'"%
                                        (';'.join('%id'%(i+1) for i in range(n_head))),hepmc_file]))
                            else:
                                # other UNIX systems
                                os.system(' '.join(['sed','-i']+["-e '%id'"%(i+1) for i in range(n_head)]+
                                                                            ["-e '$d'",hepmc_file]))

                        os.system(' '.join(['cat',pjoin(tmp_dir,'header.hepmc')]+all_hepmc_files+
                                                    [pjoin(tmp_dir,'tail.hepmc'),'>',hepmc_output]))

Could you try check in your case which of those system call is responsible from such output?
That code before 2.7.0 was
                             if sys.platform == 'darwin':
                                # sed on MAC has slightly different synthax than on
                                os.system(' '.join(['sed','-i',"''","'%s;$d'"%
                                        (';'.join('%id'%(i+1) for i in range(n_head))),hepmc_file]))
                            else:
                                # other UNIX systems
                                os.system(' '.join(['sed','-i']+["-e '%id'"%(i+1) for i in range(n_head)]+
                                                                            ["-e '$d'",hepmc_file]))

                        os.system(' '.join(['cat',pjoin(tmp_dir,'header.hepmc')]+all_hepmc_files+
                                                    [pjoin(tmp_dir,'tail.hepmc'),'>',hepmc_output]))

So in principle the error is not in that part.

Thanks,

Olivier

Revision history for this message
Koay Yong Sheng (kysheng95) said :
#5

i commented out a few lines as shown below and everthing went fine with no weird output anymore. But I'm not particularly sure why, I tried comment out the line individually one by one, although it wont print weird output it still prints out the madevent_interface.py file.

for hepmc_file in all_hepmc_files:
                            # Remove in an efficient way the starting and trailing HEPMC tags
                            # check for support of negative argument in head
                            #pid = os.system('head -n -1 %s &> /dev/null' % __file__)
                            #if pid == 0:
                            # os.system('head -n -1 %s | tail -n +%d > %s/tmpfile' %
                            # (hepmc_file, n_head, os.path.dirname(hepmc_file)))
                            # misc.call(['mv', 'tmp', os.path.basename(hepmc_file)], cwd=os.path.dirname(hepmc_file))
                            #el
                            if sys.platform == 'darwin':
                                # sed on MAC has slightly different synthax than on
                                os.system(' '.join(['sed','-i',"''","'%s;$d'"%
                                        (';'.join('%id'%(i+1) for i in range(n_head))),hepmc_file]))
                            else:
                                # other UNIX systems
                                os.system(' '.join(['sed','-i']+["-e '%id'"%(i+1) for i in range(n_head)]+
                                                                            ["-e '$d'",hepmc_file]))

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#6

Thanks,

The line that should be responsible for the printout is likely to be
pid = os.system('head -n -1 %s &> /dev/null' % __file__)
even if I do not see how this is possible since the "&> /dev/null" should prevent any printout...

But could you check if in your case, pid=0 or something else?

Thanks,

Olivier

Revision history for this message
Koay Yong Sheng (kysheng95) said :
#7

Yes, pid=0.

I modified to:
 for hepmc_file in all_hepmc_files:
                            # Remove in an efficient way the starting and trailing HEPMC tags
                            # check for support of negative argument in head
                            pid = os.system('head -n -1 %s &> /dev/null' % __file__)
                            f = open("pid.txt", "a")
                            f.write("%s" % pid)
                            f.close()
                            #if pid == 0:
                            # os.system('head -n -1 %s | tail -n +%d > %s/tmpfile' %
                            # (hepmc_file, n_head, os.path.dirname(hepmc_file)))
                            # misc.call(['mv', 'tmp', os.path.basename(hepmc_file)], cwd=os.path.dirname(hepmc_file))
                            #el
                            if sys.platform == 'darwin':
                                # sed on MAC has slightly different synthax than on
                                os.system(' '.join(['sed','-i',"''","'%s;$d'"%
                                        (';'.join('%id'%(i+1) for i in range(n_head))),hepmc_file]))
                            else:
                                # other UNIX systems
                                os.system(' '.join(['sed','-i']+["-e '%id'"%(i+1) for i in range(n_head)]+
                                                                            ["-e '$d'",hepmc_file]))

I get "0000" in my pid.txt.

Regards,
Yong Sheng

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#8

Thanks,

And did you see any weird output on your screen during that run?

Cheers,

Olivier

> On 13 Mar 2020, at 15:32, Koay Yong Sheng <email address hidden> wrote:
>
> Question #689228 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/689228
>
> Koay Yong Sheng posted a new comment:
> Yes, pid=0.
>
> I modified to:
> for hepmc_file in all_hepmc_files:
> # Remove in an efficient way the starting and trailing HEPMC tags
> # check for support of negative argument in head
> pid = os.system('head -n -1 %s &> /dev/null' % __file__)
> f = open("pid.txt", "a")
> f.write("%s" % pid)
> f.close()
> #if pid == 0:
> # os.system('head -n -1 %s | tail -n +%d > %s/tmpfile' %
> # (hepmc_file, n_head, os.path.dirname(hepmc_file)))
> # misc.call(['mv', 'tmp', os.path.basename(hepmc_file)], cwd=os.path.dirname(hepmc_file))
> #el
> if sys.platform == 'darwin':
> # sed on MAC has slightly different synthax than on
> os.system(' '.join(['sed','-i',"''","'%s;$d'"%
> (';'.join('%id'%(i+1) for i in range(n_head))),hepmc_file]))
> else:
> # other UNIX systems
> os.system(' '.join(['sed','-i']+["-e '%id'"%(i+1) for i in range(n_head)]+
> ["-e '$d'",hepmc_file]))
>
> I get "0000" in my pid.txt.
>
> Regards,
> Yong Sheng
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Revision history for this message
Koay Yong Sheng (kysheng95) said :
#9

Yes, it prints out the the madevent_interface.py file itself rather than the weird output.

Yong Sheng

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#10

Hi,

Can I ask you
1) which os you are using?
2) wich version of tail/head/sed/python you are using.

I will need to create a virtual machine matching your system to understand what's going on.
I do not have such behaviour on my cluster (SLC7) on my mac and on one ubuntu18 virtual machine..

Thanks,

Olivier

Revision history for this message
Koay Yong Sheng (kysheng95) said :
#11

Hi,
Im using
OS: Ubuntu 18.04.4 LTS (Bionic Beaver)
Python: Python 2.7.17
head and tail : (GNU coreutils) 8.28
sed: (GNU sed) 4.4

Yong Sheng

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#12

Thanks I can reproduce the issue now. Weird that 18.04.3 is not impacted.

Cheers,

Olivier

Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said :
#13

Thanks this is the proper patch:

=== modified file 'madgraph/interface/madevent_interface.py'
--- madgraph/interface/madevent_interface.py 2020-03-09 15:47:02 +0000
+++ madgraph/interface/madevent_interface.py 2020-03-16 08:29:47 +0000
@@ -4663,11 +4663,14 @@
                         for hepmc_file in all_hepmc_files:
                             # Remove in an efficient way the starting and trailing HEPMC tags
                             # check for support of negative argument in head
- pid = os.system('head -n -1 %s &> /dev/null' % __file__)
+ devnull = open(os.path.devnull, 'w')
+ pid = misc.call(['head','-n', '-1', __file__], stdout=devnull, stderr=devnull)
+ devnull.close()
+ pid = 0
                             if pid == 0:
- os.system('head -n -1 %s | tail -n +%d > %s/tmpfile' %
- (hepmc_file, n_head, os.path.dirname(hepmc_file)))
- misc.call(['mv', 'tmp', os.path.basename(hepmc_file)], cwd=os.path.dirname(hepmc_file))
+ misc.call('head -n -1 %s | tail -n +%d > %s/tmpfile' %
+ (hepmc_file, n_head, os.path.dirname(hepmc_file)), shell=True)
+ misc.call(['mv', 'tmpfile', os.path.basename(hepmc_file)], cwd=os.path.dirname(hepmc_file))
                             elif sys.platform == 'darwin':
                                 # sed on MAC has slightly different synthax than on
                                 os.system(' '.join(['sed','-i',"''","'%s;$d'"%

Looks like a combination of python/ubuntu/... have a specific issue with the handling of stdout/stderr when using os.system.
If have use other method to call the shell and this fix the issue in my virtual machine.

Thanks a lot,

Olivier

PS: I will release this asap

Can you help with this problem?

Provide an answer of your own, or ask Koay Yong Sheng for more information if necessary.

To post a message you must log in.