cannot find numpy.f2py

Asked by Ilirjan Margjeka on 2019-04-30

Hi there,

I am trying to run the reweight-mode for a model at NLO.
I am using a CMSSW environment (CMSSW_10_2_5), and I have imported the numpy ancd scipi environment requested, in order to reweight the unweighted events.

source /cvmfs/sft.cern.ch/lcg/releases/LCG_92/numpy/1.13.3/x86_64-slc6-gcc7-opt/numpy-env.sh
source /cvmfs/sft.cern.ch/lcg/releases/LCG_92/scipy/0.18.1/x86_64-slc6-gcc7-opt/scipy-env.s

Anyway, I've got the error about the f2py compiler, so I thought to add the path of it in this way:
export PATH=$PATH:/cvmfs/cms.cern.ch/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_2_5/external/slc6_amd64_gcc700/bin/f2py

But I'm not succeeding and I am getting this bug:

#************************************************************
#* MadGraph5_aMC@NLO *
#* *
#* * * *
#* * * * * *
#* * * * * 5 * * * * *
#* * * * * *
#* * * *
#* *
#* *
#* VERSION 5.2.6.0 20xx-xx-xx *
#* *
#* The MadGraph5_aMC@NLO Development Team - Find us at *
#* https://server06.fynu.ucl.ac.be/projects/madgraph *
#* and *
#* http://amcatnlo.cern.ch *
#* *
#************************************************************
#* *
#* Command File for aMCatNLO *
#* *
#* run as ./bin/aMCatNLO.py filename *
#* *
#************************************************************
reweight run_24 -from_cards
Traceback (most recent call last):
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/gghhnlo/bin/internal/extended_cmd.py", line 1438, in onecmd
    return self.onecmd_orig(line, **opt)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/gghhnlo/bin/internal/extended_cmd.py", line 1392, in onecmd_orig
    return func(arg, **opt)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/gghhnlo/bin/internal/common_run_interface.py", line 1996, in do_reweight
    reweight_cmd.import_command_file(path)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/interface/extended_cmd.py", line 1580, in import_command_file
    self.exec_cmd(line, precmd=True)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/interface/extended_cmd.py", line 1465, in exec_cmd
    stop = Cmd.onecmd_orig(current_interface, line, **opt)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/interface/extended_cmd.py", line 1392, in onecmd_orig
    return func(arg, **opt)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/various/misc.py", line 99, in f_with_no_logger
    out = f(self, *args, **opt)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/interface/reweight_interface.py", line 476, in do_launch
    self.create_standalone_directory()
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/various/misc.py", line 99, in f_with_no_logger
    out = f(self, *args, **opt)
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/interface/reweight_interface.py", line 1569, in create_standalone_directory
    misc.compile(cwd=pjoin(path_me, data['paths'][1], 'Source'))
  File "/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/madgraph/various/misc.py", line 479, in compile
    raise MadGraph5Error, error_text
MadGraph5Error: A compilation Error occurs when trying to compile /lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/gghhnlo/rw_mevirt/Source.
The compilation fails with the following output message:
    cd PDF; make
    make[1]: Entering directory '/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/gghhnlo/rw_mevirt/Source/PDF'
    gfortran -O -w -fbounds-check -fPIC -ffixed-line-length-132 -c -o pdfwrap_lhapdf.o pdfwrap_lhapdf.f
    gfortran -O -w -fbounds-check -fPIC -ffixed-line-length-132 -c -o pdf_lhapdf.o pdf_lhapdf.f
    gfortran -O -w -fbounds-check -fPIC -ffixed-line-length-132 -c -o pdg2pdf_lhapdf.o pdg2pdf_lhapdf.f
    gfortran -O -w -fbounds-check -fPIC -ffixed-line-length-132 -c -o opendata.o opendata.f
    ar cru ../../lib/libpdf.a pdfwrap_lhapdf.o pdf_lhapdf.o pdg2pdf_lhapdf.o opendata.o
    ar: `u' modifier ignored since `D' is the default (see `U')
    ranlib ../../lib/libpdf.a
    make[1]: Leaving directory '/lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/gghhnlo/rw_mevirt/Source/PDF'
    make: True: Command not found
    touch __init__.py
    touch ../__init__.py
    f2py -L../lib/ -lpdf -L/cvmfs/cms.cern.ch/slc6_amd64_gcc700/external/lhapdf/6.2.1-gnimlf2/lib -lLHAPDF -lLHAPDF -lstdc++ -c rwgt.f -m rwgt2py --f77exec=gfortran
    Traceback (most recent call last):
      File "/cvmfs/cms.cern.ch/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_2_5/external/slc6_amd64_gcc700/bin/f2py", line 24, in <module>
        from numpy.f2py import main
    ImportError: No module named numpy.f2py
    make: *** [makefile:13: rwgt2py.so] Error 1

Please try to fix this compilations issue and retry.
Help might be found at https://answers.launchpad.net/mg5amcnlo.
If you think that this is a bug, you can report this at https://bugs.launchpad.net/mg5amcnlo
Value of current Options:
              text_editor : None
              web_browser : None
        cluster_temp_path : None
                  timeout : 60
       cluster_local_path : None
            cluster_queue : None
         madanalysis_path : None
                   lhapdf : lhapdf-config
             cluster_size : 100
           cluster_memory : None
    cluster_status_update : (600, 30)
             cluster_time : None
            f2py_compiler : None
                    ninja : /lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/HEPTools/lib
               hepmc_path : None
             pythia8_path : None
                hwpp_path : None
   automatic_html_opening : True
       cluster_retry_wait : 300
             stdout_level : None
          pythia-pgs_path : None
                 mg5_path : /lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0
                  td_path : None
             delphes_path : None
              thepeg_path : None
             cluster_type : condor
        madanalysis5_path : None
      exrootanalysis_path : /lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/ExRootAnalysis
         fortran_compiler : None
                  nb_core : 24
                  collier : /lustrehome/imargjek/CMSSW_10_2_5/src/MG5_aMC_v2_6_0/HEPTools/lib
              auto_update : 7
         cluster_nb_retry : 1
               eps_viewer : None
             syscalc_path : None
                  fastjet : fastjet-config
             cpp_compiler : None
      notification_center : True
                 run_mode : 2
#************************************************************
#* MadGraph5_aMC@NLO *
#* *
#* * * *
#* * * * * *
#* * * * * 5 * * * * *
#* * * * * *
#* * * *
#* *
#* *
#* VERSION 2.6.0 2017-08-16 *
#* *
#* The MadGraph5_aMC@NLO Development Team - Find us at *
#* https://server06.fynu.ucl.ac.be/projects/madgraph *
#* *
#************************************************************
#* *
#* Command File for MadGraph5_aMC@NLO *
#* *
#* run as ./bin/mg5_aMC filename *
#* *
#************************************************************
set group_subprocesses Auto
set ignore_six_quark_processes False
set loop_optimized_output True
set low_mem_multicore_nlo_generation False
set loop_color_flows False
set gauge unitary
set complex_mass_scheme False
set max_npoint_for_channel 0
import model sm
define p = g u c d s u~ c~ d~ s~
define j = g u c d s u~ c~ d~ s~
define l+ = e+ mu+
define l- = e- mu-
define vl = ve vm vt
define vl~ = ve~ vm~ vt~
import model SMEFT_UFO-no_b_mass
define p = 21 2 4 1 3 -2 -4 -1 -3 5 -5
define j = p
generate p p > H H [QCD]
output gghhnlo
######################################################################
## PARAM_CARD AUTOMATICALY GENERATED BY MG5 FOLLOWING UFO MODEL ####
######################################################################
## ##
## Width set on Auto will be computed following the information ##
## present in the decay.py files of the model. ##
## See arXiv:1402.1178 for more details. ##
## ##
######################################################################

###################################
## INFORMATION FOR MASS
###################################
Block mass
    6 1.730000e+02 # MT
   15 1.777000e+00 # MTA
   23 9.118760e+01 # MZ
   25 1.250000e+02 # MH
## Dependent parameters, given by model restrictions.
## Those values should be edited following the
## analytical expression. MG5 ignores those values
## but they are important for interfacing the output of MG5
## to external program such as Pythia.
  1 0.000000 # d : 0.0
  2 0.000000 # u : 0.0
  3 0.000000 # s : 0.0
  4 0.000000 # c : 0.0
  5 0.000000 # b : 0.0
  11 0.000000 # e- : 0.0
  12 0.000000 # ve : 0.0
  13 0.000000 # mu- : 0.0
  14 0.000000 # vm : 0.0
  16 0.000000 # vt : 0.0
  21 0.000000 # g : 0.0
  22 0.000000 # a : 0.0
  24 79.824360 # w+ : cmath.sqrt(MZ__exp__2/2. + cmath.sqrt(MZ__exp__4/4. - (aEW*cmath.pi*MZ__exp__2)/(Gf*sqrt__2)))
  9000002 91.187600 # ghz : MZ
  9000003 79.824360 # ghwp : MW
  9000004 79.824360 # ghwm : MW
  250 91.187600 # g0 : MZ
  251 79.824360 # g+ : MW

###################################
## INFORMATION FOR SMINPUTS
###################################
Block sminputs
    1 1.279000e+02 # aEWM1
    2 1.166370e-05 # Gf
    3 1.184000e-01 # aS

###################################
## INFORMATION FOR YUKAWA
###################################
Block yukawa
    6 1.730000e+02 # ymt
   15 1.777000e+00 # ymtau

###################################
## INFORMATION FOR DECAY
###################################
DECAY 6 1.508336e+00 # WT
DECAY 23 2.495200e+00 # WZ
DECAY 24 2.085000e+00 # WW
DECAY 25 5.753088e-03 # WH
## Dependent parameters, given by model restrictions.
## Those values should be edited following the
## analytical expression. MG5 ignores those values
## but they are important for interfacing the output of MG5
## to external program such as Pythia.
DECAY 1 0.000000 # d : 0.0
DECAY 2 0.000000 # u : 0.0
DECAY 3 0.000000 # s : 0.0
DECAY 4 0.000000 # c : 0.0
DECAY 5 0.000000 # b : 0.0
DECAY 11 0.000000 # e- : 0.0
DECAY 12 0.000000 # ve : 0.0
DECAY 13 0.000000 # mu- : 0.0
DECAY 14 0.000000 # vm : 0.0
DECAY 15 0.000000 # ta- : 0.0
DECAY 16 0.000000 # vt : 0.0
DECAY 21 0.000000 # g : 0.0
DECAY 22 0.000000 # a : 0.0
DECAY 9000002 2.495200 # ghz : WZ
DECAY 9000003 2.085000 # ghwp : WW
DECAY 9000004 2.085000 # ghwm : WW
DECAY 250 2.495200 # g0 : WZ
DECAY 251 2.085000 # g+ : WW
#===========================================================
# QUANTUM NUMBERS OF NEW STATE(S) (NON SM PDG CODE)
#===========================================================

Block QNUMBERS 250 # g0
        1 0 # 3 times electric charge
        2 1 # number of spin states (2S+1)
        3 1 # colour rep (1: singlet, 3: triplet, 8: octet)
        4 0 # Particle/Antiparticle distinction (0=own anti)
Block QNUMBERS 251 # g+
        1 3 # 3 times electric charge
        2 1 # number of spin states (2S+1)
        3 1 # colour rep (1: singlet, 3: triplet, 8: octet)
        4 1 # Particle/Antiparticle distinction (0=own anti)
#***********************************************************************
# MadGraph5_aMC@NLO *
# *
# run_card.dat aMC@NLO *
# *
# This file is used to set the parameters of the run. *
# *
# Some notation/conventions: *
# *
# Lines starting with a hash (#) are info or comments *
# *
# mind the format: value = variable ! comment *
# *
# Some of the values of variables can be list. These can either be *
# comma or space separated. *
#***********************************************************************
#
#*******************
# Running parameters
#*******************
#
#***********************************************************************
# Tag name for the run (one word) *
#***********************************************************************
  tag_1 = run_tag ! name of the run
#***********************************************************************
# Number of LHE events (and their normalization) and the required *
# (relative) accuracy on the Xsec. *
# These values are ignored for fixed order runs *
#***********************************************************************
 500 = nevents ! Number of unweighted events requested
 -1.0 = req_acc ! Required accuracy (-1=auto determined from nevents)
 -1 = nevt_job! Max number of events per job in event generation.
                 ! (-1= no split).
#***********************************************************************
# Normalize the weights of LHE events such that they sum or average to *
# the total cross section *
#***********************************************************************
 average = event_norm ! valid settings: average, sum, bias
#***********************************************************************
# Number of points per itegration channel (ignored for aMC@NLO runs) *
#***********************************************************************
 0.01 = req_acc_FO ! Required accuracy (-1=ignored, and use the
                     ! number of points and iter. below)
# These numbers are ignored except if req_acc_FO is equal to -1
 5000 = npoints_FO_grid ! number of points to setup grids
 4 = niters_FO_grid ! number of iter. to setup grids
 10000 = npoints_FO ! number of points to compute Xsec
 6 = niters_FO ! number of iter. to compute Xsec
#***********************************************************************
# Random number seed *
#***********************************************************************
 0 = iseed ! rnd seed (0=assigned automatically=default))
#***********************************************************************
# Collider type and energy *
#***********************************************************************
 1 = lpp1 ! beam 1 type (0 = no PDF)
 1 = lpp2 ! beam 2 type (0 = no PDF)
 7000.0 = ebeam1 ! beam 1 energy in GeV
 7000.0 = ebeam2 ! beam 2 energy in GeV
#***********************************************************************
# PDF choice: this automatically fixes also alpha_s(MZ) and its evol. *
#***********************************************************************
 nn23nlo = pdlabel ! PDF set nn23nlo
 244600 = lhaid ! If pdlabel=lhapdf, this is the lhapdf number. Only
              ! numbers for central PDF sets are allowed. Can be a list;
              ! PDF sets beyond the first are included via reweighting.
#***********************************************************************
# Include the NLO Monte Carlo subtr. terms for the following parton *
# shower (HERWIG6 | HERWIGPP | PYTHIA6Q | PYTHIA6PT | PYTHIA8) *
# WARNING: PYTHIA6PT works only for processes without FSR!!!! *
#***********************************************************************
  HERWIG6 = parton_shower
  1.0 = shower_scale_factor ! multiply default shower starting
                                  ! scale by this factor
#***********************************************************************
# Renormalization and factorization scales *
# (Default functional form for the non-fixed scales is the sum of *
# the transverse masses divided by two of all final state particles *
# and partons. This can be changed in SubProcesses/set_scales.f or via *
# dynamical_scale_choice option) *
#***********************************************************************
 False = fixed_ren_scale ! if .true. use fixed ren scale
 False = fixed_fac_scale ! if .true. use fixed fac scale
 91.118 = muR_ref_fixed ! fixed ren reference scale
 91.118 = muF_ref_fixed ! fixed fact reference scale
 10 = dynamical_scale_choice ! Choose one (or more) of the predefined
           ! dynamical choices. Can be a list; scale choices beyond the
           ! first are included via reweighting
 1.0 = muR_over_ref ! ratio of current muR over reference muR
 1.0 = muF_over_ref ! ratio of current muF over reference muF
#***********************************************************************
# Reweight variables for scale dependence and PDF uncertainty *
#***********************************************************************
 1.0, 2.0, 0.5 = rw_rscale ! muR factors to be included by reweighting
 1.0, 2.0, 0.5 = rw_fscale ! muF factors to be included by reweighting
 True = reweight_scale ! Reweight to get scale variation using the
            ! rw_rscale and rw_fscale factors. Should be a list of
            ! booleans of equal length to dynamical_scale_choice to
            ! specify for which choice to include scale dependence.
 False = reweight_PDF ! Reweight to get PDF uncertainty. Should be a
            ! list booleans of equal length to lhaid to specify for
            ! which PDF set to include the uncertainties.
#***********************************************************************
# Store reweight information in the LHE file for off-line model- *
# parameter reweighting at NLO+PS accuracy *
#***********************************************************************
 True = store_rwgt_info ! Store info for reweighting in LHE file
#***********************************************************************
# ickkw parameter: *
# 0: No merging *
# 3: FxFx Merging - WARNING! Applies merging only at the hard-event *
# level. After showering an MLM-type merging should be applied as *
# well. See http://amcatnlo.cern.ch/FxFx_merging.htm for details. *
# 4: UNLOPS merging (with pythia8 only). No interface from within *
# MG5_aMC available, but available in Pythia8. *
# -1: NNLL+NLO jet-veto computation. See arxiv:1412.8408 [hep-ph]. *
#***********************************************************************
 0 = ickkw
#***********************************************************************
#
#***********************************************************************
# BW cutoff (M+/-bwcutoff*Gamma). Determines which resonances are *
# written in the LHE event file *
#***********************************************************************
 30.0 = bwcutoff
#***********************************************************************
# Cuts on the jets. Jet clustering is performed by FastJet. *
# - When matching to a parton shower, these generation cuts should be *
# considerably softer than the analysis cuts. *
# - More specific cuts can be specified in SubProcesses/cuts.f *
#***********************************************************************
  1.0 = jetalgo ! FastJet jet algorithm (1=kT, 0=C/A, -1=anti-kT)
  0.7 = jetradius ! The radius parameter for the jet algorithm
 10.0 = ptj ! Min jet transverse momentum
 -1.0 = etaj ! Max jet abs(pseudo-rap) (a value .lt.0 means no cut)
#***********************************************************************
# Cuts on the charged leptons (e+, e-, mu+, mu-, tau+ and tau-) *
# More specific cuts can be specified in SubProcesses/cuts.f *
#***********************************************************************
  0.0 = ptl ! Min lepton transverse momentum
 -1.0 = etal ! Max lepton abs(pseudo-rap) (a value .lt.0 means no cut)
  0.0 = drll ! Min distance between opposite sign lepton pairs
  0.0 = drll_sf ! Min distance between opp. sign same-flavor lepton pairs
  0.0 = mll ! Min inv. mass of all opposite sign lepton pairs
 30.0 = mll_sf ! Min inv. mass of all opp. sign same-flavor lepton pairs
#***********************************************************************
# Photon-isolation cuts, according to hep-ph/9801442. When ptgmin=0, *
# all the other parameters are ignored. *
# More specific cuts can be specified in SubProcesses/cuts.f *
#***********************************************************************
 20.0 = ptgmin ! Min photon transverse momentum
 -1.0 = etagamma ! Max photon abs(pseudo-rap)
  0.4 = R0gamma ! Radius of isolation code
  1.0 = xn ! n parameter of eq.(3.4) in hep-ph/9801442
  1.0 = epsgamma ! epsilon_gamma parameter of eq.(3.4) in hep-ph/9801442
 True = isoEM ! isolate photons from EM energy (photons and leptons)
#***********************************************************************
# For aMCfast+APPLGRID use in PDF fitting (http://amcfast.hepforge.org)*
#***********************************************************************
 0 = iappl ! aMCfast switch (0=OFF, 1=prepare grids, 2=fill grids)
#***********************************************************************

How can I correctly implement the f2py compiler?

Kind regards,
Ilirjan Margjeka

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
2019-04-30
Last reply:
2019-05-01

Hi,

1) you can see that you did not specify within mg5amc, where your f2py compiler is:
I see the following
f2py_compiler : None
The easiest is to define that variable in one of the configuration file that read MG5aMC

2) You do not use the PATH environment variable correctly. You have to put the directory where the birnary is not the binary itself.
So
export PATH=$PATH:/cvmfs/cms.cern.ch/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_2_5/external/slc6_amd64_gcc700/bin/

sounds a better way to specify it.

Cheers,

Olivier

Hi Olivier,

I am getting the same error.

I also specified the path of the f2py compiler in the amcatnlo_configuration.txt oc the Cards/ directory and in the mg5_configuration.txt of the /MG5_aMC_v2_6_0/input directory by doing :

# f2py_compiler = ./cvmfs/cms.cern.ch/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_2_5/external/slc6_amd64_gcc700/bin/f2py

What else can I do?

Thx

Did you uncomment the line? If not nothing will happen obviously
Second, Why do you have ./cvmfs.
This sounds a weird local path. Please use only absolute path

Cheers,

Olivier

> On 30 Apr 2019, at 21:43, Ilirjan Margjeka <email address hidden> wrote:
>
> Question #680556 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/680556
>
> Ilirjan Margjeka posted a new comment:
> Hi Olivier,
>
> I am getting the same error.
>
> I also specified the path of the f2py compiler in the
> amcatnlo_configuration.txt oc the Cards/ directory and in the
> mg5_configuration.txt of the /MG5_aMC_v2_6_0/input directory by doing :
>
> # f2py_compiler =
> ./cvmfs/cms.cern.ch/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_2_5/external/slc6_amd64_gcc700/bin/f2py
>
> What else can I do?
>
> Thx
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

So, I un-commented it, I get this in the ME4_bug:
f2py_compiler : /cvmfs/cms.cern.ch/slc6_amd64_gcc700/cms/cmssw/CMSSW_10_2_5/external/slc6_amd64_gcc700/bin/f2py
but, the error it persists.
I am working on lxplus.cern.ch, this explains also the path.

Is there any way to do it differently?

Thx,
Ilirjan

Hi,

Looks like your version of f2py is not setup correctly.
Please contact your IT team (or I guess in this case the one of your collaboration and/or the one of lxplus to know how to setup correctly python/numpy/f2py (or likely read their instruction on that)

Cheers,

Olivier

Can you help with this problem?

Provide an answer of your own, or ask Ilirjan Margjeka for more information if necessary.

To post a message you must log in.