Enabling static linking of HepMC2 against user PYTHIA 8?

Asked by Matthew Feickert on 2020-10-19

Hi ,

I have a question about having MadGraph5_aMC@NLO use a user installed PYTHIA 8 with --with-hepmc2 enabled --- I've asked the PYTHIA 8 dev team this question already but haven't heard back and as it is MadGraph related as well thought I'd ask here. I know there are questions that include the same error message already posted, but they didn't answer the question.

TL;DR: Is it possible to tell PYTHIA 8 during the ./configure step to link HepMC2 statically instead of dynamically? Is it possible to have MadGraph not require a statically linked HepMC2?

For space optimizations reasons, I am trying to build an image that installs all of the optional externals of MadGraph5_aMC@NLO, including PYTHIA 8, manually instead of having MadGraph do so with "MG5_aMC>install pythia8". At first pass this seems fine, as I just enable all the --with-PACKAGES I want when building PYTHIA8 (https://github.com/scailfin/MadGraph5_aMC-NLO/blob/bb2ec2c0edbe903985b0353ae7d6756e97da275e/Dockerfile#L83-L106) and then make sure to set the pythia8_path for MadGraph (https://github.com/scailfin/MadGraph5_aMC-NLO/blob/bb2ec2c0edbe903985b0353ae7d6756e97da275e/Dockerfile#L118). However, when trying to then install the mg5amc_py8_interface using their included compile.py (http://madgraph.phys.ucl.ac.be/Downloads/MG5aMC_PY8_interface/MG5aMC_PY8_interface_V1.0.tar.gz) I get the error

$ python compile.py /usr/local/
The version of HEPMC2 linked to Pythia8 seems not to include a static library.
This is necessary for the default compilation of MG5aMC_PY8_interface.
You can try again with the option --pythia8_makefile but HEPMC2 will need to be available at runtime

and likewise if I try

$ python compile.py /usr/local/ --pythia8_makefile

the compilation fails due to things that I think are MadGraph's problem.

As compile.py notes in a comment that

$ head -n 21 compile.py | tail -n 2
# When using only original PY8 makefiles, HEPMC2 will be linked dynamically.
# By default, the interface's makefile (which include PY8's ones) will link HEPMC2 statically.

my question is this: Is it possible for me to tell PYTHIA 8 during the ./configure step to link HepMC2 statically instead of dynamically? Is it possible to have MadGraph not require a statically linked HepMC2?

Best,
Matthew

Question information

Language:
English Edit question
Status:
Solved
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Solved by:
Matthew Feickert
Solved:
2020-10-20
Last query:
2020-10-20
Last reply:
2020-10-20

This question was reopened

Hi,

I have never tried to change such flag.

According to the comment that you point, you have to to put that flag if you want to have what you hope for.
What are the "madgraph's problem" that you are referring to?

Cheers,

Olivier

Matthew Feickert (feickert) said : #2

Sorry, I didn't include a reproducible example right away as I wasn't sure if there was an obvious answer to you. Please see below where I show that the Makefile should be findable:

$ docker run --rm -it scailfin/madgraph5-amc-nlo:mg5_amc2.8.1
root@2a45ca2f2ddb:~/data# find /usr/local/share/ -iname "Makefile*"
/usr/local/share/Pythia8/examples/Makefile.inc
/usr/local/share/Pythia8/examples/Makefile
root@2a45ca2f2ddb:~/data# wget -q http://madgraph.phys.ucl.ac.be/Downloads/MG5aMC_PY8_interface/MG5aMC_PY8_interface_V1.0.tar.gz
root@2a45ca2f2ddb:~/data# tar xvfz MG5aMC_PY8_interface_V1.0.tar.gz
root@2a45ca2f2ddb:~/data# python compile.py /usr/local/
The version of HEPMC2 linked to Pythia8 seems not to include a static library.
This is necessary for the default compilation of MG5aMC_PY8_interface.
You can try again with the option --pythia8_makefile but HEPMC2 will need to be available at runtime.
root@2a45ca2f2ddb:~/data# python compile.py /usr/local/ --pythia8_makefile
------------------------------------------------
MG5aMC_PY8_interface compilation output log:
b'g++ main89.cc -o main89 -I/usr/local/include -O2 -std=c++11 -pedantic -W -Wall -Wshadow -fPIC -DGZIP -lz -L/usr/local/lib -Wl,-rpath,/usr/local/lib -lpythia8 -ldl -lHepMC -DHEPMC2\nmain89.cc: In function \xe2\x80\x98int main(int, char**)\xe2\x80\x99:\nmain89.cc:185:35: error: no matching function for call to \xe2\x80\x98Pythia8::Pythia::setUserHooksPtr(Pythia8::amcnlo_unitarised_interface*&)\xe2\x80\x99\n pythia.setUserHooksPtr(setting);\n ^\nIn file included from /usr/local/include/Pythia8Plugins/HepMC2.h:21,\n from main89.cc:11:\n/usr/local/include/Pythia8/Pythia.h:136:8: note: candidate: \xe2\x80\x98bool Pythia8::Pythia::setUserHooksPtr(Pythia8::UserHooksPtr)\xe2\x80\x99\n bool setUserHooksPtr(UserHooksPtr userHooksPtrIn) {\n ^~~~~~~~~~~~~~~\n/usr/local/include/Pythia8/Pythia.h:136:8: note: no known conversion for argument 1 from \xe2\x80\x98Pythia8::amcnlo_unitarised_interface*\xe2\x80\x99 to \xe2\x80\x98Pythia8::UserHooksPtr\xe2\x80\x99 {aka \xe2\x80\x98std::shared_ptr<Pythia8::UserHooks>\xe2\x80\x99}\nmain89.cc:203:36: error: no matching function for call to \xe2\x80\x98Pythia8::Pythia::setUserHooksPtr(Pythia8::JetMatchingMadgraph*&)\xe2\x80\x99\n pythia.setUserHooksPtr(matching);\n ^\nIn file included from /usr/local/include/Pythia8Plugins/HepMC2.h:21,\n from main89.cc:11:\n/usr/local/include/Pythia8/Pythia.h:136:8: note: candidate: \xe2\x80\x98bool Pythia8::Pythia::setUserHooksPtr(Pythia8::UserHooksPtr)\xe2\x80\x99\n bool setUserHooksPtr(UserHooksPtr userHooksPtrIn) {\n ^~~~~~~~~~~~~~~\n/usr/local/include/Pythia8/Pythia.h:136:8: note: no known conversion for argument 1 from \xe2\x80\x98Pythia8::JetMatchingMadgraph*\xe2\x80\x99 to \xe2\x80\x98Pythia8::UserHooksPtr\xe2\x80\x99 {aka \xe2\x80\x98std::shared_ptr<Pythia8::UserHooks>\xe2\x80\x99}\nmain89.cc:514:114: warning: unknown escape sequence: \'\\%\'\n <<" Normalisation will be decreased by" << (pythia.info.nSelected() - nEvent)/(nEvent*0.010)<<"\\%."\n ^~~~~\nmain89.cc:518:15: warning: unknown escape sequence: \'\\%\'\n cerr << "ERROR: More than 2\\% of the events fails to be showered by pythia. Stopping" << endl;\n ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nmake: *** [Makefile:78: main89] Error 1\n'
------------------------------------------------
Error during the compilation of MG5aMC_PY8_interface:
b'g++ main89.cc -o main89 -I/usr/local/include -O2 -std=c++11 -pedantic -W -Wall -Wshadow -fPIC -DGZIP -lz -L/usr/local/lib -Wl,-rpath,/usr/local/lib -lpythia8 -ldl -lHepMC -DHEPMC2\nmain89.cc: In function \xe2\x80\x98int main(int, char**)\xe2\x80\x99:\nmain89.cc:185:35: error: no matching function for call to \xe2\x80\x98Pythia8::Pythia::setUserHooksPtr(Pythia8::amcnlo_unitarised_interface*&)\xe2\x80\x99\n pythia.setUserHooksPtr(setting);\n ^\nIn file included from /usr/local/include/Pythia8Plugins/HepMC2.h:21,\n from main89.cc:11:\n/usr/local/include/Pythia8/Pythia.h:136:8: note: candidate: \xe2\x80\x98bool Pythia8::Pythia::setUserHooksPtr(Pythia8::UserHooksPtr)\xe2\x80\x99\n bool setUserHooksPtr(UserHooksPtr userHooksPtrIn) {\n ^~~~~~~~~~~~~~~\n/usr/local/include/Pythia8/Pythia.h:136:8: note: no known conversion for argument 1 from \xe2\x80\x98Pythia8::amcnlo_unitarised_interface*\xe2\x80\x99 to \xe2\x80\x98Pythia8::UserHooksPtr\xe2\x80\x99 {aka \xe2\x80\x98std::shared_ptr<Pythia8::UserHooks>\xe2\x80\x99}\nmain89.cc:203:36: error: no matching function for call to \xe2\x80\x98Pythia8::Pythia::setUserHooksPtr(Pythia8::JetMatchingMadgraph*&)\xe2\x80\x99\n pythia.setUserHooksPtr(matching);\n ^\nIn file included from /usr/local/include/Pythia8Plugins/HepMC2.h:21,\n from main89.cc:11:\n/usr/local/include/Pythia8/Pythia.h:136:8: note: candidate: \xe2\x80\x98bool Pythia8::Pythia::setUserHooksPtr(Pythia8::UserHooksPtr)\xe2\x80\x99\n bool setUserHooksPtr(UserHooksPtr userHooksPtrIn) {\n ^~~~~~~~~~~~~~~\n/usr/local/include/Pythia8/Pythia.h:136:8: note: no known conversion for argument 1 from \xe2\x80\x98Pythia8::JetMatchingMadgraph*\xe2\x80\x99 to \xe2\x80\x98Pythia8::UserHooksPtr\xe2\x80\x99 {aka \xe2\x80\x98std::shared_ptr<Pythia8::UserHooks>\xe2\x80\x99}\nmain89.cc:514:114: warning: unknown escape sequence: \'\\%\'\n <<" Normalisation will be decreased by" << (pythia.info.nSelected() - nEvent)/(nEvent*0.010)<<"\\%."\n ^~~~~\nmain89.cc:518:15: warning: unknown escape sequence: \'\\%\'\n cerr << "ERROR: More than 2\\% of the events fails to be showered by pythia. Stopping" << endl;\n ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\nmake: *** [Makefile:78: main89] Error 1\n'
root@2a45ca2f2ddb:~/data#

Are you sure that you link correctly to pythia8? it seems to fail to find some file related to pythia8.
Or maybe you try to link to a incompatible version of pythia8. I have never succeed to link to pythia8.3 for example.

Otherwise, the best person to answer this is a pythia author (Stefan Prestel) since you already contacted him, the best is to wait for his answer.

Cheers,

Olivier

Matthew Feickert (feickert) said : #4

> Are you sure that you link correctly to pythia8?

Link what? HepMC2? If so, you can see exactly what I'm doing here (https://github.com/scailfin/MadGraph5_aMC-NLO/blob/bb2ec2c0edbe903985b0353ae7d6756e97da275e/Dockerfile#L83-L106) and that the PYTHIA 8 example main89 (that MadGraph is apparently trying to compile above and failing) is able to be compiled fine in the container:

$ docker run --rm -it scailfin/madgraph5-amc-nlo:mg5_amc2.8.1
root@86026fe29c08:~/data# cd /usr/local/share/Pythia8/examples/
root@86026fe29c08:/usr/local/share/Pythia8/examples# make main89
g++ main89.cc -o main89 -I/usr/local/include -O2 -std=c++11 -pedantic -W -Wall -Wshadow -fPIC -DGZIP -lz -L/usr/local/lib -Wl,-rpath,/usr/local/lib -lpythia8 -ldl -lHepMC -DHEPMC2
root@86026fe29c08:/usr/local/share/Pythia8/examples# ./main89 main89ckkwl.cmnd test.txt
root@86026fe29c08:/usr/local/share/Pythia8/examples# # The above runs fine

> Or maybe you try to link to a incompatible version of pythia8. I have never succeed to link to pythia8.3 for example.

Is there a public list of MadGraph supported PYTHIA 8 releases? What is the most recent version of PYTHIA 8 that you've been able to link to?

Hi,

> What is the most recent version of PYTHIA 8 that you've been able to link to?

That's the one install via "install pythia8"
All previous version of pythia8.2 were working without any issue.
Change in main version of program are one typical reason of breaking inter-operability between tools.

Cheers,

Olivier

> On 20 Oct 2020, at 17:45, Matthew Feickert <email address hidden> wrote:
>
> Question #693533 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/693533
>
> Matthew Feickert posted a new comment:
>> Are you sure that you link correctly to pythia8?
>
> Link what? HepMC2? If so, you can see exactly what I'm doing here
> (https://github.com/scailfin/MadGraph5_aMC-
> NLO/blob/bb2ec2c0edbe903985b0353ae7d6756e97da275e/Dockerfile#L83-L106)
> and that the PYTHIA 8 example main89 (that MadGraph is apparently trying
> to compile above and failing) is able to be compiled fine in the
> container:
>
> $ docker run --rm -it scailfin/madgraph5-amc-nlo:mg5_amc2.8.1
> root@86026fe29c08:~/data# cd /usr/local/share/Pythia8/examples/
> root@86026fe29c08:/usr/local/share/Pythia8/examples# make main89
> g++ main89.cc -o main89 -I/usr/local/include -O2 -std=c++11 -pedantic -W -Wall -Wshadow -fPIC -DGZIP -lz -L/usr/local/lib -Wl,-rpath,/usr/local/lib -lpythia8 -ldl -lHepMC -DHEPMC2
> root@86026fe29c08:/usr/local/share/Pythia8/examples# ./main89 main89ckkwl.cmnd test.txt
> root@86026fe29c08:/usr/local/share/Pythia8/examples# # The above runs fine
>
>> Or maybe you try to link to a incompatible version of pythia8. I have
> never succeed to link to pythia8.3 for example.
>
> Is there a public list of MadGraph supported PYTHIA 8 releases? What is
> the most recent version of PYTHIA 8 that you've been able to link to?
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Matthew Feickert (feickert) said : #6

> All previous version of pythia8.2 were working without any issue.

I've tried again with building a local test image that uses PYTHIA v8.243 and that at least compiled:

root@a969ba77ac5a:~/data#
root@a969ba77ac5a:~/data# wget -q http://madgraph.phys.ucl.ac.be/Downloads/MG5aMC_PY8_interface/MG5aMC_PY8_interface_V1.0.tar.gz
root@a969ba77ac5a:~/data# tar xvfz MG5aMC_PY8_interface_V1.0.tar.gz
COMPATIBILITY
SyscalcVeto.h
get_pythia8_version.py
make_tarball.py
Makefile_mg5amc_py8_interface_static
pythia8_version.cc
VERSION
MG5aMC_PY8_interface.cc
MultiHist.h
compile.py
root@a969ba77ac5a:~/data# python compile.py /usr/local/ --pythia8_makefile
------------------------------------------------
MG5aMC_PY8_interface compilation output log:
b'g++ main89.cc -o main89 -I./ -I/usr/local/include -O2 -pedantic -W -Wall -Wshadow -fPIC -L/usr/local/lib -Wl,-rpath,/usr/local/lib -lpythia8 -ldl \\\n -L./ -Wl,-rpath,./ -lHepMC\\\n -DGZIPSUPPORT -I./ -L./ -Wl,-rpath,./ -lz\nmain89.cc: In function \xe2\x80\x98int main(int, char**)\xe2\x80\x99:\nmain89.cc:514:114: warning: unknown escape sequence: \'\\%\'\n <<" Normalisation will be decreased by" << (pythia.info.nSelected() - nEvent)/(nEvent*0.010)<<"\\%."\n ^~~~~\nmain89.cc:518:15: warning: unknown escape sequence: \'\\%\'\n cerr << "ERROR: More than 2\\% of the events fails to be showered by pythia. Stopping" << endl;\n ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n'
------------------------------------------------
Successful compilation of the MG5aMC_PY8_interface.

So that's probably enough of a start.

Are there any plans to support PYTHIA 8.3 in the future?

For pythia8.3, I will likeluy need support from py8 autthor.

Cheers,

Olivier

> On 20 Oct 2020, at 22:50, Matthew Feickert <email address hidden> wrote:
>
> Question #693533 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/693533
>
> Matthew Feickert posted a new comment:
>> All previous version of pythia8.2 were working without any issue.
>
> I've tried again with building a local test image that uses PYTHIA
> v8.243 and that at least compiled:
>
> root@a969ba77ac5a:~/data#
> root@a969ba77ac5a:~/data# wget -q http://madgraph.phys.ucl.ac.be/Downloads/MG5aMC_PY8_interface/MG5aMC_PY8_interface_V1.0.tar.gz
> root@a969ba77ac5a:~/data# tar xvfz MG5aMC_PY8_interface_V1.0.tar.gz
> COMPATIBILITY
> SyscalcVeto.h
> get_pythia8_version.py
> make_tarball.py
> Makefile_mg5amc_py8_interface_static
> pythia8_version.cc
> VERSION
> MG5aMC_PY8_interface.cc
> MultiHist.h
> compile.py
> root@a969ba77ac5a:~/data# python compile.py /usr/local/ --pythia8_makefile
> ------------------------------------------------
> MG5aMC_PY8_interface compilation output log:
> b'g++ main89.cc -o main89 -I./ -I/usr/local/include -O2 -pedantic -W -Wall -Wshadow -fPIC -L/usr/local/lib -Wl,-rpath,/usr/local/lib -lpythia8 -ldl \\\n -L./ -Wl,-rpath,./ -lHepMC\\\n -DGZIPSUPPORT -I./ -L./ -Wl,-rpath,./ -lz\nmain89.cc: In function \xe2\x80\x98int main(int, char**)\xe2\x80\x99:\nmain89.cc:514:114: warning: unknown escape sequence: \'\\%\'\n <<" Normalisation will be decreased by" << (pythia.info.nSelected() - nEvent)/(nEvent*0.010)<<"\\%."\n ^~~~~\nmain89.cc:518:15: warning: unknown escape sequence: \'\\%\'\n cerr << "ERROR: More than 2\\% of the events fails to be showered by pythia. Stopping" << endl;\n ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n'
> ------------------------------------------------
> Successful compilation of the MG5aMC_PY8_interface.
>
>
> So that's probably enough of a start.
>
> Are there any plans to support PYTHIA 8.3 in the future?
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Matthew Feickert (feickert) said : #8

Okay, that's good to know. I'll relay that when I respond to the PYTHIA 8 dev team mailing list with this information.