how to do 5 jets or more in the final states

Asked by Yuri Gershtein on 2012-02-21

Hi,

I am working with version 5.1.4.1
I generated pp->diphoton+up to 5 jets.
The python step generating diagrams took a loooong while, but seem to have finished fine.

When I try to generate events though, I get a gfortran compilation error attached below.
I'm stuck and would appreciate advice on where to go from here.
Is it Gfortran bug? Is it MG bug?

Thanks,
Yuri
#************************************************************
#* MadGraph/MadEvent 5 *
#* *
#* * * *
#* * * * * *
#* * * * * 5 * * * * *
#* * * * * *
#* * * *
#* *
#* *
#* *
#* VERSION 1.4.1 2012-06-02 *
#* *
#* The MadGraph Development Team - Please visit us at *
#* https://server06.fynu.ucl.ac.be/projects/madgraph *
#* *
#************************************************************
#* *
#* Command File for MadEvent *
#* *
#* run as ./bin/madevent.py filename *
#* *
#************************************************************
generate_events run0 --cluster
Traceback (most recent call last):
  File "/cms/data27/gershtein/MadGraph5_v1_4_1/madgraph/interface/extended_cmd.py", line 405, in onecmd
    return cmd.Cmd.onecmd(self, line)
  File "/cms/base/python-2.7.1/lib/python2.7/cmd.py", line 219, in onecmd
    return func(arg)
  File "/cms/data27/gershtein/MadGraph5_v1_4_1/madgraph/interface/madevent_interface.py", line 1712, in do_generate_events
    postcmd=False)
  File "/cms/data27/gershtein/MadGraph5_v1_4_1/madgraph/interface/extended_cmd.py", line 443, in exec_cmd
    stop = cmd.Cmd.onecmd(current_interface, line)
  File "/cms/base/python-2.7.1/lib/python2.7/cmd.py", line 219, in onecmd
    return func(arg)
  File "/cms/data27/gershtein/MadGraph5_v1_4_1/madgraph/interface/madevent_interface.py", line 1960, in do_survey
    misc.compile(['gensym'], cwd=Pdir)
  File "/cms/data27/gershtein/MadGraph5_v1_4_1/madgraph/iolibs/misc.py", line 194, in compile
    raise MadGraph5Error, error_text
MadGraph5Error: A compilation Error occurs when trying to compile /cms/data27/gershtein/MadGraph5_v1_4_1/AA5J/SubProcesses/P6_gg_aagqqqq.
The compilation fails with the following output message:
    gfortran -O -w -ffixed-line-length-132 -w -c -o symmetry.o symmetry.f
    touch qmass.inc
    gfortran -O -w -ffixed-line-length-132 -w -c -o setcuts.o setcuts.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o cuts.o cuts.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o cluster.o cluster.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o myamp.o myamp.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o genps.o genps.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o initcluster.o initcluster.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o setscales.o setscales.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o reweight.o reweight.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o get_color.o get_color.f
    gfortran -O -w -ffixed-line-length-132 -w -c -o matrix1.o matrix1.f
    matrix1.f: In function â:
    matrix1.f:119005: internal compiler error: Segmentation fault
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <URL:http://bugzilla.redhat.com/bugzilla> for instructions.
    make: *** [matrix1.o] Error 1

Please try to fix this compilations issue and retry.
Help might be found at https://answers.launchpad.net/madgraph5.
If you think that this is a bug, you can report this at https://bugs.launchpad.net/madgraph5
Value of current Options:
              web_browser : None
              text_editor : None
            cluster_queue : madgraph
         madanalysis_path :
       group_subprocesses : Auto
ignore_six_quark_processes : False
             cluster_mode : 0
             pythia8_path :
   automatic_html_opening : False
          pythia-pgs_path :
                  td_path :
             delphes_path :
             cluster_type : condor
         fortran_compiler : None
      exrootanalysis_path :
               eps_viewer : None
                  nb_core : 8
                 run_mode : 0

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
2012-02-21
Last reply:
2012-02-24
Yuri Gershtein (gershtein) said : #1

a follow-up.
A closer look at the matrix1.f reveals a line that starts like this:

      JAMP(1)=+1./2.*(-AMP(3)-AMP(8)-IMAG1*AMP(43)+AMP(44)-1./3.*IMAG1
     $ *AMP(45)-AMP(57)-IMAG1*AMP(58)-AMP(59)-1./3.*AMP(61)-1./3.
     $ *AMP(62)+IMAG1*AMP(92)-IMAG1*AMP(94)+IMAG1*AMP(95)-IMAG1
     $ *AMP(97)-1./3.*IMAG1*AMP(98)-IMAG1*AMP(101)-1./3.*IMAG1
     $ *AMP(102)+AMP(106)-IMAG1*AMP(117)-IMAG1*AMP(118)+AMP(119)
     $ -AMP(125)-IMAG1*AMP(127)-AMP(128)+AMP(148)-IMAG1*AMP(149)
     $ +IMAG1*AMP(150)-IMAG1*AMP(152)+IMAG1*AMP(153)-1./3.*IMAG1
     $ *AMP(175)-1./3.*IMAG1*AMP(176)-1./3.*AMP(182)-1./3.*AMP(183)

and continues for many thousands of lines. Could it be that the compiler (which, to make things worse,
is called with -O option) chokes trying to process such a long string of additions?

Hi Yuri,

The official limit for the number of jet is currently 4j in final
states.
The problem is that after 4j the description in term of Feynman
diagram is
very heavy.
As you point MG5 is able to produce a valid output, but this one is
much too heavy to be compile in fortran.
(one of the problem is JAMP, another one is the color matrix)

By refactoring the output file, one user/collaborator succeeds to
computes
g g > 7 g
https://indico.cern.ch/contributionDisplay.py?sessionId=22&contribId=114&confId=121170

On our side, we have different idea/ work in progress to solve such
problem, but you will need to be patient.

Cheers,

Olivier

On Feb 21, 2012, at 9:35 AM, Yuri Gershtein wrote:

> Question #188406 on MadGraph5 changed:
> https://answers.launchpad.net/madgraph5/+question/188406
>
> Yuri Gershtein gave more information on the question:
> a follow-up.
> A closer look at the matrix1.f reveals a line that starts like this:
>
> JAMP(1)=+1./2.*(-AMP(3)-AMP(8)-IMAG1*AMP(43)+AMP(44)-1./3.*IMAG1
> $ *AMP(45)-AMP(57)-IMAG1*AMP(58)-AMP(59)-1./3.*AMP(61)-1./3.
> $ *AMP(62)+IMAG1*AMP(92)-IMAG1*AMP(94)+IMAG1*AMP(95)-IMAG1
> $ *AMP(97)-1./3.*IMAG1*AMP(98)-IMAG1*AMP(101)-1./3.*IMAG1
> $ *AMP(102)+AMP(106)-IMAG1*AMP(117)-IMAG1*AMP(118)+AMP(119)
> $ -AMP(125)-IMAG1*AMP(127)-AMP(128)+AMP(148)-IMAG1*AMP(149)
> $ +IMAG1*AMP(150)-IMAG1*AMP(152)+IMAG1*AMP(153)-1./3.*IMAG1
> $ *AMP(175)-1./3.*IMAG1*AMP(176)-1./3.*AMP(182)-1./3.*AMP(183)
>
>
> and continues for many thousands of lines. Could it be that the
> compiler (which, to make things worse,
> is called with -O option) chokes trying to process such a long
> string of additions?
>
> --
> You received this question notification because you are a member of
> MadTeam, which is an answer contact for MadGraph5.

Yuri Gershtein (gershtein) said : #3

Hi Olivier,

how patient would you like me to be?
Is this something in the works close to resolution or is it a long-term project?

Do you understand why the compiler chokes on the output?
Where is the limitation - is it the file size, length of lines, etc?

thanks,
Yuri

Hi Yuri,

>how patient would you like me to be?
>Is this something in the works close to resolution or is it a long-term project?

Personally I don't know how things are moving in that direction. Johan? Fabio?

>Do you understand why the compiler chokes on the output?
>Where is the limitation - is it the file size, length of lines, etc?

I think this both file size and length size, and probably also memory size (for the size of the color matrix)
But Yoshitaro should have more insight about this.

Cheers,

Olivier

Yuri Gershtein (gershtein) said : #5

by the way, we've figured out that once can compile matrix1.f (that was the original error) by hand if -O option is dropped.
I'll try to do that and see how far I'd get.

Johan Alwall (johan-alwall) said : #6

Hello Yuri,

For multijet final states (> 4) work is, as Olivier mentioned, ongoing but it's difficult to give you a definite time-line. If you have a paper in the works, I would suggest not to wait for us but run up to 4 jets with ickkw set to 1 and let Pythia take care of the 5th jet for now, it's usually not doing a bad job for high jet multiplicities (depending on the particular jet configurations you need).

All the best,
Johan

Hi Yuri,

Some time ago I was able to calculate full p p > j j j j j (five jets) process by changing gfortran flags in makefile. As was mentioned before the problem with this process is mainly the size of color matrix. By adding flag -fno-automatic you force compiler to flush parts that are ready out from memory (reducing optimization level is than not necessary, you can actually increase it to get faster code). I got something like 2k jobs so this must be run on a cluster. Mind you, that although I got a result I didn't check it's validity. It's highly probable that this trick can brake something so you need to validate somehow your result.

Best regard,
Wojtek

Yuri Gershtein (gershtein) said : #8

Hi,

thanks everyone for comments.

Even without -O flag, 10 out of 127 matrix*.f files gave seg fault.

I definitively narrowed it down to the part of the code where
JAMP()'s are filled. The length of the lines is HUGE. I wrote a simple code that splits that line into several,
and things compile. Even the ones that compiled before compile much, much faster.

I think that one simple change that authors can make in the next version is to ask PYTHON to split it when the
file is generated.

Another important but minor change needs to be made to the symfact.dat.
Currently the format is too short, so the two integers merge if the second is negative, i.e.
 54054-17847

I just replaced '-' with ' -' and that worked.

regards,
Yuri

Thanks for those comments,

> I think that one simple change that authors can make in the next
> version is to ask PYTHON to split it when the
> file is generated.

Johan, would it be possible to do that in the FortranWriter class?
(If a line is too big and start by "XXXX =" split it automatically in
a smart way?)

> Another important but minor change needs to be made to the
> symfact.dat.
> Currently the format is too short, so the two integers merge if the
> second is negative, i.e.
> 54054-17847
>
> I just replaced '-' with ' -' and that worked.

That I don't understand, the line formating as a mandatory space...

Note that for such process, you probably also need to change the size
definition of various vector.
like
       integer max_amps
       parameter (max_amps=9999)

On 23-févr.-12, at 23:20, Yuri Gershtein wrote:

> Question #188406 on MadGraph5 changed:
> https://answers.launchpad.net/madgraph5/+question/188406
>
> Yuri Gershtein posted a new comment:
> Hi,
>
> thanks everyone for comments.
>
> Even without -O flag, 10 out of 127 matrix*.f files gave seg fault.
>
> I definitively narrowed it down to the part of the code where
> JAMP()'s are filled. The length of the lines is HUGE. I wrote a
> simple code that splits that line into several,
> and things compile. Even the ones that compiled before compile much,
> much faster.
>
> I think that one simple change that authors can make in the next
> version is to ask PYTHON to split it when the
> file is generated.
>
> Another important but minor change needs to be made to the
> symfact.dat.
> Currently the format is too short, so the two integers merge if the
> second is negative, i.e.
> 54054-17847
>
> I just replaced '-' with ' -' and that worked.
>
> regards,
> Yuri
>
> --
> You received this question notification because you are a member of
> MadTeam, which is an answer contact for MadGraph5.

Johan Alwall (johan-alwall) said : #10

Hello Yuri, Olivier

> I think that one simple change that authors can make in the next
> version is to ask PYTHON to split it when the
> file is generated.

> Johan, would it be possible to do that in the FortranWriter class?
> (If a line is too big and start by "XXXX =" split it automatically in
> a smart way?)

This can be done, either in the FortranWriter or in the generating code. However, the code is really not meant to generate so big processes, so I would definitely suggest to use 4j+Pythia for now and wait for the multiparton code, which should be out sometime this spring.

> Another important but minor change needs to be made to the
> symfact.dat.
> Currently the format is too short, so the two integers merge if the
> second is negative, i.e.
> 54054-17847
>
> I just replaced '-' with ' -' and that worked.

I think the formatting is indeed with a fixed number of spaces. This we should certainly fix. Thanks!

> Note that for such process, you probably also need to change the size
> definition of various vector.
> like
> integer max_amps
> parameter (max_amps=9999)

I'm pretty sure there should be no such places left (they should all be generated by the python code). Let me know if you find any, and I'll make sure to fix them.

Cheers,
Johan

Hi,

Just to say that
symfact.dat should be correctly formatted in 1.4.3 even with such large number of diagram
and that all fixed memory stuff that I pointed in my precedent post are also now passed as dynamical value.

On the other hand, the automatic splitting of the line is NOT implemented. (maybe for a future version)

Cheers,

Olivier

Can you help with this problem?

Provide an answer of your own, or ask Yuri Gershtein for more information if necessary.

To post a message you must log in.