Directory naming when using mode output for plugin

Hi Zachary,
actually I never used the plugins this way.
What I do is to start mg5_amc with the --mode=MadSTR flag

./bin/mg5_aMC --mode=MadSTR

then, generate/output as usual.

Note that MadSTR also changes the behaviour of the generate command, when relevant (for your process, ttbar, there are no on-shell resonances appearing in the real emissions, so it should have no effect)

Let me know if you need more

Cheers,

Marco

Hi Marco,

Sure. Two comments about that method.

First, the code in mg5_aMC is:

if options.plugin:
    if not os.path.exists(os.path.join(root_path, 'PLUGIN', options.plugin)):
        print( "ERROR: %s is not present in the PLUGIN directory. Please install it first")
...

So if the plugin is not directly installed as a part of the MadGraph5_aMC@NLO installation, this doesn't work. This breaks the extensibility of the PLUGIN system.

Second, as I mentioned, it looks to a naive user like me as though this "mode" and the command-line mode should be interchangeable. If they're not – sounds like they aren't for this plugin? – then it'd be great to make that more clear. It'd also be great to offer a way of using these plugins that doesn't require a different command-line argument; can we use multiple plugins at the same time? If so, how right now?

Thanks,
Zach

Hi Zachary,
right, you need to install the plugin first
(either with install MadSTR from the mg5amc command interface, or manually).

For some plugins, those that only affect the output command, the command line specification should work. However, this is not the case for MadSTR.
About the inter-compatibility of different plugins, this is not at all guaranteed.

Cheers,

Marco

Hi Marco,

Ok, this is a shame. I thought one of the key points of the PLUGIN system was that it would be extensible – just like models are. There's documentation around the use of the MG5aMC_PLUGIN directory in PYTHONPATH (or MG5aMC_PLUGIN environment variable, depending on what documentation I read, but the directory name seems to be the one that's really used in the code). Can we at least try to use the same code as in extended_cmd.py?

    if 'PYTHONPATH' in os.environ:
        for PluginCandidate in os.environ['PYTHONPATH'].split(':'):
            try:
                dirlist = os.listdir(PluginCandidate)
            except OSError:
                continue
            for onedir in dirlist:
                if onedir == 'MG5aMC_PLUGIN':
                    plugin_path.append(pjoin(PluginCandidate, 'MG5aMC_PLUGIN'))
                    break
            else:
                continue
            break

which allows extensions through the PYTHONPATH?

I agree that if something affects generate, then having a mode in the output command is insufficient. What I was proposing was something like

import plugin MadSTR

which would do the same as --mode=MadSTR without requiring a command-line change (and could be used consistently for all plugins).

Cheers,
Zach

Hi Zachary,

> if options.plugin:
> if not os.path.exists(os.path.join(root_path, 'PLUGIN', options.plugin)):
> print( "ERROR: %s is not present in the PLUGIN directory. Please install it first")
> ...
>
> So if the plugin is not directly installed as a part of the
> MadGraph5_aMC@NLO installation, this doesn't work. This breaks the
> extensibility of the PLUGIN system.

I will double check this.
As far as I remember, a way exists for plugin to be in PYTHONPATH, but maybe I have not did it for this type of PLUGIN.

> Second, as I mentioned, it looks to a naive user like me as though this
> "mode" and the command-line mode should be interchangeable. If they're
> not – sounds like they aren't for this plugin? – then it'd be great to
> make that more clear. It'd also be great to offer a way of using these
> plugins that doesn't require a different command-line argument; can we
> use multiple plugins at the same time? If so, how right now?

As marco said, they are various type of plugin.
Some of them does define a new output type and some other does not
(like cluster related plugin and/or reweighting related plugin/...)

Some PLUGIN defines their own interface (like maddm, maddump and ... madstr)
and for those, you do have a dedicated script in the bin directory (when install via the install command)
./bin/madstr.py
which actually redirects to the ./bin/mg5_aMC --mode=madstr

> can we
> use multiple plugins at the same time? If so, how right now?

Well in principle each "type" of plugin should be independent of each other.
So you should be able to change
- the output
- the reweight module
- the cluster submition
with three different plugin.

When a plugin change the interface, they are no way to insure inter-operability with the other type of module.
Since they can de some deep change (and that's the point)

Concerning the idea of doing:
"""
import plugin MadSTR
"""
This will be quite costly to implement and can creates "political" issue.
Since it is important (in general) that such type of PLUGIN have their own visibility and therefore their own executables.
But let me think more about it.

Cheers,

Olivier

> On 4 Nov 2022, at 11:15, Zachary Marshall <email address hidden> wrote:
>
> Question #703711 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/703711
>
> Zachary Marshall gave more information on the question:
> Hi Marco,
>
> Sure. Two comments about that method.
>
> First, the code in mg5_aMC is:
>
> if options.plugin:
> if not os.path.exists(os.path.join(root_path, 'PLUGIN', options.plugin)):
> print( "ERROR: %s is not present in the PLUGIN directory. Please install it first")
> ...
>
> So if the plugin is not directly installed as a part of the
> MadGraph5_aMC@NLO installation, this doesn't work. This breaks the
> extensibility of the PLUGIN system.
>
> Second, as I mentioned, it looks to a naive user like me as though this
> "mode" and the command-line mode should be interchangeable. If they're
> not – sounds like they aren't for this plugin? – then it'd be great to
> make that more clear. It'd also be great to offer a way of using these
> plugins that doesn't require a different command-line argument; can we
> use multiple plugins at the same time? If so, how right now?
>
> Thanks,
> Zach
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Thanks Olivier! Would you like a separate “Question” to track these other items?

Best,
Zach

HI,

As you want, as usual on topic per thread will allow to have all topic to reach completion.

So keeping this thread for plugin path issue here.
I do see in 3.4.1 the following lines:

> if options.plugin:
> if not os.path.exists(os.path.join(root_path, 'PLUGIN', options.plugin)):
> print( "ERROR: %s is not present in the PLUGIN directory. Please install it first")
> __import__('PLUGIN.%s' % options.plugin)
> plugin = sys.modules['PLUGIN.%s' % options.plugin]
> if not plugin.new_interface:
> logging.warning("Plugin: %s do not define dedicated interface and should be used without the --mode options" % op\
> tions.plugin)
> sys.exit()

So even if it is true that if the directory does not exist in the PLUGIN directory, you will get the error message, the code is actually not crashing and still tries to do __import__('PLUGIN.%s' % options.plugin)
So if your pythonpath is setup in a way to be compatible with such call, then it should be working.

Those line seems to be implement 7 years ago (but the print statement one -> 4 years ago updated because of python3 support).
So this should not be MG5aMC version dependent.

So is this not working?

Cheers,

Olivier

> On 4 Nov 2022, at 12:05, Zachary Marshall <email address hidden> wrote:
>
> Question #703711 on MadGraph5_aMC@NLO changed:
> https://answers.launchpad.net/mg5amcnlo/+question/703711
>
> Status: Answered => Open
>
> Zachary Marshall is still having a problem:
> Thanks Olivier! Would you like a separate “Question” to track these
> other items?
>
> Best,
> Zach
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Thanks Olivier! Unfortunately, this doesn't seem to work, no.

Earlier within the executable, you push the MadGraph directory onto the front of the PYTHONPATH:

root_path = os.path.split(os.path.dirname(os.path.realpath( __file__ )))[0]
sys.path.insert(0, root_path)

when the import goes, it finds PLUGIN in the MadGraph directory. That module doesn't have the submodule we want (MadSTR), so we get an import error (it doesn't continue to traverse the PYTHONPATH, because it's already located the PLUGIN module). I think we could work around that by defining a __path__ variable in the __init__.py of the PLUGIN module. I think that'll work for us in this case – but again, this makes these PLUGINs not really extensible.

Thanks again,
Zach

Thanks very much, Olivier! I think this will sort our use-case out, yes!

Best,
Zach

Thanks Olivier Mattelaer, that solved my question.