MadGraph5_aMC@NLO

htcondor compatibility for lxplus

Asked by Josh McFayden

Hi,

As far as I can tell the CondorCluster mode is not really suited to htcondor on lxplus. Specifically, I can't set the queue I want it to go to (i.e. the JobFlavour). The relevant lines in 2.7.0 are madgraph/various/cluster.py: L878 and 953 (one for submit() and one for submit2()):
            requirement = 'Requirements = %s=?=True' % self.cluster_queue

The suggested syntax from the documentation is a line in the submit file like this:
+JobFlavour = "<queue name>"
(see https://batchdocs.web.cern.ch/local/submit.html#job-flavours)

Would it be possible to either pass the requirements as a completely free field e.g. I set cluster_queue='+JobFlavour = "workday"' and then have:

            requirement = '%s' % self.cluster_queue

Or perhaps better leave the backwards compatibility for everything except JobFlavour e.g.:

        if 'JobFlavour' in self.cluster_queue:
            requirement = '%s' % self.cluster_queue
        elif self.cluster_queue not in ['None', None]:
            requirement = 'Requirements = %s=?=True' % self.cluster_queue
        else:
            requirement = ''

Or maybe even neater: create an HTCondorCluster class and then it would be possible to do whatever htcondor-specific things we want?

Cheers,

Josh

Question information

Language:
English Edit question
Status:
Answered
For:
MadGraph5_aMC@NLO Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Olivier Mattelaer (olivier-mattelaer) said : 		#1

Hi,

My advice in your case is to create a PLUGIN class and setup your cluster as you want.
This is what CMS is actually doing.

Cheers,

Olivier

https://cp3.irmp.ucl.ac.be/projects/madgraph/wiki/Plugin

> On 31 Jan 2020, at 16:43, Josh McFayden <email address hidden> wrote:
>
> New question #688386 on MadGraph5_aMC@NLO:
> https://answers.launchpad.net/mg5amcnlo/+question/688386
>
> Hi,
>
> As far as I can tell the CondorCluster mode is not really suited to htcondor on lxplus. Specifically, I can't set the queue I want it to go to (i.e. the JobFlavour). The relevant lines in 2.7.0 are madgraph/various/cluster.py: L878 and 953 (one for submit() and one for submit2()):
> requirement = 'Requirements = %s=?=True' % self.cluster_queue
>
> The suggested syntax from the documentation is a line in the submit file like this:
> +JobFlavour = "<queue name>"
> (see https://batchdocs.web.cern.ch/local/submit.html#job-flavours)
>
> Would it be possible to either pass the requirements as a completely free field e.g. I set cluster_queue='+JobFlavour = "workday"' and then have:
>
> requirement = '%s' % self.cluster_queue
>
> Or perhaps better leave the backwards compatibility for everything except JobFlavour e.g.:
>
> if 'JobFlavour' in self.cluster_queue:
> requirement = '%s' % self.cluster_queue
> elif self.cluster_queue not in ['None', None]:
> requirement = 'Requirements = %s=?=True' % self.cluster_queue
> else:
> requirement = ''
>
> Or maybe even neater: create an HTCondorCluster class and then it would be possible to do whatever htcondor-specific things we want?
>
> Cheers,
>
> Josh
>
>
>
> --
> You received this question notification because you are an answer
> contact for MadGraph5_aMC@NLO.

Can you help with this problem?

Provide an answer of your own, or ask Josh McFayden for more information if necessary.

To post a message you must log in.