esys-escript

running escript in parallel

Asked by Louise Olsen-Kettle

I am not sure of the best inputs to my PBS script for running escript in parallel in terms of number of nodes: processors to threads. For the supercomputer I'm using it needs: min nodes/max ppn is 9/12, ideal is 10.

Is there a way to choose number of nodes, processors and threads to optimise the code? This is the pbs script I am currently running.

#!/bin/csh
# specify the queue name
#PBS -q gstar
# resource requests
#PBS -l nodes=3:ppn=12
#PBS -l walltime=01:00:00:00

# list the assigned CPUs and GPUs
echo Deploying job to CPUs ...
cat $PBS_NODEFILE
echo and using GPU ...
cat $PBS_GPUFILE

echo Working directory is $PBS_O_WORKDIR
cd $PBS_O_WORKDIR

# run process

module purge
module load escript/x86_64/gnu/5.1
run-escript -n3 -p12 -t4 FlawAngle75SanjibRFPA.py

Question information

Language:
English Edit question
Status:
Answered
For:
esys-escript Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
Bob (caltinay) said : 		#1

Hi Louise,

Firstly, I think in your example you are oversubscribing:
You requested 3 x 12 cores but are asking escript to use 3 x 12 x 4. This will negatively affect performance.

In terms of what you should use, there is no one correct answer as it depends on a lot of factors (including domain configuration, solver type, CPU/memory/NUMA setup etc).
But from experience it's worthwhile trying a few options and using the one that performs best on the platform. In your example with 3 nodes and 12 cores per node I would try:

-n3 -p2 -t6
-n3 -p4 -t3
-n3 -p6 -t2

Most importantly avoid oversubscribing.

From the information you provided it's not possible to tell how many nodes you should be using. It really depends on the problem size.

Can you help with this problem?

Provide an answer of your own, or ask Louise Olsen-Kettle for more information if necessary.

To post a message you must log in.