Yade

About batch mode

Asked by Cloud

Hello guys,

There is a problem that bothers me now. My computer has Intel® Core™ i7-8700 CPU @ 3.20GHz × 12, I'm trying to make full use of CPU in batch mode. Here is my input in the terminal:

yadedaily-batch -j12 paras.table meizizi.py

Then the terminal reply :

Will run simulation(s) ['meizizi.py'] using `/usr/bin/yadedaily', nice value 10, using max 12 cores.
Will use table `paras.table', with available lines 2, 3, 4, 5.
......

but I check it in http://localhost:9080, the page shows

Pid 14186
12 cores available, 4 used + 8 free.

So it doesn't seem to take full advantage of CPU, What is going on here?

Regards,
Cloud

Question information

Language:
English Edit question
Status:
Solved
For:
Yade Edit question
Assignee:
No assignee Edit question
Solved by:
Chareyre
Solved:
Last query:
Last reply:
Revision history for this message
Jan Stránský (honzik) said : 		#1

Hello,

> What is going on here?

see
yadedaily-batch --help
and --job-threads option. The default value is 1, therefore you see 4 cores, for each task 1 core

> to take full advantage of CPU

this is a tricky part.
If your simulations take comparable amount of time, I would use
yadedaily-batch -j 12 --job-threads 3 ...
to use 12 cores, but to decrease "parallel overhead"
If your simulations differ very much in length, you can use --job-threads 12 not to wait too long for unfinished jobs.
BUT, it may happen that using 12 cores take more time then, e.g., 6 cores :-)

So to take really "full advantage of CPU", you have to play a bit :-)

cheers
Jan

Revision history for this message
Best Chareyre (bruno-chareyre-9) said : 		#2

Hi,
You need to assign openmp threads to each job if you want them to run in
parallel, by default they take one core each. See batch example for details.

I doubt running 4x3 cores will be very fast though. The OS needs something
to run itself, and there are hardware limitations to.
Bruno

Le mer. 13 mars. 2019 04:52, Cloud <email address hidden> a
écrit :

> New question #679153 on Yade:
> https://answers.launchpad.net/yade/+question/679153
>
> Hello guys,
>
> There is a problem that bothers me now. My computer has Intel® Core™
> i7-8700 CPU @ 3.20GHz × 12, I'm trying to make full use of CPU in batch
> mode. Here is my input in the terminal:
>
> yadedaily-batch -j12 paras.table meizizi.py
>
> Then the terminal reply :
>
> Will run simulation(s) ['meizizi.py'] using `/usr/bin/yadedaily', nice
> value 10, using max 12 cores.
> Will use table `paras.table', with available lines 2, 3, 4, 5.
> ......
>
> but I check it in http://localhost:9080, the page shows
>
> Pid 14186
> 12 cores available, 4 used + 8 free.
>
> So it doesn't seem to take full advantage of CPU, What is going on here?
>
> Regards,
> Cloud
>
> --
> You received this question notification because your team yade-users is
> an answer contact for Yade.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~yade-users
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~yade-users
> More help : https://help.launchpad.net/ListHelp
>

Revision history for this message
Jérôme Duriez (jduriez) said : 		#3

Hi,

From yade-batch -h or yadedaily-batch -h, we can see that -jX is a maximum of cores to be used by the whole batch process.

The number of cores a batch process requires, depends upon
- the number of lines in paras.table. Here, your table defines just 4 cases ("available lines 2, 3, 4, 5").
- the number of thread for each case of the batch simulation = for each line in paras.table. This is --job-threads option, and it defaults to 1

In the end, what your whole batch simulation requires, as presently defined, is just 4 threads ie cores (I'm mixing the two words..). It's then normal 8 out of your 12 cores have nothing to do.

It seems what you want is the --job-threads X option. Probably
yadedaily-batch --job-threads 3 paras.table meizizi.py

will ask your computer everything it has.

Revision history for this message
Cloud (strife-cloud) said : 		#4

Thanks Chareyre, that solved my question.

Revision history for this message
Cloud (strife-cloud) said : 		#5

Thank you very much !