Basic Problem using parallel run in Yade

Asked by Rioual on 2020-03-04

Hello,

I try to use the possibility of parallel calculations with Yade for the first time thanks to Openmp.
This is a very basic question, sorry.
When I add -j X to the Yade command where X is the number of available cores
 of my machine then I can check with htop that there is only one core running
at 100 %CPU ; the other cores seem not to be called for running by Yade (S state).
I find the same behavior using yade-2016.06a on my machine and yade-2019-01-28 on a cluster.

Did I miss something in the process of using Yade in parallel ??

Thanks for your cooperation,

Best

Vincent

Question information

Language:
English Edit question
Status:
Solved
For:
Yade Edit question
Assignee:
No assignee Edit question
Solved by:
Rioual
Solved:
2020-03-11
Last query:
2020-03-11
Last reply:
2020-03-10
Robert Caulk (rcaulk) said : #1

Hello,

It depends on the script you are running.

Cheers,

Robert

Rioual (francois-rioual-v) said : #2

Dear Robert,

Can you be more precise ??
How can I adapt my script in order to have the benefits
 of parallel computing ??

Thanks,
Cheers,

V.

Robert Caulk (rcaulk) said : #3

-It is possible there are two few spheres to take advantage of parallel computing.
-It is possible you have an expensive PyRunner running on each iteration and consuming the majority of the computational time

No way to know without a script.

Robert Caulk (rcaulk) said : #5

Please use yade.timing module and provide yade.timing.stats [1]

[1]https://gitlab.com/yade-dev/trunk/-/blob/master/examples/FluidCouplingPFV/oedometer.py#L166

> plot.plot()

Not a minimal script.
https://www.yade-dem.org/wiki/Howtoask (3)
Please.
Bruno

Rioual (francois-rioual-v) said : #7

.... OK, here is a self contained shorter script that reproduces the problem, I got rid of the stl importations.

Thanks for your feed-back,
Best,
Vincent,

#***************************************************************
#SCRIPT:
#***************************************************************

#gravity deposition (1), (2) continuing with oedometric test after stabilization (3)

# load parameters from file if run in batch
# default values are used if not run from batch
readParamsFromTable(rMean=0.005,rRelFuzz=0.002,maxLoad=1e7,minLoad=1e5)
# make rMean, rRelFuzz, maxLoad accessible directly as variables later
from yade.params.table import *

# create box with free top, and ceate loose packing inside the box
from yade import pack, plot

## PhysicalParameters
Density=2400
frictionAngle=radians(25)
tc = 0.01
en = 0.0001
et = 0.0001

## Import wall's geometry
facetMat=O.materials.append(ViscElMat(frictionAngle=frictionAngle,tc=tc, en=en, et=et))
sphereMat=O.materials.append(ViscElMat(density=Density,frictionAngle=frictionAngle,tc=tc,en=en,et=et))
from yade import ymport

fctIdscylinder = O.bodies.append(geom.facetBox((.5,.5,.5),(.5,.5,.5),wallMask=55,material=facetMat))

###########################################

sp=pack.SpherePack()
sp.makeCloud((0,0,0),(1,1,1),rMean=rMean,rRelFuzz=rRelFuzz)
sp.toSimulation(color=(0,0,1),material=sphereMat)

O.engines=[
 ForceResetter(),
 # sphere, facet, wall
 InsertionSortCollider([Bo1_Sphere_Aabb(),Bo1_Facet_Aabb(),Bo1_Wall_Aabb()]),
 InteractionLoop(
  # the loading plate is a wall, we need to handle sphere+sphere, sphere+facet, sphere+wall
  [Ig2_Sphere_Sphere_ScGeom(),Ig2_Facet_Sphere_ScGeom(),Ig2_Wall_Sphere_ScGeom()],
  [Ip2_FrictMat_FrictMat_FrictPhys()],
  [Law2_ScGeom_FrictPhys_CundallStrack()]
 ),
 NewtonIntegrator(gravity=(0,-9.81,0),damping=0.1),
 # the label creates an automatic variable referring to this engine
 # we use it below to change its attributes from the functions called
 PyRunner(command='checkUnbalanced()',realPeriod=2,label='checker'),
]
O.dt=.5*PWaveTimeStep()

# the following checkUnbalanced, unloadPlate and stopUnloading functions are all called by the 'checker'
# (the last engine) one after another; this sequence defines progression of different stages of the
# simulation, as each of the functions, when the condition is satisfied, updates 'checker' to call
# the next function when it is run from within the simulation next time

# check whether the gravity deposition has already finished
# if so, add wall on the top of the packing and start the oedometric test

def checkUnbalanced():
 # at the very start, unbalanced force can be low as there is only few contacts, but it does not mean the packing is stable
 if O.iter<9000: return
 # the rest will be run only if unbalanced is < .1 (stabilized packing)
 if unbalancedForce()>0.2: return

 # add plate at the position on the top of the packing

 fctIdsbouchonI = O.bodies.append(wall(max([b.state.pos[1]+b.shape.radius for b in O.bodies if isinstance(b.shape,Sphere)]),axis=1,material=facetMat))

 global fctIdsbouchon

 fctIdsbouchon=[]
        fctIdsbouchon.append(fctIdsbouchonI)

 TransEngload= TranslationEngine(ids=fctIdsbouchon,translationAxis=[0,-1,0],velocity=1,label='load')

 O.engines=O.engines+[TransEngload]

 global TransEngload

 # next time, do not call this function anymore, but the next one (unloadPlate) instead
 checker.command='unloadPlate()'

def unloadPlate():
 # if the force on plate exceeds maximum load, start unloading

 Fn = sum(O.forces.f(O.bodies[facetid].id)[1] for facetid in fctIdsbouchon)

 if abs(Fn)>maxLoad:

  TransEngload.Velocity = 0
  TransEngunload = TranslationEngine(ids=fctIdsbouchon,translationAxis=[0,1,0],velocity=0.1,label='unload')
  O.engines=O.engines+[TransEngunload]
  # next time, do not call this function anymore, but the next one (stopUnloading) instead
  checker.command='stopUnloading()'

def stopUnloading():

 Fn = sum(O.forces.f(O.bodies[facetid].id)[1] for facetid in fctIdsbouchon)

 if abs(Fn)<minLoad:

# On supprime le bouchon
  for facet in fctIdsbouchon:
   O.bodies.erase(facet)

  O.save('init_Final_state_packing.yade') # on enregistre le resultats de la simulation au format .yade
  print '********fin de la construction de l''empilement***********' # impression d'un message de fin
  O.pause()

from yade import timing

O.run()

O.wait()

timing.stats()

#***************************************************************
#END SCRIPT:
#***************************************************************

Jan Stránský (honzik) said : #8

Please follow Robert's #5 request:
> ... provide yade.timing.stats [1]
thanks
Jan

Rioual (francois-rioual-v) said : #9

Hello,

Thanks for your suggestions.
Actually I tested the code for a lower number of particles and several cores are used in this case, it works....
The point is that for a higher number of particles, the script is stuck at the operation " sp.makeCloud((0,0,0),(1,1,1),rMean=rMean,rRelFuzz=rRelFuzz)" (at the beginning of the script, before the engines) and the script
doesn't seem to use the different cores required, at this stage.
Would there eventually be an efficient way to create a cloud of particles for a high number of particles ???

Thanks for your reply,

Best,
Vincent

Chareyre (bruno-chareyre-9) said : #10

So in the end the question is not about any parallel portion of the code.

Duplicate a small cloud?

Note that nothing is «stuck». It just takes more time (quadratically i
think).

Bruno

Le ven. 6 mars. 2020 13:32, Rioual <email address hidden> a
écrit :

> Question #689131 on Yade changed:
> https://answers.launchpad.net/yade/+question/689131
>
> Status: Needs information => Open
>
> Rioual gave more information on the question:
> Hello,
>
> Thanks for your suggestions.
> Actually I tested the code for a lower number of particles and several
> cores are used in this case, it works....
> The point is that for a higher number of particles, the script is stuck at
> the operation "
> sp.makeCloud((0,0,0),(1,1,1),rMean=rMean,rRelFuzz=rRelFuzz)" (at the
> beginning of the script, before the engines) and the script
> doesn't seem to use the different cores required, at this stage.
> Would there eventually be an efficient way to create a cloud of particles
> for a high number of particles ???
>
> Thanks for your reply,
>
> Best,
> Vincent
>
> --
> You received this question notification because your team yade-users is
> an answer contact for Yade.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~yade-users
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~yade-users
> More help : https://help.launchpad.net/ListHelp
>
>
>

Jan Stránský (honzik) said : #11

the makeCloud code is not paraleized.
If you need it for really very high number of particles, consider creating a smaller packing and copy it wherever needed (as suggested by Bruno)
cheers
Jan

Rioual (francois-rioual-v) said : #12

...Thank you very much for these suggestions.
But basically, how can I make this duplication of packing and write it in Yade??
How do I glue the pieces of packing to make a large global packing ??

Thanks for your input,
Best,
Vincent

Jan Stránský (honzik) said : #13

Have a look how pack.randomDensePack does it [1] using sp.cellFill [2] (and optionally sp.filterSPherePack [3])

A MWE:
###
from yade import pack
r = 0.005
num = 20
dimSmall = (0.04, 0.04, 0.04)
dimFull = (1, 1, 1)
sp = pack.SpherePack()
sp.makeCloud((0,0,0),dimSmall,r,num=num,periodic=True,seed=1)
print(len(sp),sp.dim()) # 20 particles in this case
sp.cellFill(dimFull)
print(len(sp),sp.dim()) # 312500 particles in this case (is "instantaneous")
#filterSpherePack(...) # if needed
#sp.toSimulation() # takes 30 s on my laptop
###

sp.toSimulation() might take some time, adding to simulation is not prallelized :-)
but should be significantly faster than makeCloud with all the particles

cheers
Jan

[1] https://gitlab.com/yade-dev/trunk/-/blob/master/py/pack/pack.py#L560
[2] https://yade-dem.org/doc/yade.pack.html#yade._packSpheres.SpherePack.cellFill
[3] https://yade-dem.org/doc/yade.pack.html#yade.pack.filterSpherePack

Jérôme Duriez (jduriez) said : #14

> How do I glue the pieces of packing to make a large global packing ??

makeCloud will anyway not reach this goal, if I understand correctly what you have in mind.

You have anyway to compress your initial cloud to reach a dense, solid- (not gas-)like state. Before this subsequent compression, you can as well replicate a smaller cloud as many times as you want, shifting it in space at each replication, in order to get a bigger cloud.

See e.g. https://yade-dem.org/doc/yade.pack.html#yade._packSpheres.SpherePack.translate for the shift in space

> You have anyway to compress your initial cloud

This is not my suggestion.
Accelerating makeCloud is efficiently exemplified in #13, instead.
I was expecting some python loops in the script to duplicate/shift an inititial packing, but I learned the existence of "sp.cellFill()" which makes it automatically. Thanks Jan!
B

Rioual (francois-rioual-v) said : #16

...I will try that; looks very relevant...

All the best

V.