O.load and O.loadTmp influences calculation speed ?!

Asked by Christian Jakob on 2012-04-03

When I measure calculation time for a model I detect differences when using O.load or O.loadTmp.
For an example with 8000 spheres I get

32 seconds for 1000 steps (without O.load or O.loadTmp)

and

43 seconds for 1000 steps (with O.load or O.loadTmp)

Can someone explain that differences?

Here the part for time measurement in my script:

 O.load('model-speed-test-start-condition.xml')
 O.run(10,True)
 start_time = O.realtime
 O.run(1000,True)
 stop_time = O.realtime

 calc_time_sec = stop_time-start_time

Question information

Language:
English Edit question
Status:
Solved
For:
Yade Edit question
Assignee:
No assignee Edit question
Solved by:
Christian Jakob
Solved:
2012-04-03
Last query:
2012-04-03
Last reply:
2012-04-03

No surprise: load() reads HDrive while loadTmp() reads in RAM, which is
way faster.
In your case, it is even worst since you save in xml format. It would be
faster in binary (give filename "model.yade").
You can also save disk space space by using compressed files (give
filename filename "model.yade.gz" or "model.xml.gz")

Christian Jakob (jakob-ifgt) said : #2

>No surprise: load() reads HDrive while loadTmp() reads in RAM, which is
>way faster.

Thats clear to me, but this is not what I mean. Lets assume, I use only O.loadTmp().

If I create a model, this model information is stored in RAM.
If I save this information with O.saveTmp() and load it afterwards with O.loadTmp(), it is also loaded in RAM.

While I do not measure loading time, just calculation time, why are there differences in calculation time when running the script without O.loadTmp() and with O.loadTmp()?
It influences the calculation time! Thats what I do not understand.

Anton Gladky (gladky-anton) said : #3

Try to use timings to see, where is the problem exactly:

https://github.com/yade/trunk/blob/master/examples/packs/packs.py#L150

Anton

Oh ok. I didn't read your question carefully enough, sorry.
Is it something you can reproduce? Which is the fastest one, loadTmp() or load()?

Christian Jakob (jakob-ifgt) said : #5

#To reproduce, you can use this script:

load_ = 1 #=0 for NOT using O.loadTmp(), =1 for using O.loadTmp()

num_balls1D = 20
num_balls = num_balls1D*num_balls1D*num_balls1D
origin_wall = num_balls1D/2
x_pos = 0
y_pos = 0
z_pos = 0
friction=0.5
angle=atan(friction)

id_FacetMat=O.materials.append(ViscElMat(kn=1e8,ks=1e8,cn=0.0,cs=0.0,frictionAngle=angle))
id_SphereMat=O.materials.append(ViscElMat(kn=1e6,ks=1e6,cn=0.0,cs=0.0,density=1000,frictionAngle=angle))

FacetMat=O.materials[id_FacetMat]
SphereMat=O.materials[id_SphereMat]

O.engines=[
 ForceResetter(),
 InsertionSortCollider([Bo1_Sphere_Aabb(),Bo1_Facet_Aabb()]),
 InteractionLoop(
  [Ig2_Sphere_Sphere_ScGeom6D(),Ig2_Facet_Sphere_ScGeom()],
  [Ip2_ViscElMat_ViscElMat_ViscElPhys()],
  [Law2_ScGeom_ViscElPhys_Basic()],
 ),
 NewtonIntegrator(damping=0.7,gravity=(0,0,-9.81)),
]

from yade import timing
O.timingEnabled=True

O.dt=1e-3

#create spheres:
for ii in range(1,num_balls1D+1):
 x_pos = x_pos + 1
 y_pos = 0
 z_pos = 0
 for jj in range(1,num_balls1D+1):
  y_pos = y_pos + 1
  z_pos = 0
  for kk in range(1,num_balls1D+1):
   z_pos = z_pos + 1
   O.bodies.append(utils.sphere([x_pos,y_pos,z_pos], material=SphereMat, radius=0.5))

#create facet:
id_facet=O.bodies.append(geom.facetBox((origin_wall,origin_wall,0),(200,200,0),material=FacetMat))

O.run(100,True)

if load_ == 1:
 O.saveTmp()
 O.loadTmp()

start_time = O.realtime
O.run(1000,True)
stop_time = O.realtime

calc_time_sec = stop_time-start_time

print calc_time_sec

timing.stats()

Christian Jakob (jakob-ifgt) said : #6

Results with load_ = 0:

me@debian ~/calc-time-test-results/speed-test-YADE-201204 >yadetest test-loadTmp.py
Welcome to Yade unknown
TCP python prompt on localhost:9001, auth cookie `yedksa'
XMLRPC info provider on http://localhost:21001
Running script test-loadTmp.py
36.251
Name Count Time Rel. time
-------------------------------------------------------------------------------------------------------
ForceResetter 1100 210349us 0.52%
InsertionSortCollider 14 452465us 1.13%
InteractionLoop 1100 34746997us 86.51%
NewtonIntegrator 1100 4753948us 11.84%
TOTAL 40163759us 100.00%

Results with load_ = 1:

me@debian ~/calc-time-test-results/speed-test-YADE-201204 >yadetest test-loadTmp.py
Welcome to Yade unknown
TCP python prompt on localhost:9001, auth cookie `ucseka'
XMLRPC info provider on http://localhost:21001
Running script test-loadTmp.py
45.425
Name Count Time Rel. time
-------------------------------------------------------------------------------------------------------
ForceResetter 1000 186531us 0.41%
InsertionSortCollider 11 364810us 0.80%
InteractionLoop 1000 38993167us 85.87%
NewtonIntegrator 1000 5866674us 12.92%
TOTAL 45411183us 100.00%

Hehehe

You forgot to mention these in your question:

if bla:
  O.saveTmp()
  O.loadTmp()
start_time = O.realtime
O.run(1000,True)
stop_time = O.realtime

So, you are not comparing loadTmp() vs. load(). You comparing loading vs. NOT loading.

Comparing would need this:
if bla:
  O.saveTmp()
  O.loadTmp()
else:
   O.save("bla.yade")
   O.loadTmp("bla.yade")

Clearly, some internals are not saved with the simulation (e.g. the collider has to regenerate itself), so loading introduces some initialization operations.
In addition, be careful, you are not comparing the same number of iterations in your exemple above (1000 vs. 1100).

Christian Jakob (jakob-ifgt) said : #8

Aaaah, loading a saved model leads to more calculation time for initial steps. Ok, that was the problem with my script:

###wrong:

O.run(100,True)

if load_ == 1:
 O.saveTmp()
 O.loadTmp()

start_time = O.realtime
O.run(1000,True)
stop_time = O.realtime

###right:

if load_ == 1:
 O.saveTmp()
 O.loadTmp()

O.run(100,True)

start_time = O.realtime
O.run(1000,True)
stop_time = O.realtime

Thank you.