Memory usage is too high

Asked by Chu on 2019-08-23

I'm facing big consumptions of memory with my simulation.My computer has 16G RAM.In my simulation,the memory usage is 2G at first,and it will increase to 15G in 24 hours.When the RAM is fully used,Yade crash.

Thanks for any suggestion.

############MWE#####################
from yade import pack#,qt

psdSizes,psdCumm=[0.0005,0.0006,0.003,0.004],[0,0.25,0.25,1.]
num_spheres=15000
young=1e9
compFricDegree = 0 # initial contact friction during the confining phase
finalFricDegree = 26.6 # contact friction during the deviatoric loading
mn,mx=Vector3(0,0,0),Vector3(0.025,0.025,0.025)

O.materials.append(FrictMat(young=young,poisson=0.2,frictionAngle=radians(compFricDegree),density=2600,label='spheres'))
O.materials.append(FrictMat(young=1e11,poisson=0.2,frictionAngle=0,density=0,label='walls'))
walls=aabbWalls([mn,mx],thickness=0,material='walls')
wallIds=O.bodies.append(walls)

sp=pack.SpherePack()
sp.makeCloud(mn,mx,psdSizes=psdSizes,psdCumm=psdCumm,distributeMass=True,num=num_spheres,porosity=0.5,seed=1)
sp.toSimulation(material='spheres')

triax=TriaxialStressController(
 thickness = 0,
 stressMask = 7,
 internalCompaction=False, # If true the confining pressure is generated by growing particles
    label="triax"
)

newton=NewtonIntegrator(damping=0.7,label="newton")

O.engines=[
 ForceResetter(),
 InsertionSortCollider([Bo1_Sphere_Aabb(),Bo1_Box_Aabb()]),
 InteractionLoop(
  [Ig2_Sphere_Sphere_ScGeom(),Ig2_Box_Sphere_ScGeom()],
  [Ip2_FrictMat_FrictMat_FrictPhys()],
  [Law2_ScGeom_FrictPhys_CundallStrack()],label="iloop"
 ),
 FlowEngine(dead=1,label="flow"),#introduced as a dead engine for the moment, see 2nd section
 GlobalStiffnessTimeStepper(active=1,timeStepUpdateInterval=100,timestepSafetyCoefficient=0.8),
 triax,
 newton
]

triax.goal1=triax.goal2=triax.goal3=-20000

while 1:
  O.run(1000, True)
  unb=unbalancedForce()
  if unb<0.001 and abs(-20000-triax.meanStress)/20000<0.001:
    break

triax.dead=1

setContactFriction(radians(finalFricDegree))
newton.damping=0

flow.dead=0
flow.defTolerance=0.3
flow.meshUpdateInterval=200
flow.useSolver=3
flow.permeabilityFactor=1
flow.viscosity=0.001
flow.bndCondIsPressure=[1,1,0,0,0,0]
flow.bndCondValue=[0,38,0,0,0,0]
flow.updateTriangulation=True

Question information

Language:
English Edit question
Status:
Answered
For:
Yade Edit question
Assignee:
No assignee Edit question
Last query:
2019-10-23
Last reply:
2019-11-07

This question was reopened

  • 2019-10-23 by Chu
Robert Caulk (rcaulk) said : #1

Please report which version of yade you are using and how it was installed.

Chu (arcoubuntu) said : #2
Robert Caulk (rcaulk) said : #3

Ok, well I doubt this is the problem but the first step is to compile and rerun with the newest trunk [1]. Generally speaking, compiling off merge request branches is probably not a great idea, even if they were merged eventually.

Do you observe the increase of memory when you set

flow.defTolerance=-1
flow.meshUpdateInterval=-1

?

[1]https://gitlab.com/yade-dev/trunk

Launchpad Janitor (janitor) said : #4

This question was expired because it remained in the 'Needs information' state without activity for the last 15 days.

Chu (arcoubuntu) said : #5

Thanks for Robert.
I compiled newest trunk and rerun my script.It seems work sometimes.Actually,I did not find the condition that memory usage increase,and it still occur sometimes.Fortunately,I can continue my work.

Robert Caulk (rcaulk) said : #6

The janitor cleaned this up but it should remain in the forum.

Robert Caulk (rcaulk) said : #7

An issue has been created regarding this memory leak, track its progress [1].

[1]https://gitlab.com/yade-dev/trunk/issues/116

Chu (arcoubuntu) said : #8

Thanks for Robert.
I may have caused a misunderstanding in #5.Please allow me to re-describe it.
I compiled newest trunk and rerun my script.It seems work sometimes.Actually,I did not know when and why memory usage increase,and it still occur sometimes,but not often.

I set

flow.defTolerance=-1
flow.meshUpdateInterval=-1

and it doesn't seem to work.

Fortunately,I can continue my work.If memory usage doesn't increase during the initial period of the simulation,it will never increase.Similarly,if it increase,it will continuously increase,even if I pause the simulation.

Robert Caulk (rcaulk) said : #9

I think you can help me solve the problem :-) but I need you to carefully answer the following questions:

Just to clarify, you are saying that you observe a memory usage increase when you set:

flow.defTolerance=-1
flow.meshUpdateInterval=-1

??

>If memory usage doesn't increase during the initial period of the simulation,it will never increase.Similarly,if it increase,it will continuously increase,even if I pause the simulation.

Please elaborate on this behavior. What is the "initial period"? Memory continues to increase when you pause the simulation?

Will you please run the following script and tell me what you observe in the mem plot with and without meshUpdateInterval=200? (dont forget to run pip3 install psutil --user, to get the library for tracking memory):

from yade import pack, ymport,export
from yade import timing
import numpy as np
import shutil
import psutil

young=10e7 #250e7
finalFricDegree = 30
density = 2600
rate = 0.01
iterper=400
intRadius=1.0

packingName = '30000.spheres'

sp = O.bodies.append(ymport.textExt(packingName, 'x_y_z_r'))

# get packing dimensions
dim=utils.aabbExtrema()
xinf=dim[0][0]
xsup=dim[1][0]
X=xsup-xinf
yinf=dim[0][1]
ysup=dim[1][1]
Y=ysup-yinf
zinf=dim[0][2]
zsup=dim[1][2]
Z=zsup-zinf

mn,mx=Vector3(xinf, yinf,zinf),Vector3(xsup, ysup, zsup)

O.reset()

O.materials.append(FrictMat(density=density,young=young,poisson=.3,frictionAngle=radians(18.),label='spheres'))
O.materials.append(FrictMat(density=0,young=8.8e10,poisson=.8,frictionAngle=0.,label='walls'))

walls=aabbWalls([mn,mx],thickness=0,material='walls')
wallIds=O.bodies.append(walls)

sp = O.bodies.append(ymport.textExt(packingName, 'x_y_z_r',color=(0,0.2,0.7), material='spheres'))

O.engines=[
 ForceResetter(),
 InsertionSortCollider([Bo1_Sphere_Aabb(aabbEnlargeFactor=intRadius, label='Saabb'),Bo1_Box_Aabb()]),
 InteractionLoop(
  [Ig2_Sphere_Sphere_ScGeom6D(interactionDetectionFactor=intRadius, label='SSgeom'),Ig2_Box_Sphere_ScGeom6D()],
  [Ip2_FrictMat_FrictMat_MindlinPhys(label="hertzIp")],
  [Law2_ScGeom_MindlinPhys_Mindlin(includeAdhesion=True,label='hertzLaw')],label="iloop"
 ),
 FlowEngine(dead=1,label="flow",multithread=False), GlobalStiffnessTimeStepper(active=1,timeStepUpdateInterval=100,timestepSafetyCoefficient=0.8),
 TriaxialStressController(thickness=0,stressMask=7,internalCompaction=False,label='triax'),
 NewtonIntegrator(gravity=(0,0,0), damping=0.4,label='newton')
]
setContactFriction(radians(finalFricDegree))
SSgeom.interactionDetectionFactor=1.
Saabb.aabbEnlargeFactor=1.
flow.dead=0
flow.debug=0
#flow.fluidBulkModulus=2.2e9
flow.defTolerance=-1
flow.meshUpdateInterval=200
flow.useSolver=3
flow.permeabilityFactor=1
flow.viscosity=0.001

O.dynDt=False

newton.damping=0

triax.internalCompaction=False
triax.wall_bottom_activated=False
triax.stressMask=2
triax.goal1=triax.goal3=0
triax.goal2=-11000

flow.bndCondIsPressure=[0,0,0,1,0,0]
flow.bndCondValue=[0,0,0,0,0,0]
flow.boundaryUseMaxMin=[0,0,0,0,0,0]
O.dt=0.1e-3
O.dynDt=False

from yade import plot

def history():
 plot.addData(t=O.time,p=flow.getPorePressure((xsup/2,ysup/2,zsup/2)),
 unbal = unbalancedForce(),
 displacement=triax.depth0-triax.depth,
 P = abs(O.forces.f(4)[2]),
 mem = psutil.Process(os.getpid()).memory_info().rss/1e6 ,
 speed = O.speed
 )

O.engines=O.engines+[PyRunner(dead=0,iterPeriod=iterper,command='history()',label='recorder')]

plot.plots={'t':(('speed')),'t ':(('unbal')),' t':(('mem'))}
plot.plot()

O.run()

Chu (arcoubuntu) said : #10

Hi Robert,
I read the issue and I realized you have done a lot of tests.It's an excellent work.
I notice that you think that the leak only occurs when a meshUpdateInterval>0. This is different from my experience.Unfortunately, I have not recorded it.So you can ignore my conclusion in #8.I'll try again and report the result.

Chu (arcoubuntu) said : #13

Thanks Robert Caulk, that solved my question.

Chu (arcoubuntu) said : #14

Thanks Robert Caulk again.You have done a lot of great work.

Robert Caulk (rcaulk) said : #15

>Thanks Robert Caulk again.You have done a lot of great work.

I appreciate that, Chu. I was meaning to ask if the patch fixed your problem. I suppose it did :-)?

Chu (arcoubuntu) said : #16

Dear Robert,

I'm sorry to tell you that the memory leak may still occur. I run the script in #9, and the result[1].I have some details:
1. After I pause Yade, the memory usage is still increase, at about 500MB/h.
2. The memory leak occur randomly.I run the same script 5 times,and the leak occur 2 times.It's weird, right?
3. When I set flow.defTolerance=-1 and flow.meshUpdateInterval=-1 , the leak still occur.[2] It means the memory leak without re-triangulation?

I use Yade 20191010-2428~bf906f7~bionic1.
Thanks for your any suggestion.

[1]https://s2.ax1x.com/2019/10/22/KGVce1.png
[2]https://s2.ax1x.com/2019/10/23/KJiXdK.png

Launchpad Janitor (janitor) said : #17

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Robert Caulk (rcaulk) said : #18

Changing question status to "answered" so that the launchpad bot does not remove it. However, it appears the question may be still unresolved.

Robert Caulk (rcaulk) said : #19

"
1. After I pause Yade, the memory usage is still increase, at about 500MB/h.
2. The memory leak occur randomly.I run the same script 5 times,and the leak occur 2 times.It's weird, right?
3. When I set flow.defTolerance=-1 and flow.meshUpdateInterval=-1 , the leak still occur.[2] It means the memory leak without re-triangulation?
"

Hey Chu,

Yes, it is weird. Especially seeing memory increase while the simulation is paused, or when the retriangulation is suppressed.

To be certain, are you using the latest compiled trunk version? Also, can you try running some stuff without activating FlowEngine (flow.dead=1)?

Cheers,

Robert

Chu (arcoubuntu) said : #20

Hi Robert,

>To be certain, are you using the latest compiled trunk version?
Yes, I use the newest Yadedaily version, which is same to trunk version.

>Also, can you try running some stuff without activating FlowEngine (flow.dead=1)?
I never see memory increase without activating FlowEngine. However, I think I need more test to tell you about it.

Can you help with this problem?

Provide an answer of your own, or ask Chu for more information if necessary.

To post a message you must log in.