Memory usage is too high

Please report which version of yade you are using and how it was installed.

Hi Robert,
I use source code from https://gitlab.com/yade-dev/trunk/merge_requests/162

Ok, well I doubt this is the problem but the first step is to compile and rerun with the newest trunk [1]. Generally speaking, compiling off merge request branches is probably not a great idea, even if they were merged eventually.

Do you observe the increase of memory when you set

flow.defTolerance=-1
flow.meshUpdateInterval=-1

?

[1]https://gitlab.com/yade-dev/trunk

This question was expired because it remained in the 'Needs information' state without activity for the last 15 days.

Thanks for Robert.
I compiled newest trunk and rerun my script.It seems work sometimes.Actually,I did not find the condition that memory usage increase,and it still occur sometimes.Fortunately,I can continue my work.

The janitor cleaned this up but it should remain in the forum.

An issue has been created regarding this memory leak, track its progress [1].

[1]https://gitlab.com/yade-dev/trunk/issues/116

Thanks for Robert.
I may have caused a misunderstanding in #5.Please allow me to re-describe it.
I compiled newest trunk and rerun my script.It seems work sometimes.Actually,I did not know when and why memory usage increase,and it still occur sometimes,but not often.

I set

flow.defTolerance=-1
flow.meshUpdateInterval=-1

and it doesn't seem to work.

Fortunately,I can continue my work.If memory usage doesn't increase during the initial period of the simulation,it will never increase.Similarly,if it increase,it will continuously increase,even if I pause the simulation.

I think you can help me solve the problem :-) but I need you to carefully answer the following questions:

Just to clarify, you are saying that you observe a memory usage increase when you set:

flow.defTolerance=-1
flow.meshUpdateInterval=-1

??

>If memory usage doesn't increase during the initial period of the simulation,it will never increase.Similarly,if it increase,it will continuously increase,even if I pause the simulation.

Please elaborate on this behavior. What is the "initial period"? Memory continues to increase when you pause the simulation?

Will you please run the following script and tell me what you observe in the mem plot with and without meshUpdateInterval=200? (dont forget to run pip3 install psutil --user, to get the library for tracking memory):

from yade import pack, ymport,export
from yade import timing
import numpy as np
import shutil
import psutil

young=10e7 #250e7
finalFricDegree = 30
density = 2600
rate = 0.01
iterper=400
intRadius=1.0

packingName = '30000.spheres'

sp = O.bodies.append(ymport.textExt(packingName, 'x_y_z_r'))

# get packing dimensions
dim=utils.aabbExtrema()
xinf=dim[0][0]
xsup=dim[1][0]
X=xsup-xinf
yinf=dim[0][1]
ysup=dim[1][1]
Y=ysup-yinf
zinf=dim[0][2]
zsup=dim[1][2]
Z=zsup-zinf

mn,mx=Vector3(xinf, yinf,zinf),Vector3(xsup, ysup, zsup)

O.reset()

O.materials.append(FrictMat(density=density,young=young,poisson=.3,frictionAngle=radians(18.),label='spheres'))
O.materials.append(FrictMat(density=0,young=8.8e10,poisson=.8,frictionAngle=0.,label='walls'))

walls=aabbWalls([mn,mx],thickness=0,material='walls')
wallIds=O.bodies.append(walls)

sp = O.bodies.append(ymport.textExt(packingName, 'x_y_z_r',color=(0,0.2,0.7), material='spheres'))

O.engines=[
 ForceResetter(),
 InsertionSortCollider([Bo1_Sphere_Aabb(aabbEnlargeFactor=intRadius, label='Saabb'),Bo1_Box_Aabb()]),
 InteractionLoop(
  [Ig2_Sphere_Sphere_ScGeom6D(interactionDetectionFactor=intRadius, label='SSgeom'),Ig2_Box_Sphere_ScGeom6D()],
  [Ip2_FrictMat_FrictMat_MindlinPhys(label="hertzIp")],
  [Law2_ScGeom_MindlinPhys_Mindlin(includeAdhesion=True,label='hertzLaw')],label="iloop"
 ),
 FlowEngine(dead=1,label="flow",multithread=False), GlobalStiffnessTimeStepper(active=1,timeStepUpdateInterval=100,timestepSafetyCoefficient=0.8),
 TriaxialStressController(thickness=0,stressMask=7,internalCompaction=False,label='triax'),
 NewtonIntegrator(gravity=(0,0,0), damping=0.4,label='newton')
]
setContactFriction(radians(finalFricDegree))
SSgeom.interactionDetectionFactor=1.
Saabb.aabbEnlargeFactor=1.
flow.dead=0
flow.debug=0
#flow.fluidBulkModulus=2.2e9
flow.defTolerance=-1
flow.meshUpdateInterval=200
flow.useSolver=3
flow.permeabilityFactor=1
flow.viscosity=0.001

O.dynDt=False

newton.damping=0

triax.internalCompaction=False
triax.wall_bottom_activated=False
triax.stressMask=2
triax.goal1=triax.goal3=0
triax.goal2=-11000

flow.bndCondIsPressure=[0,0,0,1,0,0]
flow.bndCondValue=[0,0,0,0,0,0]
flow.boundaryUseMaxMin=[0,0,0,0,0,0]
O.dt=0.1e-3
O.dynDt=False

from yade import plot

def history():
 plot.addData(t=O.time,p=flow.getPorePressure((xsup/2,ysup/2,zsup/2)),
 unbal = unbalancedForce(),
 displacement=triax.depth0-triax.depth,
 P = abs(O.forces.f(4)[2]),
 mem = psutil.Process(os.getpid()).memory_info().rss/1e6 ,
 speed = O.speed
 )

O.engines=O.engines+[PyRunner(dead=0,iterPeriod=iterper,command='history()',label='recorder')]

plot.plots={'t':(('speed')),'t ':(('unbal')),' t':(('mem'))}
plot.plot()

O.run()

Hi Robert,
I read the issue and I realized you have done a lot of tests.It's an excellent work.
I notice that you think that the leak only occurs when a meshUpdateInterval>0. This is different from my experience.Unfortunately, I have not recorded it.So you can ignore my conclusion in #8.I'll try again and report the result.

Thanks Robert Caulk, that solved my question.

Thanks Robert Caulk again.You have done a lot of great work.

>Thanks Robert Caulk again.You have done a lot of great work.

I appreciate that, Chu. I was meaning to ask if the patch fixed your problem. I suppose it did :-)?

Dear Robert,

I'm sorry to tell you that the memory leak may still occur. I run the script in #9, and the result[1].I have some details:
1. After I pause Yade, the memory usage is still increase, at about 500MB/h.
2. The memory leak occur randomly.I run the same script 5 times,and the leak occur 2 times.It's weird, right?
3. When I set flow.defTolerance=-1 and flow.meshUpdateInterval=-1 , the leak still occur.[2] It means the memory leak without re-triangulation?

I use Yade 20191010-2428~bf906f7~bionic1.
Thanks for your any suggestion.

[1]https://s2.ax1x.com/2019/10/22/KGVce1.png
[2]https://s2.ax1x.com/2019/10/23/KJiXdK.png

This question was expired because it remained in the 'Open' state without activity for the last 15 days.

Changing question status to "answered" so that the launchpad bot does not remove it. However, it appears the question may be still unresolved.

"
1. After I pause Yade, the memory usage is still increase, at about 500MB/h.
2. The memory leak occur randomly.I run the same script 5 times,and the leak occur 2 times.It's weird, right?
3. When I set flow.defTolerance=-1 and flow.meshUpdateInterval=-1 , the leak still occur.[2] It means the memory leak without re-triangulation?
"

Hey Chu,

Yes, it is weird. Especially seeing memory increase while the simulation is paused, or when the retriangulation is suppressed.

To be certain, are you using the latest compiled trunk version? Also, can you try running some stuff without activating FlowEngine (flow.dead=1)?

Cheers,

Robert

Hi Robert,

>To be certain, are you using the latest compiled trunk version?
Yes, I use the newest Yadedaily version, which is same to trunk version.

>Also, can you try running some stuff without activating FlowEngine (flow.dead=1)?
I never see memory increase without activating FlowEngine. However, I think I need more test to tell you about it.