continuous creation and destruction of bodies

Hello,

Did you try using O.bodies.erase() (I guess you identified this function) and see what happens, measuring if necessary memory consumption ?

If you're a specialist of how pointer types affect memory consumption, you may also give a look at the source code of this function in [1,2].

Jerome

(In short and as far as I'm concerned, I would not expect anything "bad" as such an issue has already been discussed here / taken into account in the code, if I'm not mistaken)

[1] https://github.com/yade/trunk/blob/master/core/BodyContainer.hpp
[2] https://github.com/yade/trunk/blob/master/core/BodyContainer.cpp

Yade uses smart pointers (shared_ptr in this case)[1], which automatically manage memory deallocation when the last shared_ptr pointing to an object is deleted or shared_ptr::reset is called. In the case of bodies, it looks like the "erase" function is calling shared_ptr::reset, so I believe the freed memory is available for newly created bodies. You should be able to run your conveyor belt simulation indefinitely.

[1]https://yade-dem.org/doc/prog.html#shared-pointers

Thanks to both Duriez and Caulk, your answers seem very useful. I study them and I will let you know if everything is ok (as it seems from your discussion).

Andrea

Hi guys,
just a theoretical comment concerning O.bodies.erase,. The memory of erased body is freed, but the space in BodyContainer remains (O.bodies.append does not reuse ids) and the size of BodyContainer itself is continuously increasing. So with continuous adding and deleting bodies, the simulation theoretically cannot be run indefinitely, but practically yes :-)
cheers
Jan

###
sphs = [sphere((2*i,0,0),1) for i in (0,1,2,3,4)]
s0,s1,s2,s3,s4 = sphs

O.bodies.append(s0)
O.bodies.append(s1)
O.bodies.append(s2)

print O.bodies[0], O.bodies[0].id
print O.bodies[1], O.bodies[1].id
print O.bodies[2], O.bodies[2].id
print len(O.bodies), len([b for b in O.bodies])

O.bodies.erase(0)
print len(O.bodies), len([b for b in O.bodies])
O.bodies.erase(1)
print len(O.bodies), len([b for b in O.bodies])
O.bodies.erase(2)
print len(O.bodies), len([b for b in O.bodies])

O.bodies.append(s3)
O.bodies.append(s4)
print len(O.bodies), len([b for b in O.bodies])
print O.bodies[0]
print O.bodies[4], O.bodies[4].id
###

Dear Jan
thanks also for your message. Your conclusion is puzzling, i.e. it seems to contradict some statements found in the .cpp and .hpp linked by Jerome above.

Indeed in [2] at line 60 I read
    "b->id=-1;//else it sits in the python scope without a chance to be inserted again"

And in [1] at line 22 I read

 "Container of bodies implemented as flat std::vector. It handles body removal and
 intelligently reallocates free ids for newly added ones.
 The nested iterators and the specialized FOREACH_BODY macros above will silently skip null body pointers which may exist after
 removal. The null pointers can still be accessed via the [] operator. "

One explanation could be that your example is too small, i.e. the engine of shared_ptr allocation does not reuse old (freed) memory until it is really necessary, that is after many new insertions. As soon as I have time I will try to conceive a modification of Jan's example with a much larger number of bodies deleted and inserted.

Any other ideas?

Thanks a lot to everybody for the prompt help

Andrea

[1] https://github.com/yade/trunk/blob/master/core/BodyContainer.hpp
[2] https://github.com/yade/trunk/blob/master/core/BodyContainer.cpp

Hello,

> "b->id=-1;//else it sits in the python scope without a chance to be inserted again"

this is just that you can re-append erased body if you have it in Python
##
s = sphere((0,0,0),1)
O.bodies.append(s)
#O.bodies.append(s) # would be error
print s.id
O.bodies.erase(0)
print s.id
O.bodies.append(s)
print s.id
##

> and intelligently reallocates free ids for newly added ones
> One explanation could be that your example is too small, i.e. the engine of shared_ptr allocation does not reuse old (freed) memory until it is really necessary, that is after many new insertions

according to the source code, the sentence of "intelligent reallocation" is not true (perhaps was some time ago), no matter if the example is too small or not..

However, my comment was really theoretical, practically it has no effect. I have tried the following code, 10e6 append/erase has no significant effect on memory. Assuming sizeof(shared_ptr)=16, it occupies 10e6*16=160e6=160MB, almost nothing)
##
while True:
   i = O.bodies.append(sphere((0,0,0),1))
   O.bodies.erase(i)
   if i%1000000==0:
      print i, len(O.bodies)
##

so, go in the direction append/erase without any worries :-)

cheers
Jan

Hi,
I would suggest to check examples/spheresFactory.py which combines the engines DomainLimiter and BoxFactory to delete/insert particles. It will be more flexible and computationally much more efficient than anything you can wruite in python.
Bruno