Question about memory allocation in classes dict

Asked by John Joseph

Hi Marcus,

Thanks for this library, it's great. I just had a question about some memory allocation you do in the two global vectors defined in EcsPython.cpp.

I see that you dynamically allocate instances of your two structs, but I can't see where you free them. Does the vector destructor do this automatically?

Again, thanks for the library.

- John

Question information

Language:
English Edit question
Status:
Solved
For:
ECS:Python Edit question
Assignee:
No assignee Edit question
Solved by:
John Joseph
Solved:
Last query:
Last reply:
Revision history for this message
Marcus Tomlinson (marcustomlinson) said :
#1

Hi John, sorry I missed this question, turns out Launchpad does mail the project maintainer about these :/

So with respect to EcsPythonClassesDict, that vector grows but does not shrink again. This I see as quite acceptable as they are class types, and typically types are expected to be persistent throughout the lifetime of an application.

What should not be persistent is objects, i.e. the EcsExposedObjects vector. You are absolutely right, there is no way currently to cleanup that vector. You can delete the actual objects they point to, but you'll still be leaking a structure containing a char pointer and 2 std::strings. Not so good.

Ok, so I propose adding a new method Ecs_Hide_Object() (the inverse of Ecs_Expose_Object()), that will remove and cleanup the entry in EcsExposedObjects.

Would that service your concerns?

Revision history for this message
John Joseph (mynameisjohnj) said :
#2

Hey Marcus, thanks for getting back to me.

You're actually bringing up a different concern I had. Before I get to that...

It's fair to say that the lifetime of the dict objects is that of the program, but what I'm referring to is what happens when the program exits. I googled it and found that most operating systems clean up any memory allocated by a program when it exits, but that it's not a part of the C++ standard. I guess what I'm asking is why those vectors have to be pointer based in the first place; why can't you just push back objects instead of allocating them on the heap and pushing back the pointer? Otherwise I guess you'd want to free them in the Finalize call.

Going back to the notion of "Hiding" exposed objects, I've been toying with the idea of a virtual Py_Exposed class whose destructor takes care of this (by freeing memory and calling "del objName" when destroyed.) Unfortunately it's a bit more complicated than that, and there could be multiple python variables that have a _self member = to the C++ pointer. However explicitly doing it like you say when you're freeing the object you allocated is a perfectly good solution, so I'll mark this solved.

Full disclosure; I've sort of cannibalized your code as well as the "pywrapper" code discussed here
http://average-coder.blogspot.com/2012/04/python-wrapper-in-c11.html

into one (very c++11 heavy) library. I'm glad I was able to get in touch with you, since I'm sure I need your permission to do stuff like this.... is "fork" the formal term? I have no idea.

Anyway, I'm hosting the project at
https://github.com/mynameisjohn/PyLiaison

I started it very recently so the code is quite a mess, and I've been working mostly out of MSVC and Xcode seems to be complaining (I'll have to try gcc ASAP.) Let me know if I'm violating any sort of terms I've probably agreed to and I'll take it down.

Revision history for this message
Marcus Tomlinson (marcustomlinson) said :
#3

Oh yeah, I am totally assuming the operating system will take care of freeing that memory. If your operating system doesn't manage its processes' memory this way, I'd suggest running away from it :P But seriously, this is a very fair assumption to make. Applications can not be trusted to do the right thing, besides, an application may be really careful about its memory, then crash mid execution for some unforeseen circumstance on an untested architecture, leaving everything behind anyway. That said, your way of looking at it is of course more correct than mine! :)

Its probably worth mentioning that I intentionally limited the implementation to C++98, for better compatibility across platforms, compilers etc. A few of my projects were written this way in and from 2012, particularly due to Microsoft's lag in keeping up with C++ standards.

With respect to your fork, I have absolutely no problem with that! Honestly, I'm just glad my code is useful, thats why I wrote it :)

Revision history for this message
John Joseph (mynameisjohnj) said :
#4

It's been very useful! And I can imagine some issues with MSVC and the C++ standard, so making the code backwards compatible was smart. Hopefully things will be smooth going forward and all this C++11 witchcraft won't come back to haunt me, but we'll see....

Anyway, thanks for the blessing. The wrapper is going well so far; the project I wrote it for may end up using PythonQt if they pick Qt for the UI, but working with EcsPython taught me more than those Python Extension/Embedding tutorials did (though they were still useful.)

All the best!

- John