Page 1 of 2
Finaliser/__del__
Posted: Thu Jun 14, 2018 4:31 pm
by BramPeeters
Hi,
Sanity check:I am looking for a way to get some kind of callback when the GC decides to remove an object to clean up some stuff.
After digging in the code I think I have found that i need to enable 'MICROPY_ENABLE_FINALISER' , and then add a method __del__ to the local dictionary of the object type (which is a bit unexpected, i was looking for the inverse of 'make_new' in _mp_obj_type_t).
STATIC const mp_rom_map_elem_t myobject_locals_dict_table[] = {
{ MP_ROM_QSTR(MP_QSTR___del__), MP_ROM_PTR(&mycleanup_obj) },
...
}
Is that correct ? Anything I have missed ? Any pitfalls I should be aware of ?
Thanks,
Bram
Re: Finaliser/__del__
Posted: Thu Jun 14, 2018 4:51 pm
by dhylands
That sounds right. __del__ was used because CPython uses that for the same purpose.
Re: Finaliser/__del__
Posted: Thu Jun 14, 2018 5:54 pm
by deshipu
Note that __del__ is not guaranteed to be called, and when it's being called, it's not guaranteed that all the functions and variables you take for grated will still be available in memory.
If you need to call some code at the end of a scope, it's much better to use a context manager.
Re: Finaliser/__del__
Posted: Fri Jun 15, 2018 6:20 am
by pythoncoder
Quite.
And if a context manager can't be used, there is the option of
Re: Finaliser/__del__
Posted: Fri Jun 15, 2018 8:33 am
by BramPeeters
Thanks for the suggestions, super interesting (I had already encountered the __enter__/__exit__ functions in the lock implementation when adding semaphore support, but was not too familiar with the context manager details)
At the moment the __del__ method still seems to hold the advantage though, because it does not depend on code in python land to be sure that it is called.
The 'with' construct also seems to limit how you can use your object. Eg how do you make an array of 5 of such objects, pass 2 of them to some other objects that hold references to them, do and do some stuff with the 3 others for a while before replacing these 3 by newly created objects (so that these original 3 become targets for GC).
But anyway if __del__ it is not always called then that advantage is not worth much.
What would be situations in which it is not called ?
Re: Finaliser/__del__
Posted: Fri Jun 15, 2018 11:24 am
by deshipu
BramPeeters wrote: ↑Fri Jun 15, 2018 8:33 am
What would be situations in which it is not called ?
It's implementation-dependent. In CPython 2.x it wouldn't get called if the object is a part of a cycle of references, for example — because it's then impossible to guess the correct order of destroying such objects, as the __del__ methods in them might use the referenced objects. Such objects would stay in memory until the program finished, and then be just destroyed together with the whole process, without calling __del__.
I don't know the specifics of MicroPython's internals well enough to list all the cases in there. The Python language specification simply says that you can't rely on __del__ always being called, or on objects being destroyed the moment there are no references to them.
Re: Finaliser/__del__
Posted: Fri Jun 15, 2018 2:10 pm
by Damien
As mentioned, it's best practice to always clean up objects explicitly (or implicit with a context manger) e.g. close files and sockets.
__del__ is always called when an object is collected by the GC (but be sure to use m_new_obj_with_finaliser). But when it is collected is another question. If there is a some data like an integer that looks exactly like a pointer and points to your object, then your object won't be collected even if it's no longer used.
Recently there was some code added which guarantees that all remaining finalisers are called upon shift reset.
Re: Finaliser/__del__
Posted: Fri Jun 15, 2018 5:10 pm
by jickster
Damien wrote: ↑Fri Jun 15, 2018 2:10 pm
As mentioned, it's best practice to always clean up objects explicitly (or implicit with a context manger) e.g. close files and sockets.
__del__ is always called when an object is collected by the GC (but be sure to use m_new_obj_with_finaliser). But when it is collected is another question. If there is a some data like an integer that looks exactly like a pointer and points to your object, then your object won't be collected even if it's no longer used.
Recently there was some code added which guarantees that all remaining finalisers are called upon shift reset.
What's the syntax (and member functions) you have to add to an object to make it compatible with a context manager?
Re: Finaliser/__del__
Posted: Sat Jun 16, 2018 5:49 am
by pythoncoder
This is a generic query about the Python3 language. A quick Google picked up
this article but I'm sure there are plenty of other good tutorials on writing CM classes. It is easy
![Wink ;)](./images/smilies/icon_e_wink.gif)
Re: Finaliser/__del__
Posted: Thu Oct 11, 2018 9:28 am
by BramPeeters
A limitation not yet mentioned here seems to be that with the current state of code, __del__ is not called when the the object is freed not by the garbage collector, but with a gc_free.
( I assume this will change at some point since there is a TODO for it)
And a side question: any reason why m_new_obj does not check the type's dictionary to scan for the presence of a __del__ to automatically determine that a finaliser should be enabled ? Speed ?