Finaliser/__del__

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
BramPeeters
Posts: 54
Joined: Wed Jan 31, 2018 3:10 pm

Finaliser/__del__

Post by BramPeeters » Thu Jun 14, 2018 4:31 pm

Hi,

Sanity check:I am looking for a way to get some kind of callback when the GC decides to remove an object to clean up some stuff.

After digging in the code I think I have found that i need to enable 'MICROPY_ENABLE_FINALISER' , and then add a method __del__ to the local dictionary of the object type (which is a bit unexpected, i was looking for the inverse of 'make_new' in _mp_obj_type_t).

STATIC const mp_rom_map_elem_t myobject_locals_dict_table[] = {
{ MP_ROM_QSTR(MP_QSTR___del__), MP_ROM_PTR(&mycleanup_obj) },
...
}

Is that correct ? Anything I have missed ? Any pitfalls I should be aware of ?

Thanks,
Bram

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Finaliser/__del__

Post by dhylands » Thu Jun 14, 2018 4:51 pm

That sounds right. __del__ was used because CPython uses that for the same purpose.

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Finaliser/__del__

Post by deshipu » Thu Jun 14, 2018 5:54 pm

Note that __del__ is not guaranteed to be called, and when it's being called, it's not guaranteed that all the functions and variables you take for grated will still be available in memory.

If you need to call some code at the end of a scope, it's much better to use a context manager.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Finaliser/__del__

Post by pythoncoder » Fri Jun 15, 2018 6:20 am

Quite.

And if a context manager can't be used, there is the option of

Code: Select all

try:
   # code
finally:
   # Clean up
Peter Hinch
Index to my micropython libraries.

BramPeeters
Posts: 54
Joined: Wed Jan 31, 2018 3:10 pm

Re: Finaliser/__del__

Post by BramPeeters » Fri Jun 15, 2018 8:33 am

Thanks for the suggestions, super interesting (I had already encountered the __enter__/__exit__ functions in the lock implementation when adding semaphore support, but was not too familiar with the context manager details)

At the moment the __del__ method still seems to hold the advantage though, because it does not depend on code in python land to be sure that it is called.
The 'with' construct also seems to limit how you can use your object. Eg how do you make an array of 5 of such objects, pass 2 of them to some other objects that hold references to them, do and do some stuff with the 3 others for a while before replacing these 3 by newly created objects (so that these original 3 become targets for GC).

But anyway if __del__ it is not always called then that advantage is not worth much.
What would be situations in which it is not called ?

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Finaliser/__del__

Post by deshipu » Fri Jun 15, 2018 11:24 am

BramPeeters wrote:
Fri Jun 15, 2018 8:33 am
What would be situations in which it is not called ?
It's implementation-dependent. In CPython 2.x it wouldn't get called if the object is a part of a cycle of references, for example — because it's then impossible to guess the correct order of destroying such objects, as the __del__ methods in them might use the referenced objects. Such objects would stay in memory until the program finished, and then be just destroyed together with the whole process, without calling __del__.

I don't know the specifics of MicroPython's internals well enough to list all the cases in there. The Python language specification simply says that you can't rely on __del__ always being called, or on objects being destroyed the moment there are no references to them.

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: Finaliser/__del__

Post by Damien » Fri Jun 15, 2018 2:10 pm

As mentioned, it's best practice to always clean up objects explicitly (or implicit with a context manger) e.g. close files and sockets.

__del__ is always called when an object is collected by the GC (but be sure to use m_new_obj_with_finaliser). But when it is collected is another question. If there is a some data like an integer that looks exactly like a pointer and points to your object, then your object won't be collected even if it's no longer used.

Recently there was some code added which guarantees that all remaining finalisers are called upon shift reset.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: Finaliser/__del__

Post by jickster » Fri Jun 15, 2018 5:10 pm

Damien wrote:
Fri Jun 15, 2018 2:10 pm
As mentioned, it's best practice to always clean up objects explicitly (or implicit with a context manger) e.g. close files and sockets.

__del__ is always called when an object is collected by the GC (but be sure to use m_new_obj_with_finaliser). But when it is collected is another question. If there is a some data like an integer that looks exactly like a pointer and points to your object, then your object won't be collected even if it's no longer used.

Recently there was some code added which guarantees that all remaining finalisers are called upon shift reset.
What's the syntax (and member functions) you have to add to an object to make it compatible with a context manager?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Finaliser/__del__

Post by pythoncoder » Sat Jun 16, 2018 5:49 am

This is a generic query about the Python3 language. A quick Google picked up this article but I'm sure there are plenty of other good tutorials on writing CM classes. It is easy ;)
Peter Hinch
Index to my micropython libraries.

BramPeeters
Posts: 54
Joined: Wed Jan 31, 2018 3:10 pm

Re: Finaliser/__del__

Post by BramPeeters » Thu Oct 11, 2018 9:28 am

A limitation not yet mentioned here seems to be that with the current state of code, __del__ is not called when the the object is freed not by the garbage collector, but with a gc_free.
( I assume this will change at some point since there is a TODO for it)

And a side question: any reason why m_new_obj does not check the type's dictionary to scan for the presence of a __del__ to automatically determine that a finaliser should be enabled ? Speed ?

Post Reply