finaliser proxy

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

finaliser proxy

Post by ttmetro » Fri Dec 25, 2020 12:29 am

Edit: See warning below. I'm no longer sure this is useful for anything.

Micropython does not call finalisers for user defined classes. Although alternatives are usually preferable, there are situations were finalisers are useful.

To enable finalisers with user define classes, I created a class FinaliserProxy in C. User defined classes derive from this class as follows:

Code: Select all

def cb(self):
    print("finaliser for", self.desc, "called")

class FP(FinaliserProxy):
    def __init__(self, cb, desc):
        self.desc = desc
        super().__init__(cb, self)

f = FP(cb, "my custom class with finalizer")

# ... use f, then
# del f or f = None 
When f is eventually collected, cb_function is called to do whatever cleanup is required (I have an application that cleans up a related object on a remote server).

Implementation detail: the constructor of FinaliserProxy takes self of the derived class as an argument. The reason is that when the C finaliser is called it always receives FinaliserProxy, not the derived class as self. Perhaps there is a better way to do this?

I welcome feedback about this solution including potential pitfalls (I am aware that the object is never be collected if the gc does not run).
Last edited by ttmetro on Thu Dec 31, 2020 3:26 am, edited 1 time in total.
Bernhard Boser

stijn
Posts: 573
Joined: Thu Apr 24, 2014 9:13 am

Re: finaliser proxy

Post by stijn » Fri Dec 25, 2020 8:18 am

One problem with this solution is that it is non-standard both in principle ('finaliser in python' makes people thing about __del__) and code (won't run on CPython unless you also provide a similar class there). You could part f this by implementing __del__ in C and then you probably don't need to pass self anymore, not 100% sure, but I've done somethings with deriving from classes defined in C and never had an issue with that. But then it's still not portable and quite verbose. All in all if I'd need this I'd probably run a custom MicroPython which does this: https://github.com/micropython/micropyt ... -530333249 pretty simple, just works (as far as I'm aware, not tested thoroughly) and portable. Given the change is so small, I'm fairly positive that if you wrap that in an #if MICROPY_PY_CLASS_FINALISER or so and create a PR for it it can be merged into master.

Online
User avatar
pythoncoder
Posts: 4938
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Putting this into context with CPython

Post by pythoncoder » Fri Dec 25, 2020 9:10 am

The issue of the del operator is one which crops up periodically. My understanding of CPython is that the delete special method is not guaranteed to be called at all promptly and the only guarantee is that it will be called by the time Python exits to the OS. Given that firmware-type applications normally run forever, this means no guarantee at all - so coders of such applications never use it. All AIUI, of course.

This solution presumably carries a guarantee of immediate execution, making it usable in a firmware environment, which is great :) But it is an enhancement to CPython which someone writing CPython-portable code would be unlikely to use.
Peter Hinch

stijn
Posts: 573
Joined: Thu Apr 24, 2014 9:13 am

Re: finaliser proxy

Post by stijn » Fri Dec 25, 2020 3:57 pm

This solution presumably carries a guarantee of immediate execution
I don't think it does; I mean I don't know the code behind it but since the OP calls it a finaliser I assume it does what finalisers do i.e. get called when the object is garbage collected.

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: finaliser proxy

Post by ttmetro » Fri Dec 25, 2020 6:46 pm

This solution presumably carries a guarantee of immediate execution
No guarantees, as CPython gives no guarantee.

Only IF the gc collects the object, the proxy function is called.

I have an application where I "join" two Python interpreters. Objects on the server are proxies in the client. When the client cleans up an object, the server does not know and the "actual" gets never recycled, resulting in a memory leak. For such cases, this solution works, and since it's localized it's not to hard to write code that works also in CPython, if that's a requirement.

No presumption that it solves everyones problem ...
Bernhard Boser

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: finaliser proxy

Post by ttmetro » Fri Dec 25, 2020 7:59 pm

Link to the code: https://github.com/iot49/micropython/bl ... serproxy.c

Usage pattern:

Code: Select all

try:
    from finaliser_proxy import FinaliserProxy
except ImportError:
    # CPython compatibility
    class FinaliserProxy:
        def __init__(self, cleanup): pass

class FP(FinaliserProxy):
    def __init__(self, desc):
        self.desc = desc
        super().__init__(self.__del__)
        
    def __del__(self):
        print("cleanup:", self.desc)

f = FP("my custom class with finalizer")

# ... use f, then
del f

# __del__ will be called if and when the gc collects the object
Bernhard Boser

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: finaliser proxy

Post by ttmetro » Thu Dec 31, 2020 3:25 am

Warning:

No memory allocation (heap) in the finaliser. I don't know the VM enough to vouch there are no other issues.
MemoryError spells trouble (obviously).

Although I managed to get around that problem in my application, I ran into other issues.
As already pointed out by others, finalisers have very limited use, I advise to look for other solutions.
Bernhard Boser

Post Reply