Are anonymous objects at risk of GC? SOLVED

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
User avatar
pythoncoder
Posts: 3538
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Are anonymous objects at risk of GC? SOLVED

Post by pythoncoder » Tue Jul 24, 2018 12:46 pm

I can't get my head round this one. Perhaps someone can help with my headache ;)

Code: Select all

class MyClass():
    pass

def foo():
   a = MyClass()
When foo() terminates a goes out of scope and may be GC'd. So far, so good.

But what if there is no specific reference to the MyClass instance:

Code: Select all

class MyClass():
    pass

def foo():
   MyClass()
   while True:
       # Run code
       # Might the instance be GC'd here?
To further illustrate how this arises I use this closure to create a singleton decorator:

Code: Select all

def singleton(cls):
    instances = {}
    def getinstance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]
    return getinstance
And I use it as follows:

Code: Select all

@singleton
class MyClass():
    def method(self):
        pass

def foo():
   MyClass()  # Instantiate
   while True:
       MyClass().method()  # Get an instance reference, use it and lose it
       # Might the instance be GC'd here?
Last edited by pythoncoder on Thu Jul 26, 2018 4:52 am, edited 1 time in total.
Peter Hinch

stijn
Posts: 348
Joined: Thu Apr 24, 2014 9:13 am

Re: Are anonymous objects at risk of GC?

Post by stijn » Tue Jul 24, 2018 2:22 pm

In the second example you show the instance can definitely get GC'ed since no references to it exist.

But with the singleton example that is, I think, not the case: getinstance closes over instances and instances stores an instance of MyClass. Because of decoration the name 'MyClass' now refers to that closure object instead of directly to the class implementation. So there's a closure object with a reference to an instance stored in the locals/globals dict (both are the same there), so that instance cannot be GC'd.

User avatar
pythoncoder
Posts: 3538
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Are anonymous objects at risk of GC?

Post by pythoncoder » Tue Jul 24, 2018 3:36 pm

Thank you. I think my aged brain is getting to grips with this ;)

It evidently comes down to my not fully appreciating how closures work. The singleton function has global scope, so I guess the instances variable also has global scope. Otherwise you'd be unable to access a singleton object after it had first been instantiated. Are nonlocal variables like a bound variables in an object: their lifetime being that of the closure/object instance?

After posting it struck me that in my application the singleton class has an asynchronous bound method launched by the constructor. Consequently there will always be a reference to the bound method in the uasyncio run queue; the instance must persist. But I am interested in the general case.
Peter Hinch

Christian Walther
Posts: 55
Joined: Fri Aug 19, 2016 11:55 am

Re: Are anonymous objects at risk of GC?

Post by Christian Walther » Tue Jul 24, 2018 6:36 pm

The instances variable does not have global scope, it is a local variable of function singleton and is initialized to a new dict every time that function is called. Which means that when you apply your @singleton decorator to two classes, you will not get a dict with two entries, as I suspect you think, but two dicts with one entry each. You can easily verify that by inserting

Code: Select all

print(instances, id(instances))
into the getinstance function and then instantiating each class a couple of times.

This also means that you don’t need a dict at all, you can also use a single-element list or any other thing that you can assign to without creating a new local variable:

Code: Select all

def singleton(cls):
    instance = [None]
    def getinstance(*args, **kwargs):
        if not instance[0]:
            instance[0] = cls(*args, **kwargs)
        return instance[0]
    return getinstance
(That is, at least, my experimentally verified understanding. I don’t claim to be an expert on this, but your question intrigued me enough to try it out.)

User avatar
pythoncoder
Posts: 3538
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Are anonymous objects at risk of GC?

Post by pythoncoder » Wed Jul 25, 2018 4:23 am

Thanks for that. My fault for cut'n'pasting from stackoverflow ;) Evidently the author didn't understand his own code either.

Your suggestion can be simplified further as there is little point in a one element list:

Code: Select all

def singleton(cls):
    instance = None
    def getinstance(*args, **kwargs):
        nonlocal instance
        if instance is None:
            instance = cls(*args, **kwargs)
        print(instance, id(instance))
        return instance
    return getinstance

@singleton
class MyClass1():
    def method(self):
        pass

@singleton
class MyClass2():
    def method(self):
        pass

mc1 = MyClass1()
mc2 = MyClass2()

mc11 = MyClass1()
mc22 = MyClass2()

print(mc11 is mc1)
print(mc22 is mc2)
Produces

Code: Select all

<MyClass1 object at 7faed9d4d9a0> 140388955642272
<MyClass2 object at 7faed9d4d9e0> 140388955642336
<MyClass1 object at 7faed9d4d9a0> 140388955642272
<MyClass2 object at 7faed9d4d9e0> 140388955642336
True
True
Interesting!
Peter Hinch

Christian Walther
Posts: 55
Joined: Fri Aug 19, 2016 11:55 am

Re: Are anonymous objects at risk of GC?

Post by Christian Walther » Wed Jul 25, 2018 4:56 am

Heh, my Python 2 background is showing through – I wasn’t aware of the nonlocal statement, thanks!

User avatar
pythoncoder
Posts: 3538
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Singletons

Post by pythoncoder » Wed Jul 25, 2018 5:56 am

Reading up on singletons I traced the original code back to a PEP318 example, claimed to be from Shane Hathaway on python-dev. So even Python gurus write code they don't understand :lol: Well spotted by you.

For reasons I only half understand singletons seem to be a cause of religious flamewars. In a hardware context it seems to me they are everywhere: a Pyboard has just one RTC, one array of battery backed RAM and so on. Singletons seem a natural way to express this. They avoid explicit globals or having to pass an instance around the place.

One option, used by Paul in uasyncio, is to instantiate the object in the module: users access the instance (e.g. the sleep_ms object). This is fine if you know that all users of the module will need the instance. But if instantiation is optional it wastes resources.

Evidently MicroPython uses singletons at the C level:

Code: Select all

>>> import machine
>>> a = machine.RTC()
>>> b = machine.RTC()
>>> a is b
True
>>> 
Peter Hinch

stijn
Posts: 348
Joined: Thu Apr 24, 2014 9:13 am

Re: Singletons

Post by stijn » Wed Jul 25, 2018 8:47 am

pythoncoder wrote:
Wed Jul 25, 2018 5:56 am
For reasons I only half understand singletons seem to be a cause of religious flamewars. In a hardware context it seems to me they are everywhere
Until they are not, which is why the flamewars exist. Sure there's one RTC but if your application suddenly runs on some shiny new type of pyboard which happens to have an alternative clock source and you want to use it instead you're up for some refactoring. Python makes this easy since you could just point the RTC function to something else but it's not always that simple, plus it doesn't help if you want to use both RTC and the other clock source. Admittedly the clock example may be a bit far fetched but here's a better one: I once had to work on a graphical application which assumed there could be only 1 monitor. Sounded reasonable in the early nineties but later on it became a source of pain :]

Like with pretty much all concepts out there singletons have their place but overdoing it potentially leads to suboptimal code. Which doesn't matter for small local applications but quickly becomes a burden in long-lived and/or large applications especially if they run on multiple platforms.

Anyway, on topic: forgot to mention that figuring out things like this is relatively easy if you know the basics of using a debugger; build the unix port with verbose output for runtime.c (and possible other files if you want), put a breakpoint on e.g. mp_call_function_0(module_fun) in lexer.c or deeper and step through the code. The output window shos verbose output while at the same time you can see those addresses in the debugger, etc.

User avatar
pythoncoder
Posts: 3538
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Thanks

Post by pythoncoder » Thu Jul 26, 2018 4:58 am

Thanks for your help @stijn @Christian Walther. In retrospect it's obvious that there must be a reference to the instance somewhere otherwise the singleton decorator couldn't possibly work. I just couldn't figure out where it lurked ;)

Apologies for raising what turned out to be a generic Python question.
Peter Hinch

Post Reply