How to free memory after unused imports?

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
VladVons
Posts: 19
Joined: Sun Feb 12, 2017 6:49 pm
Location: Ukraine

How to free memory after unused imports?

Post by VladVons » Thu Jun 14, 2018 5:51 am

My pure free memory is 30720
Than i import App.py (free memory 17216)
Than i try to free App (free memory still 18688)

Code: Select all

>>> os.uname()
MicroPython v1.9.4-8-ga9a3caad0 on 2018-05-11; ESP module with ESP8266

>>> gc.collect()
>>> gc.mem_free()
30720

>>> globals()
{
'bdev': <FlashBdev object at 3ffef610>,.
'gc': <module 'gc'>,.
'__name__': '__main__',.
'micropython': <module 'micropython'>,.
'uos': <module 'uos'>,.
'sys': <module 'sys'>,.
'os': <module 'uos'>,.
'machine': <module 'umachine'>
}

>>> import App
>>> gc.mem_free()
17216

>>> del App
>>> gc.collect()
>>> gc.mem_free()
18688

>>> globals()
{
'bdev': <FlashBdev object at 3ffef610>,.
'gc': <module 'gc'>,.
'__name__': '__main__',.
'micropython': <module 'micropython'>,.
'uos': <module 'uos'>,.
'sys': <module 'sys'>,.
'os': <module 'uos'>,.
'machine': <module 'umachine'>
}

User avatar
pythoncoder
Posts: 2967
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: How to free memory after unused imports?

Post by pythoncoder » Thu Jun 14, 2018 7:31 am

This is actually a generic question about the Python language rather than a specific MicroPython issue. See this on stackoverflow - I have no idea whether the suggested solutions (hacks?) will reclaim RAM.

tl;dr It's problematic. If possible do a soft reset.
Peter Hinch

User avatar
deshipu
Posts: 1329
Joined: Thu May 28, 2015 5:54 pm

Re: How to free memory after unused imports?

Post by deshipu » Thu Jun 14, 2018 11:12 am

It's still kept in sys.modules, you have to delete it from there.

User avatar
pythoncoder
Posts: 2967
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: How to free memory after unused imports?

Post by pythoncoder » Thu Jun 14, 2018 12:32 pm

Did you read the stackoverflow reference? I am aware of the clever reload hack, but deleting from sys.modules doesn't actually unload the code. It's a neat trick to make __import__() work.

Unless I'm very mistaken there is no way to do this which will work in all cases (in CPython or MicroPython).
Peter Hinch

User avatar
deshipu
Posts: 1329
Joined: Thu May 28, 2015 5:54 pm

Re: How to free memory after unused imports?

Post by deshipu » Thu Jun 14, 2018 5:36 pm

It's an object as anything else, and it gets released as soon as there are no more references to it. When you only import a module, and don't do anything else with it, like in this example, there are only two places that have references to it (assuming the module itself doesn't do any monkeypatching): the global namespace where you imported it, and sys.modules. Deleting the references from those two places is enough to have the module garbage collected and the memory reclaimed.

Of course, if you actually use anything from the module, create any objects or inherit from its classes, etc. — then you would also need to delete all those references.

User avatar
pythoncoder
Posts: 2967
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: How to free memory after unused imports?

Post by pythoncoder » Fri Jun 15, 2018 4:57 am

In my testing this simply isn't the case. Module lbt is large and complex, pulling in uasyncio and an entire GUI:

Code: Select all

>>> import micropython, sys, gc
>>> gc.collect()
>>> micropython.mem_info()
stack: 468 out of 15360
GC: total: 102400, used: 9456, free: 92944
 No. of 1-blocks: 31, 2-blocks: 12, max blk sz: 441, max free sz: 5779
>>> import lbt
Testing TFT...
>>> sys.modules
{'asyn': <module 'asyn' from 'asyn.py'>, 'constants': <module 'constants' from 'constants.py'>, 'asyncio_priority': <module 'asyncio_priority' from 'asyncio_priority.py'>, 'lbt': <module 'lbt' from 'lbt.py'>, 'lcd160cr': <module 'lcd160cr'>, 'lcd160_gui': <module 'lcd160_gui'>, 'lcd_local': <module 'lcd_local' from 'lcd_local.py'>, 'font10': <module 'font10'>, 'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'aswitch': <module 'aswitch' from 'aswitch.py'>, 'upysh': <module 'upysh'>}
>>> del sys.modules['lbt']
>>> del lbt
>>> gc.collect()
>>> micropython.mem_info()
stack: 468 out of 15360
GC: total: 102400, used: 75808, free: 26592
 No. of 1-blocks: 955, 2-blocks: 249, max blk sz: 441, max free sz: 463
>>> 
Peter Hinch

Drako
Posts: 12
Joined: Thu Oct 19, 2017 9:28 am

Re: How to free memory after unused imports?

Post by Drako » Fri Jun 15, 2018 6:39 am

I have had the same experience as Peter. I cleared the sys.modules and delteted everything from the globals and still couldn't free up the whole memory.

Code: Select all

from gc import collect  # @UnresolvedImport
from sys import modules, path

path.clear()
path.insert(0, '')
path.insert(0, '/sd')

modules.clear()

collect()

for x in dir():
    exec('del {}'.format(x))


try:
    from gc import collect, threshold  # @UnresolvedImport @Reimport
    from micropython import alloc_emergency_exception_buf  # @UnresolvedImport
    from sys import print_exception # @UnresolvedImport

    alloc_emergency_exception_buf(100)

    collect()
    threshold(1024 * 32)

    from Devs import BaseDevice

    base = BaseDevice.BaseDevice()
    base.start()

except Exception as e:
    print_exception(e)

    from sys import modules, path # @Reimport
    from gc import collect # @Reimport

    path.clear()
    path.insert(0, '')
    path.insert(0, '/sd/Backup')

    modules.clear()
   
    collect()

    for x in dir():
        exec('del {}'.format(x))

    try:
        from gc import collect, threshold  # @UnresolvedImport @Reimport
        from micropython import alloc_emergency_exception_buf  # @UnresolvedImport @Reimport
        from sys import print_exception # @UnresolvedImport @Reimport

        alloc_emergency_exception_buf(100)

        collect()
        threshold(1024 * 32)

        from Devs import BaseDevice # @Reimport

        base = BaseDevice.BaseDevice()
        base.start()

    except Exception as e:
        print_exception(e)

At the moment a reset seems to be the only way of clearing the memory reliably.

User avatar
deshipu
Posts: 1329
Joined: Thu May 28, 2015 5:54 pm

Re: How to free memory after unused imports?

Post by deshipu » Fri Jun 15, 2018 11:01 am

The more complex it is, the bigger a chance that you will have some reference to one of its variables or classes left somewhere. Especially when the module runs code on import, and not only defines some variables and classes.

Also, strings literals and variable names are interned and never released.

User avatar
pythoncoder
Posts: 2967
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: How to free memory after unused imports?

Post by pythoncoder » Sat Jun 16, 2018 5:59 am

It's also of note that some libraries retain state after use. This is particularly evident in uasyncio where a soft reset is always required between application runs. The reason may be the fact that it implements singleton objects as global instances (I prefer to design singleton classes) but I haven't adequately thought this through to be sure it's the cause.
Peter Hinch

Post Reply