Remove imported module from RAM

[jickster]

I'm having problems with unloading modules, the RAM does not get freed.

I'm using this function to delete a module:

Code: Select all

def unloadModule(mod):
    # removes module from the system
    mod_name = mod.__name__
    if mod_name in sys.modules:
        del sys.modules[mod_name]

I am importing a module and call one function. Then I delete it like that:

Code: Select all

from pysmartnode.helpers import helper
helper.help()
unloadModule(helper)
del helper

But the RAM is only a small amount higher than with the module imported, far away from the free RAM before I imported the module.

I’m not saying your approach is wrong or right but if it has any chance of working, you have to call gc.collect() are you call del

del by itself just removes an entry from the dict.

If you want the memory it used to point to to be freed, you have to call gc.collect()

However I’m not guaranteeing your approach will actually free the memory; gotta look at the C source code.

[jickster]

Yea , I dont think it is that simple guys .
Interned string of that module will still exist in RAM

import micropython
micropython.qstr_info(1)

There exist every string , the memory that didnt get freed is here.
How to delete interned string may you ask , working on it )

Deleting the bytecode but not the qstrings for a module will still save you the space the bytecode occupies.

[kevinkk525]

Thanks for looking at it. Of course I'm calling gc.collect() many times after deleting a module. Sadly I was never able to get more than a few Bytes back, even if the module was a few kB.

[jickster]

Thanks for looking at it. Of course I'm calling gc.collect() many times after deleting a module. Sadly I was never able to get more than a few Bytes back, even if the module was a few kB.

Interesting. I'm gonna look further into this.

[jickster]

Yea , I dont think it is that simple guys .
Interned string of that module will still exist in RAM

import micropython
micropython.qstr_info(1)

There exist every string , the memory that didnt get freed is here.
How to delete interned string may you ask , working on it )

Create a forum topic so we can follow your progress.

[jickster]

If we delete a module, then import it again later, does it leak? (I don't know the answer to this).

It seems reasonable to import a module, do some setup work (e.g. at reset), then when finished, delete the module to allow more ram for later usage.

the rest is allocated dynamically as you use it.

That is not true.

When you import a module, the bytecode for it is created in RAM immediately before you even use it.

[jickster]

I'm having problems with unloading modules, the RAM does not get freed.

I'm using this function to delete a module:

Code: Select all

def unloadModule(mod):
    # removes module from the system
    mod_name = mod.__name__
    if mod_name in sys.modules:
        del sys.modules[mod_name]

I am importing a module and call one function. Then I delete it like that:

Code: Select all

from pysmartnode.helpers import helper
helper.help()
unloadModule(helper)
del helper

But the RAM is only a small amount higher than with the module imported, far away from the free RAM before I imported the module.

That should work. Here's exactly what I did.

Created a useless module which allocates a bunch of memory

mod_test.py

Code: Select all

a = list(range(500))

and compiled it into the firmware as a frozen_str

In Python console

Code: Select all

import gc
import sys
# note the memory
gc.collect(); gc.mem_free()

import mod_test
# free memory dropped by a lot
gc.collect(); gc.mem_free()

del mod_test
# free memory barely changed
gc.collect(); gc.mem_free()

del sys.modules['mod_test']
# huge free memory increase
gc.collect(); gc.mem_free()

Order of sys.modules dict vs main dict doesn't matter.

This will not free qstrings. Currently qstrings are never freed.

This confirms that memory taken up by variables inside modules are deleted.
Next test would be to import a module that has a lot of CODE but initializes few variables.

This would test whether the code associated with a module name is correctly deleted.

[jickster]

I'm having problems with unloading modules, the RAM does not get freed.

I'm using this function to delete a module:

Code: Select all

def unloadModule(mod):
    # removes module from the system
    mod_name = mod.__name__
    if mod_name in sys.modules:
        del sys.modules[mod_name]

I am importing a module and call one function. Then I delete it like that:

Code: Select all

from pysmartnode.helpers import helper
helper.help()
unloadModule(helper)
del helper

But the RAM is only a small amount higher than with the module imported, far away from the free RAM before I imported the module.

Don't know what to tell you cause it works for me.

I created mod_test.py so that the compiled code would occupy lots of RAM but there's no data that gets created upon import unlike my previous post

Code: Select all

def b():
	b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();	
	return 1

Then I executed the following on the console

Code: Select all

>>> import gc
Type "help()" for more information.

>>> import sys

>>> gc.collect(); gc.mem_free()
4800

>>> import mod_test

>>> gc.collect(); gc.mem_free()
4288

>>> del mod_test

>>> gc.collect(); gc.mem_free()
4400

>>> del sys.modules['mod_test']

>>> gc.collect(); gc.mem_free()
4752

>>>

You can see almost all is reclaimed.

So, I don't know why it doesn't work for you.

[jickster]

Yea , I dont think it is that simple guys .
Interned string of that module will still exist in RAM

import micropython
micropython.qstr_info(1)

There exist every string , the memory that didnt get freed is here.
How to delete interned string may you ask , working on it )

Let's brainstorm here
https://github.com/micropython/micropython/issues/4053

[kevinkk525]

Thanks for looking into this. I don't know what changed, but now it looks like most of the RAM is freed after removing of modules.. bit weird.