Page 5 of 7
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:24 pm
by kevinkk525
ok I ran it:
Code: Select all
dir()
['uos', 'gc', '__name__', 'main']
>>> sys.modules
{'main': <module 'main'>}
>>> from pysmartnode.networking import mqtt_receive_config
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys', 'mqtt_receive_config']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking.mqtt_receive_config': <module 'pysmartnode.networking.mqtt_receive_config'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
>>> del mqtt_receive_config
>>> del sys.modules["pysmartnode.networking.mqtt_receive_config"]
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
>>> del sys.modules["pysmartnode.networking"]
>>> del sys.modules["pysmartnode"]
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'main': <module 'main'>}
>>>
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:26 pm
by jickster
kevinkk525 wrote: ↑Fri Sep 21, 2018 8:24 pm
ok I ran it:
Code: Select all
dir()
['uos', 'gc', '__name__', 'main']
>>> sys.modules
{'main': <module 'main'>}
>>> from pysmartnode.networking import mqtt_receive_config
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys', 'mqtt_receive_config']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking.mqtt_receive_config': <module 'pysmartnode.networking.mqtt_receive_config'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
>>> del mqtt_receive_config
>>> del sys.modules["pysmartnode.networking.mqtt_receive_config"]
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
>>> del sys.modules["pysmartnode.networking"]
>>> del sys.modules["pysmartnode"]
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'main': <module 'main'>}
>>>
Do it
without
We're trying to see if you can delete only submodules.
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:28 pm
by kevinkk525
Sorry that I added these lines below.. it was all there:
Code: Select all
dir()
['uos', 'gc', '__name__', 'main']
>>> sys.modules
{'main': <module 'main'>}
>>> from pysmartnode.networking import mqtt_receive_config
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys', 'mqtt_receive_config']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking.mqtt_receive_config': <module 'pysmartnode.networking.mqtt_receive_config'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
>>> del mqtt_receive_config
>>> del sys.modules["pysmartnode.networking.mqtt_receive_config"]
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:30 pm
by jickster
kevinkk525 wrote: ↑Fri Sep 21, 2018 8:28 pm
Sorry that I added these lines below.. it was all there:
Code: Select all
dir()
['uos', 'gc', '__name__', 'main']
>>> sys.modules
{'main': <module 'main'>}
>>> from pysmartnode.networking import mqtt_receive_config
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys', 'mqtt_receive_config']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking.mqtt_receive_config': <module 'pysmartnode.networking.mqtt_receive_config'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
>>> del mqtt_receive_config
>>> del sys.modules["pysmartnode.networking.mqtt_receive_config"]
>>> dir()
['uos', '__name__', 'main', 'gc', 'sys']
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'pysmartnode.networking': <module 'pysmartnode.networking'>, 'main': <module 'main'>, 'pysmartnode': <module 'pysmartnode'>}
Ok progress. So we are allowed to del submodules.
Question is why isn't the RAM freed.
Are all the modules that you import .py files (and not builtin C-code)?
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:32 pm
by kevinkk525
All files are frozen bytecode.
If you look at the thread I linked, I described that
and then deleting some.module does work in contrast to
. Even that does not work in my program but that's another thing..
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:51 pm
by jickster
kevinkk525 wrote: ↑Fri Sep 21, 2018 8:32 pm
All files are frozen bytecode.
If you look at the thread I linked, I described that
and then deleting some.module does work in contrast to
. Even that does not work in my program but that's another thing..
Wow. I just missed the forest for the trees.
Code: Select all
>>> sys.modules
{'uasyncio': <module 'uasyncio'>, 'uasyncio.core': <module 'uasyncio.core'>, 'main': <module 'main'>}
>>>
Look at those modules in memory!!!!
Those are present even after you delete
pysmartnode.networking.mqtt_receive_config
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 8:58 pm
by kevinkk525
Of course they are, because I did not remove them after unloading the module. But they have nothing to do with no RAM getting freed at all when removing the module. But when importing like "import pysmartnode.networking.mqtt_receive_config", I'm able to unload the module and get some RAM back.
Re: Remove imported module from RAM
Posted: Fri Sep 21, 2018 9:09 pm
by jickster
kevinkk525 wrote: ↑Fri Sep 21, 2018 8:58 pm
Of course they are, because I did not remove them after unloading the module. But they have nothing to do with no RAM getting freed at all when removing the module.
Do you know how much RAM the code of
pysmartnode.networking.mqtt_receive_config takes?
Do you know how much RAM the import statements in
pysmartnode.networking.mqtt_receive_config take?
It's very possible the code of
pysmartnode.networking.mqtt_receive_config takes almost no RAM but the import statements take a ton.
To deduce the relative ratio of RAM usage, need to create experiments on
pysmartnode.networking.mqtt_receive_config
(A) comment out everything except the import statements.
Only the import statements will take RAM because there is literally no code!
(B) comment out the import statements - and the rest of your code since it won't work without the import statements - and put a dummy function that takes A LOT of RAM like this:
Code: Select all
def b():
b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b();b(); return 0
Then rerun the sequence of commands for each case and reply with results for A and B.
Re: Remove imported module from RAM
Posted: Sat Sep 22, 2018 7:23 am
by kevinkk525
The code of pysmartnode.networking.mqtt_receive_config takes about 200Bytes. The import statements a few kB which is asyncio.
Of course I did not intend to get to the same amount of free RAM as before the import of the module as asyncio of course stays in RAM. My problem is, that no RAM gets freed if I unload the module. I should get at least 100Bytes back (which I do if I import it like "import pysmartnode.networking.mqtt_receive_config" which I described in the other thread. This of course is now more fitting to what I described in the other thread but we were not discussing that curiosity here before).
Your dummy function did work the last time when I placed it into a module in the root directory. If I put that function (and only that function) in the pysmartnode.networking.mqtt_receive_config, that same thing happens as before, no RAM gets free if I unload the module. Only once I unload the parent packages "pysmartnode.networking" and "pysmartnode", I get the RAM back.
More interestingly the other method importing "pysmartnode.networking.mqtt_receive_config", which did release the RAM of the module before, now completely refuses to release any RAM even after removing parent packages and pysmartnode stays in dir() which is weird:
Code: Select all
>>> import gc
>>> import sys
>>> gc.collect();gc.mem_free()
22464
>>> import pysmartnode.networking.mqtt_receive_config
>>> gc.collect();gc.mem_free()
22080
>>> del pysmartnode.networking.mqtt_receive_config
>>> del sys.modules["pysmartnode.networking.mqtt_receive_config"]
>>> gc.collect();gc.mem_free()
22032
>>> del sys.modules["pysmartnode.networking"]
>>> gc.collect();gc.mem_free()
22000
>>> del sys.modules["pysmartnode"]
>>> gc.collect();gc.mem_free()
22032
>>> sys.modules
{'main': <module 'main'>}
>>> dir()
['pysmartnode', 'gc', '__name__', 'uos', 'a', 'main', 'sys']
>>>
Remove imported module from RAM
Posted: Sun Sep 23, 2018 2:25 am
by jickster
If you want to save ram you can use FROZEN_MPY.
That’s different than FROZEN_STR.
FROZEN_MPY pre-compiles to bytecode which doesn’t take up RAM when being executed.
Seems like this would solve your problem of RAM usage.
Sent from my iPhone using Tapatalk Pro