MemoryError when importing a custom module

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: MemoryError when importing a custom module

Post by SpotlightKid » Mon May 09, 2016 11:58 am

deshipu wrote:I'm afraid const() will only save memory when used within the same module. If you use it from another module, it has to live in memory.
That's surprising. Can you elaborate on why this is so?

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: MemoryError when importing a custom module

Post by deshipu » Mon May 09, 2016 1:41 pm

SpotlightKid wrote:
deshipu wrote:I'm afraid const() will only save memory when used within the same module. If you use it from another module, it has to live in memory.
That's surprising. Can you elaborate on why this is so?
Because the const() optimization happens at the compile time, see: http://forum.micropython.org/viewtopic. ... onst#p2561

Actually, after re-reading that, I think that it won't save any memory at all in either case.

SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: MemoryError when importing a custom module

Post by SpotlightKid » Mon May 09, 2016 1:54 pm

So, if I want to put constant definitions in a separate module, to be imported by my other modules or main program, "const" won't be of any use to me at all? That's unfortunate, because it puts a penalty on modularizing the code.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MemoryError when importing a custom module

Post by pfalcon » Mon May 09, 2016 3:53 pm

deshipu wrote:maybe by also deactivating the AP/STA interfaces -- whichever you are not using.
This won't help - MicroPython heap, used for Python objects, and OS heap, used largely for networking-related activity, are separate. This is both good and bad. Good because out-of-memory situation in one won't affect another (well, OOM in OS heap can easily lead to system crash), bad, because a particular application may get not as much memory as it could otherwise. Given that we're trying to get more reliability from ESP8266 (which is otherwise pretty buggy), the choice is obvious.

@deshipu, @platforma please mind the above and help spread the word.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MemoryError when importing a custom module

Post by pfalcon » Mon May 09, 2016 4:09 pm

In the meantime, I bumped uPy heap size in the master to 28K as I suggested in another topic. Note that this leaves ~24K after filesystem mount (it requires 4K buffer). That should give a hint how to get more memory for an adhoc application - don't mount filesystem, put your code in frozen modules, start from _boot.py.

Please test networking functionality with the above heap size and share experiences (then also repeat with 32K).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: MemoryError when importing a custom module

Post by deshipu » Mon May 09, 2016 5:02 pm

pfalcon wrote:
deshipu wrote:maybe by also deactivating the AP/STA interfaces -- whichever you are not using.
This won't help - MicroPython heap, used for Python objects, and OS heap, used largely for networking-related activity, are separate.
Thanks for correction. I had the impression that before connecting to my wifi the esp had more ram available, but that might as well have been the result of the fact that it was the first boot after flashing. I suppose that when things stabilize a little, I should try and do some experiments determining what helps the memory use and what doesn't.

Llwy
Posts: 34
Joined: Thu Mar 10, 2016 7:43 am

Re: MemoryError when importing a custom module

Post by Llwy » Mon May 09, 2016 9:31 pm

Dear Paul,
Thank you for the explanation on the two heaps being separated.
I will try to compile the master and see if the increase in RAM is enough for me to load my modules.

@Deshipu : and now I understand better why const() does not really help here, thank you.

Llwy

User avatar
platforma
Posts: 258
Joined: Thu May 28, 2015 5:08 pm
Location: Japan

Re: MemoryError when importing a custom module

Post by platforma » Tue May 10, 2016 10:22 am

Thanks Paul. That's good to know, will take this into account.

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: MemoryError when importing a custom module

Post by Roberthh » Tue May 10, 2016 11:34 am

Again - are there any plans to support pre-compiled bytcode, e.g. the output of mpy-cross? That would also help to deal with memory shortage. Most of the time, it's the compile phase which fails.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MemoryError when importing a custom module

Post by pfalcon » Tue May 10, 2016 9:45 pm

Llwy wrote: I will try to compile the master and see if the increase in RAM is enough for me to load my modules.
Please do. There were further changes to optimize memory usage.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

Post Reply