Maximise available RAM

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Post Reply
kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Maximise available RAM

Post by kevinkk525 » Sat May 19, 2018 9:10 pm

[Update: My firmware build had an error resulting in 16kB less free RAM. Therefore this is not as important now]

As my project grew, I was constantly hitting the RAM limit of the esp8266. Using frozen modules wasn't enough anymore so I started searching and getting "creative".
This are ways I've found to maximise the available RAM. Feel free to comment and add or criticize the methods and point out the dangers.
I thought it would be nice to have an entry about this on the forum, so it is easier to find.
(Maybe it is somewhere else but I did not find a good summary).


1) Disable the filesystem
Disabling the filesystem by commenting out the filesystem initialization in the _boot.py made an additional ~4kB of RAM usable.
Of course you won't be able to upload, write or read any files anymore. Everything has to be frozen into the firmware.

Code: Select all

"""
from flashbdev import bdev
try:
    if bdev:
        uos.mount(bdev, '/')
except OSError:
    import inisetup
    inisetup.setup()
"""
2) Changing the heap size in main.c
This is admittedly more like an unsafe hack but in combination with 1) I was able to increase it to 44 * 1024B without any problem.
If you increase this too much, allocation can fail or the esp might be stuck rebooting.

Code: Select all

STATIC char heap[36 * 1024];
3) Storing qstr in flash
If not all your modules are frozen bytecode you can store qstr in flash according to this http://docs.micropython.org/en/latest/p ... ained.html.


The first 2 methods made ~12kB of RAM available, which already is quite an impressive amount making development a lot easier.
The 3rd does not apply do me as all my modules are part of the firmware as frozen bytecode.
(Tested with uPy 1.9.4 running a project of mine)
Last edited by kevinkk525 on Fri Dec 21, 2018 7:49 pm, edited 2 times in total.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

OutoftheBOTS_
Posts: 847
Joined: Mon Nov 20, 2017 10:18 am

Re: Maximise available RAM

Post by OutoftheBOTS_ » Sun May 20, 2018 1:06 am

U know there is the cheat method, switch from ESP8266 to ESP32 with psRAM and then you can write the most sloppiest memory wasteful code and still have RAM to burn ;)

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Maximise available RAM

Post by kevinkk525 » Sun May 20, 2018 7:32 am

But that's a very expensive cheat method ;)
An esp8266 costs about 2.5€ but an esp32 costs at least 6€ without psram...
Oh and I would like to keep my 10 esp8266 useful :D

I do have 2 esp32, one with psram, though.. for the really big projects.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Maximise available RAM

Post by pythoncoder » Sun May 20, 2018 7:48 am

@kevinkk525 Have you tried the technique of storing qstr's in flash? documented here.
Peter Hinch
Index to my micropython libraries.

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Maximise available RAM

Post by kevinkk525 » Sun May 20, 2018 12:04 pm

no because I do not even understand what this is :/
I can execute micropython.qstr_info(1) and get a lot of entries that are my module names (with path) and a few constants I use, which do not really fit to the modules but how do I know which ones are invalid?
And what does putting these into qstrdefsport.h change?

As I understand it, you can also use it to store any string in flash and access it by using a number (but how do I access it?).

Don't all strings in a module frozen as bytecode remain in flash?
According to the documentation:
When considering the implications of frozen bytecode, note that in Python strings, floats, bytes, integers and complex numbers are immutable. Accordingly these will be frozen into flash. Thus, in the line
mystring = "The quick brown fox"
the actual string “The quick brown fox” will reside in flash. At runtime a reference to the string is assigned to the variable mystring. The reference occupies a single machine word.
That means all my strings are already stored in flash, not in RAM.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Maximise available RAM

Post by pythoncoder » Mon May 21, 2018 6:09 am

You're right if all your modules are frozen. The technique of storing qstrs in Flash can save RAM during debugging where the module under development is not frozen.
Peter Hinch
Index to my micropython libraries.

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: Maximise available RAM

Post by kevinkk525 » Mon May 21, 2018 8:06 am

Ah ok. Good that I understood that correctly. My modules are all frozen bytecode so I don't gain anything from this.
It's not an easy method though.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

Post Reply