It would be very sad to have to switch to a Pi this late in the game, as well as it being impossible to buy bulk Pi Zero boards for our project, (meaning a budget explosion of Pi Model A boards given the number of boxes we want as well as reopening a hella can of worms on power issues for battery-powered ones, and delaying our imminent install).
CANDIDATES FOR GAINING RAM
I speculate I can gain RAM from the following...
1) Shutting down functions/configurations not needed. E.g. can we save RAM by...
* ...disabling networking/Wifi? Our deployment doesn't use it. If so, what steps are needed to reclaim this RAM. I can't find any relevant options.
* ...dropping line-number or other debugging metadata? Once the boxes are running, debug is irrelevant. Is reclaiming RAM from unnecessary bytecode metadata possible, how?
* ...disabling the VFAT filesystem? Assuming I can use frozen _boot.py for startup I don't need it. Is MICROPY_FATFS = 0 in the Makefile the only change needed?
2) Re-architecting where I've misunderstood which operations use RAM.
* See also THINGS I AM ALREADY DOING and THINGS I BELIEVE below.
3) Other interventions I haven't even thought of
* Are there standard things which people drop to gain themselves an extra few bytes of RAM given the freedom to modify firmware image?
THINGS I AM ALREADY DOING
I have spent an extraordinarily long time optimising our RFID text adventure to minimise RAM use on a NodeMCUv2 running a dedicated Micropython firmware image built from Git. This image includes as many optimisations I can identify and I'm sure I've missed some.
At this point I'm so close to RAM limits that the final features I need to add will tip over the limit. I really hope I have missed some obvious solutions to reclaim RAM as we're so close. Currently we can load screen and RFID reader, write to the screen and run the text adventures from the console, but reading and writing RFID needs a bunch more RAM (the RFID cards are 1k, currently I have < 4k left with smaller text adventures and the larger text adventure stories we have are impossible to load).
I've had a good look at https://docs.micropython.org/en/latest/ ... ained.html and http://docs.micropython.org/en/v1.8.6/p ... ython.html and I think I've followed all of those guidelines as much as I can.
In my code I am already using...
* only Frozen Modules
* trying to import in dependency order, and running a gc.collect() after every import
* bytes objects and strings for data to avoid loading them into RAM e.g. for typography data in my bitfont library ( https://github.com/ShrimpingIt/bitfont )
* memoryview e.g. for SPI send buffer in my ST7920 library ( code example https://github.com/ShrimpingIt/micropyt ... /st7920.py)
* .format() as a substitute for concatenation of strings by plus operator
* incremental gc triggered when I suspect a large allocation has been made (e.g. on import)
* precompiled templates using https://github.com/pfalcon/utemplate to minimise eval at runtime
In my firmware build (based on fortnight-old micropython and esp_open_sdk git checkout) I'm already...
* disabling Berkeley Tree Database - in Makefile: MICROPY_PY_BTREE = 0
* increasing RAM limit to 40k - main.c: STATIC char heap[40 * 1024];
* increasing the size of irom - esp8266.ld: irom0_0_seg : org = 0x40209000, len = 0xb7000 AND in modesp.c: return MP_OBJ_NEW_SMALL_INT(0xc0000);
* populating known qstrings - qstrdefsport.h contains all dynamically loaded utemplate module names
THINGS I BELIEVE
* RAM used by the Stack (function-local variables) is garbage-collected immediately on return from a function
* Frozen modules use no RAM for the following, which remain in program memory ...
- class and function definitions
- literal strings and bytes
* RAM is allocated by a frozen module when...
- module is loaded (in-RAM metadata describing the module)
- constructing primitives or objects and storing to heap during execution of class and function definitions
THINGS I'M CONFUSED BY
Lots of RAM is used by loading stories like https://github.com/cefn/avatap/blob/mas ... enhouse.py even though they are frozen modules, all recursively-imported modules were already imported, and the stories are overwhelmingly composed of strings that I would expect to remain in ROM. You can see this in the reference code loading results below, where the step of loading the senhouse story permanently uses up 8kB of RAM!
I don't know the cost of the dirty hack of increasing RAM described at https://gerfficient.com/2016/10/19/fix- ... n-esp8266/ There must be some reason this is not the default configuration.
I found that spi sends triggered by st7920.Screen#redraw() actually DO reduce available RAM. I can't figure why (each write is backed by a locally-scoped sub-memoryview presumably disposed at the end of the function call, mapped to a pre-allocated command buffer bytearray). See example code at https://github.com/ShrimpingIt/micropyt ... /st7920.py and the memory_test reference code described below.
SUMMARY
Any thoughts on tricks and tips I have missed would be like gold-dust at this stage of proceedings. I really appreciate any effort people can make to scan our approach for any obvious issues we can address. Thanks all!
REFERENCE CODE
A reference example for the routine we are running is at https://github.com/cefn/avatap/blob/mas ... ry_test.py
The
Code: Select all
report_import
This results in the following log from a clean boot of our firmware image, the report lines are understood as {RAM free after import}+{amount reclaimed by gc}={total RAM still available} ...
Code: Select all
>>> from regimes.memory_test import *
faces.font_5x7
26624+288=26880
st7920
25984+272=26224
machine
26160+80=26208
15856+7808=23632
milecastles
18864+736=19568
boilerplate
18720+176=18864
engine
17968+224=18160
consoleEngine
17456+208=17632
stories.senhouse
9616+48=9632
7056+2016=9040
mfrc522
8240+112=8320
Immediately after loading, the memory map looks like this...
Code: Select all
>>> micropython.mem_info(1)
stack: 2144 out of 8192
GC: total: 40320, used: 32592, free: 7728
No. of 1-blocks: 569, 2-blocks: 47, max blk sz: 370, max free sz: 338
GC memory layout; from 3ffef2e0:
00000: MDhhhBMhTDMDhDhB=BBBh===h====h=hhh==h===========================
00400: ================================================================
00800: ================================================================
00c00: ================================================================
01000: ============================================BAMhDhBh=Bh==hh=hh==
01400: =====h==========================================================
01800: ================================================================
01c00: ================================================================
02000: ================================================================
02400: ================================================================
02800: =======================================================TDh=hBT=M
02c00: DhMMBBDMDhBBBAMDShh==DAhDhh=h========h=Bh=BBh===B==BBBBBMB=LB=DD
03000: hhBBBDh=BBBh==================BBBB=B=B=B=BhB=hB=h=====h===Lhh=Bh
03400: =B=hh=========================h=======================h===hhh===
03800: =====MDSMDBhTB=BBBBhhBhhDhhBhhhBhhBhhBhhBhhBhDhhhhhBhhBhhhhhBhhD
03c00: hhhhhBhhBhhhhhhMhhDhhDhhBhhDhhBhhBhhBhhBhhDhhDhhDhhhhhh=========
04000: ======hDhhhhhBhhhhhBhhBhhBhhBhhBhhBhhBhhBhhBhhBhhDhhBhhBhhDhhhhh
04400: BhhhhhBhhBhhDhhhhhLhhhhhBhhh=hh===h===h=h===h====h==h=.h===h===B
04800: Bh==h===h===h=h==h===h===h===h==h===h========BDBBhSLhh=BBBBBBBh=
04c00: ==BDBh=====Bh=Bh===BDBh====BBh===BDhBBhBBDh===h===h=============
05000: =h===h==h===hDBBhLhBB=hh===h===DBh=Dh===hDhBhBhBBhBBhh====h===B=
05400: Bh===h===h========h===MMDhDDBhBh===hDhhDh==h========B=h===h=====
05800: ===hhhDhh===hBh=h===h=====h========h=h=.hDShDh==hDhhhDhhhDhhhhhD
05c00: DhDShh..BhhBBBhDhhDhDhDhhhhBBBBB.Bh==B=hDBLBBBBBh=Dhh.Sh=h==LLhh
06000: LhLhh==hLh====LhhhLLLLLh=LhLhh===LhLhhLLh====LLh.hSh===h===h=h==
06400: ======hLhhh=Lh==hLhLhhLhh===hhh=hh==h====================h==LhLL
06800: hhhLhh====h===hh.hhLhhhh===LLDh===================hhh==Lhh=h===h
06c00: ===LLhh==h====h===hDLh===h===.LhLh===hh====h.h===DhDhh.hh====h==
07000: ================.LhDhhD.hh=Sh===h====h====Lhh===h==h===h===..Lhh
07400: ===.h...hh==================h===...h=================h===h===h==
07800: =.h==h====h====....D...Dhh.hD..hh===.....Dhh..Lh..Dhhh=h......h.
07c00: ....Dh=T...........h==Dhhh==================hh....hh==h=========
08000: =====......Sh===========...............h=h==============........
08400: ...............h=================.........h==...................
08800: .....h..........................................................
(4 lines all free)
09c00: ........................