Large Project Structure query

[pmulvey]

My board is my own design development board using the Wemos Node D1 Mini. It has LCD, LEDS, Switches, RGB, Temperature Sensor, LDR, Speaker, Motor H-Bridge. I use it to teach microcontrollers.
The demo software runs a menu system to demonstrate each of these sensors according to the users’ choice. The demo software is structured into 7 different modules and these are treated as a package using the __init__.py file. In total there is almost 1000 lines of code in these modules. These modules contain libraries for I2C, LCD, Temperature Sensor etc. The executive program resides in the parent directory of the package directory. Usually everything works fine. I had to make liberal use of gc.collect() statements throughout the code. I have added a few additional lines of code to the executive program and I’m now getting the error “MemoryError: memory allocation failed, allocating 57 bytes”. This has happened many times before so I play around with gc.collect(). At this point I cannot get rid of it with gc.collect().
Is this a spurious error message referring to something else altogether?
Is my project structure sound practice?
Should I bite the bullet and figure out the frozen module scenario?
Thank you in anticipation.

[kevinkk525]

Your modules are probably too big to be compiled on the microcontroller. You should think about precompiling the modules or freezing them into the firmware.

If you want to have a look at how I structure my big projects and use gc.collect() then take a look at: https://github.com/kevinkk525/pysmartnode

[jimmo]

Yep, Kevin is right -- module freezing will help you.

The quick explanation is that when you run Python code off the flash filesystem, it needs temporary RAM to load the text, then it stores the compiled bytecode in RAM. When you freeze a module, neither the text nor the compiled bytecode need to live in RAM, it's executed directly off the flash.

Compiling to an .mpy helps a bit because the program text doesn't need to be loaded into RAM, but it still means the bytecode runs from RAM.