ram needed during import far exceeds ram needed after

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
seandepagnier
Posts: 15
Joined: Tue Feb 25, 2020 5:10 pm

ram needed during import far exceeds ram needed after

Post by seandepagnier » Sat May 30, 2020 3:41 pm

I am putting my script into flash using ampy. The script is about 1200 lines of python.

If I cut the script to the first 580 lines, it imports and mem_free shows 44288. If I put a few more lines, it fails to import

such as:
MemoryError: memory allocation failed, allocating %u bytes

Maybe by breaking the file up into files with less than 580 lines it can load and work with this little ram? Is this a limitation of micropython?

I believe this is also maybe a small bug that it prints %u to the console without formatting it.

User avatar
tve
Posts: 216
Joined: Wed Jan 01, 2020 10:12 pm
Location: Santa Barbara, CA
Contact:

Re: ram needed during import far exceeds ram needed after

Post by tve » Sat May 30, 2020 4:49 pm

Which platform are you using?
Which version of MicroPython?
Can you provide a transcript showing the problem and error message?

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: ram needed during import far exceeds ram needed after

Post by kevinkk525 » Sat May 30, 2020 6:14 pm

This is a known behaviour and I'm sure it is documented somewhere.

If you put a normal .py file onto your microcontroller, it has to be compiled on the device. This needs lots of RAM and therefore you can run out of RAM during this process, even though after the compiling it looks like there is lots of RAM available.

The workaround is to use a precompiled .mpy file or freeze your script into the firmware.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

seandepagnier
Posts: 15
Joined: Tue Feb 25, 2020 5:10 pm

Re: ram needed during import far exceeds ram needed after

Post by seandepagnier » Sun May 31, 2020 12:25 am

I managed to break the program into 4 files and it can load now. This actually improved the code quality anyway.

Would it be possible to somehow use "flash as ram" like a swap space?

If I freeze the script in the firmware doesn't it need to be precompiled in this way? I managed to precompile a module on x86, but the recursive clone of esp32 toolchain is going to need a better connection.

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: ram needed during import far exceeds ram needed after

Post by kevinkk525 » Sun May 31, 2020 5:34 am

Breaking large scripts into small modules is always a good idea and improves code quality and the effort needed to maintain it.

You can't use flash as RAM.

On the esp32 I precompile all my files using mpy-cross and then upload the .mpy using an ftp-server on the esp. Works well so far and I don't need to build a custom firmware.

For freezing into firmware you just need to copy the .py files into the modules directory specified in your board.manifest file (I think that's how it is handled now, I'm not up to date on custom firmwares anymore).
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

Post Reply