Support loading of .mpy files

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
User avatar
Roberthh
Posts: 775
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Support loading of .mpy files

Post by Roberthh » Wed May 25, 2016 7:55 am

For everyone running into memory errors during importing a module, the following hack is useful, enabling the loading of modules precompiled with mpy-cross.
1. Add the following line to esp8266/mpconfigport.h:

Code: Select all

#define MICROPY_PERSISTENT_CODE_LOAD (1)
2. Change line 430 of py/emitglue.c into

Code: Select all

#elif defined(__thumb2__) || defined(__xtensa__)
and rebuild the image. It does not execute .mpy as a frozen module, the .mpy file have to be in the user file system. Obviously, frozen byte code is the better choice. But that will come soon, I hope.

wendlers
Posts: 47
Joined: Wed Mar 16, 2016 10:07 pm

Re: Support loading of .mpy files

Post by wendlers » Fri Jun 03, 2016 10:08 pm

Hi,

very nice :-). Just tried it, and it works. Beside the memory savings, it looks like my module is loading much faster when pre-compiled.

regards,
stefan

chrisgp
Posts: 41
Joined: Fri Apr 01, 2016 5:29 pm

Re: Support loading of .mpy files

Post by chrisgp » Sat Oct 08, 2016 8:32 pm

It looks like this still isn't available in the main repository. Is there any downside to turning this on or reason why it would not be included in the main repository?

If I understand correctly, frozen bytecode (which is now available) is still more desirable because the bytecode is executed from flash and an .mpy file results in the bytecode consuming ram. Even so, it seems like having the option of using an .mpy file is nice for cases when someone might not want to flash new firmware.

User avatar
platforma
Posts: 257
Joined: Thu May 28, 2015 5:08 pm
Location: UK

Re: Support loading of .mpy files

Post by platforma » Tue Oct 11, 2016 3:36 pm

There is a PR by Damien: https://github.com/micropython/micropython/pull/2502
That introduces this for the esp8266, I would expect it to be merged into the master fairly soon.

mianos
Posts: 84
Joined: Sat Aug 22, 2015 6:42 am

Re: Support loading of .mpy files

Post by mianos » Wed Oct 12, 2016 12:09 am

Exiting news! 'mpy' is merged in ESP8266. :D :D :D :D :D :D :D :D :D :D :D :D

Llwy
Posts: 34
Joined: Thu Mar 10, 2016 7:43 am

Re: Support loading of .mpy files

Post by Llwy » Thu Oct 13, 2016 5:40 pm

This is one of the best announcement on this project in a while !

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: Support loading of .mpy files

Post by Lornioiz » Fri Oct 21, 2016 8:14 am

Hello everyone,

as an lower-intermediate level Python desktop user I fairly early crashed into the "memory wall" of mcu world...
I started to read and learn as much as I can about the methodology needed in order to lower the footprint of my application but it is not so easy transition for me, as I'm used to use an infinite quantity of memory.
About frozen bytecode and .mpy I would like to ask a couple of questions:
Which is the difference between frozen bytecode (that, if I understand, are module loaded in a custom firmware) and using a precompiled file .mpy? aren't both bytecode, why one of them is more memory efficient?
One more question (among many that are stemmed from my ignorance): the memory saving is linear among data types? in other words, frozen bytecode or mpy file give us the same memory saving (percentual wise) if used to store a floating point number or a string?
Finally, if you can point me to some resource that can help me understand how this works I would be grateful.

Thank you!

Online
User avatar
dhylands
Posts: 2517
Joined: Mon Jan 06, 2014 6:08 pm
Location: Shuswap, BC, Canada
Contact:

Re: Support loading of .mpy files

Post by dhylands » Fri Oct 21, 2016 8:46 am

bytecode can exist as an .mpy file, in which case its loaded into RAM.
frozen bytecode is compiled the firmware, in which case the bytecode is in flash.

So regular bytecode takes up RAM where frozen bytecode doesn't. The exception to this is on the WiPy, where all of the firmware is executed from RAM, so it doesn't matter whether its frozen or not, it still takes up the same amount of RAM.

Lornioiz
Posts: 36
Joined: Wed Aug 03, 2016 11:39 am
Location: Florence, Italy

Re: Support loading of .mpy files

Post by Lornioiz » Fri Oct 21, 2016 10:52 am

[So regular bytecode takes up RAM where frozen bytecode doesn't.[/quote]

Thank you very much Dave,

That means that in a esp8266 I can import as much modules as I want (by the import statement) without using any ram if they are saved in the firmware? just curious, how severe is the speed penalty in doing so (I'm thinking about freezing as much of the application I can by splitting it in various modules)?

Thanks!!

User avatar
Roberthh
Posts: 775
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Support loading of .mpy files

Post by Roberthh » Fri Oct 21, 2016 1:58 pm

There is some speed penalty documented somewhere. As far as I recall, it was like 25%.

Post Reply