Embeddeding frozen module inside downloaded firmware

[pidou46]

Hello,

From my (short) experience, frozen byte embedded inside the firmware is a no option to start serious project with esp8266 to save RAM.

But to do so you need to:

"• Clone the MicroPython repository.
• Acquire the (platform specific) toolchain to build the firmware.
• Build the cross compiler.
• Place the modules to be frozen in a specified directory (dependent on whether the module is to be frozen as source or as bytecode).
• Build the firmware. A specific command may be required to build frozen code of either type - see the platform documentation.
• Flash the firmware to the device."

It’s quite a lot of work, and you have to maintain the repository up to date and solve issues if something goes wrong during build of the firmware.
I think it’s not a big issue for one working with micropython on a daily basis, but for an occasional user who need it a couple of time a month it's quite boring : enough to discourage the use.
It's the reason there's is firmware available to download, thanks’ a lot for that, it save me a lot of efforts.

My thought it's possible to help occasional user by giving them the possibility to embed frozen byte code inside an already build firmware.

Here is how I thought it could work:

"- download the MicroPython firmware
- download cross compiler
- build the frozen byte code of wanted modules
- use a tool to embedded frozen byte code inside the firmware
- flash the firmware to the device"

It would be even easier if cross compiler was able to embed itself the frozen byte code inside the firmware. Easier for the user and avoid maintain another external tool.

For me it looks very practical, does it make sense to someone else?

Do you think it could work?

Should I post an issue on github for that?

Please let me know

[pythoncoder]

Similar ideas have already been extensively discussed on GitHub. A solution will doubtless be implemented in due course.

[pidou46]

Thank's for the hint pythoncoder.

It's in the mpy-cross git branch?

Do you have some keyword to help me found it?

Thanks

[pythoncoder]

I don't know what progress has been made on implementation. A quick GitHub search on "dynamic frozen bytecode" turned up this quote from Damien on 24th Feb 2016 (issue #1817):

"To support "dynamic freezing" (where you can freeze a given module from the prompt) is doable but not on the TODO list just yet "

[pidou46]

Thanks Pythoncoder,

I would like to try another approch.

It seem's that esptool is able to modify a part of the firmware binary file.

If I can get the adress of the beginin of the "module" directory, I guess I can use esptool to update this area of the bin file with the frozen the code from mpy-cross.

what do you think of this approch ?

Do you know where I can find a map of the binary file ?

[pythoncoder]

I think you'll find there's more to it than that, but I have little knowledge of the build process. If you create a build with frozen bytecode you'll find that a file frozen_mpy.c is created and compiled. This contains the qstrs from your Python modules together with numerous function definitions. In my case it runs to 40,521 lines of C.

If you can get to grips with this, design, implement and test a means of doing frozen bytecode without compilation I'm sure the maintainers will welcome the contribution. But I'm well out of my depth here...