Storing and executing compiled code in ROM

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
tomlogic
Posts: 13
Joined: Tue Jul 19, 2016 10:20 pm

Storing and executing compiled code in ROM

Post by tomlogic » Mon Sep 12, 2016 9:15 pm

(After originally posting this, I found this post from almost two years ago, but I think that with mpy_cross and persistent bytecode, the situation has changed since then.)

I'm working on a port of MicroPython to a platform without a filesystem. I'm currently considering a process of uploading a .py file via serial, compiling it on the device with MicroPython, and then storing the compiled (frozen?) bytecode to a reserved area of memory-mapped flash for automatic execution on every boot.

My question is whether I can just copy a portion of the struct generated by mp_builtin_compile() into flash, and then execute it directly from flash using code_execute(). Maybe I would store the mp_raw_code_t (or the bytecode of it) in flash and create a stub mp_obj_code_t (or mp_raw_code_t structure) to point into it?

My goal is to reserve as much RAM as possible for execution by keeping bytecode in ROM, but I'm still diving through the various structures looking for the "right" place to do so. I see the code in emitglue.c related to saving off .mpy files (mp_raw_code_save()), but that doesn't seem like the correct path to follow given that .mpy files get loaded into RAM for execution.

Is there a platform already doing this that I should review (I'm currently looking at pyboard and esp8266 platforms)? Should I be trying to mimic the way MicroPython stores/loads frozen modules?

User avatar
pythoncoder
Posts: 3599
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Storing and executing compiled code in ROM

Post by pythoncoder » Tue Sep 13, 2016 8:18 am

Might it be easier to cross compile your code and upload the bytecode via serial?
Peter Hinch

tomlogic
Posts: 13
Joined: Tue Jul 19, 2016 10:20 pm

Re: Storing and executing compiled code in ROM

Post by tomlogic » Wed Sep 14, 2016 12:49 am

pythoncoder wrote:Might it be easier to cross compile your code and upload the bytecode via serial?
If I cross-compile and upload bytecode via serial, can I then execute that from its location in flash without having to load it into RAM?

User avatar
pythoncoder
Posts: 3599
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Storing and executing compiled code in ROM

Post by pythoncoder » Wed Sep 14, 2016 5:39 am

As I understand it persistent bytecode executes from flash on most platforms, the notable exception being the CC3200 where code is executed from RAM.
Peter Hinch

tannewt
Posts: 35
Joined: Thu Aug 25, 2016 2:43 am

Re: Storing and executing compiled code in ROM

Post by tannewt » Sat Oct 22, 2016 1:43 am

How big is your ROM? Most ports place a filesystem in ROM if theres enough space. It doesn't need external flash necessarily.

User avatar
dhylands
Posts: 3227
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Storing and executing compiled code in ROM

Post by dhylands » Sat Oct 22, 2016 5:20 am

tomlogic wrote:
pythoncoder wrote:Might it be easier to cross compile your code and upload the bytecode via serial?
If I cross-compile and upload bytecode via serial, can I then execute that from its location in flash without having to load it into RAM?
If you upload via serial, then presumably, the bytecode is being stored as a .mpy file in a flash filesystem, which means that the bytecode will be loaded into RAM.

Currently, in order to have the bytecode not be loaded into RAM, it has to be linked into the MicroPython firmware.

Post Reply