Freezing code and modules

[Jim.S]

I am trying to understand how I can reduce the size of the code downloaded to a pyboard (or more specifically a microbit) by compiling the accompanying module file to byte code before I download it to the board.

I have looked at the tutorial on the adafruit website that shows how to make custom firmware for their ESP8266 boards
https://learn.adafruit.com/micropython- ... en-modules but that approach (ie. having the module file in a specific place in the directory tree before building the firmware) seems very specific to the ESP8266 and I can not understand if the same approach is possible with the pyboard (or the microbit).

Normal Python will automatically compile the module file to byte code, (as a .pyc file), presumably speeding up subsequent runs of the code, is something like this possible with micropython?

I also understand Python has a freeze utility which can compile the program and module file to an executable, While I really only want to make the module file smaller, is there anything similar for micropython.

[deshipu]

You can precompile your MicroPython code to .mpy files with the mpy-cross tool that is in the repository. Note, however, that it will not make the file any smaller. It will make it slightly faster and will make it use less memory, though.

[Roberthh]

The way of freezing a python module into the flash image of the firmware is not specific to ESP8266. It is available to all ports. So you have two steps in saving RAM:
a) pre-compiling, which is done by runnig mpy-cross on your PC. The resulting file is either copied into the file system or bundled with the firmware image. If in the file system, the bytecode will be in RAM during execution.
b) Freezing into the firmware, which includes the step of pre-compiling. Them, the bytecode and static data will be in flash during execution, and RAM is used only for dynamic data. The code runs a little bit slower, but the possible code size is drastically expanded.

[Jim.S]

Thanks Deshipu and Roberthh, I understand a bit better now.

I have briefly experimented with mpy-cross and understand what Deshipu said about it not making the code smaller - but it does offer the opportunity of making the code faster which I had not appreciated and will be useful in one of my projects. I need to find the time to experiment with it.

I still don’t understand where I need to put python module files in the directory structure of the source firmware if I want to build them into the firmware. I understand there is a specific 'module' directory for the ESP8266 build but where would I put them for the pyboard firmware? I have downloaded the source code from git hub (micropython-master) but cannot see an obvious place to put python module files?

[Roberthh]

Freezing the code doe snot make it run faster. It just speeds up the import. Once imported, it runs at the same speed.

[pythoncoder]

... I understand there is a specific 'module' directory for the ESP8266 build but where would I put them for the pyboard firmware?...

ports/stm32/modules/

[Jim.S]

Thanks for that pythoncoder. It is now obvious when I go back and look at the directory structure.

Ive also had a brief test of mpy-cross (using the module code below) and realise that compiling to byte code doesnt speed up this particular example. when I find more time I intend to experiment with the viper emitter options of mpy-cross and compare them with the function decorators (i.e. @micropython-viper) to see if there is a difference.

Code: Select all

import pyb
def performanceTest():
    millis = pyb.millis
    endTime = millis() + 10000
    count = 0
    while millis() < endTime:
        count += 1
    print("Count: ", count)

[Roberthh]

mpy-cross as well as frozen bytecode do not support any emitters that create assembly code, like the native emitter, the viper emitter or the assembly code emitter. These modules still have to be as source code on your device. But typically these contain only a small section of your code.

[Jim.S]

I will admit that I dont fully understand the differences between the various code emitters, but a brief reading of the mpy-cross help file suggests that it can compile byte code using different emitters, see below. Or am I misinterpreting the help file. Do the different emitters have to be specified in the source code using decorators immediately before function definition (i.e. @micropython-viper) and I get mpy-cross to match using the -X option

usage: mpy-cross [<opts>] [-X <implopt>] <input filename>
Options:
-o : output file for compiled bytecode (defaults to input with .mpy extension)
-s : source filename to embed in the compiled bytecode (defaults to input file)
-v : verbose (trace various operations); can be multiple
-O[N] : apply bytecode optimizations of level N

Target specific options:
-msmall-int-bits=number : set the maximum bits used to encode a small-int
-mno-unicode : don't support unicode in compiled strings
-mcache-lookup-bc : cache map lookups in the bytecode

Implementation specific options:
emit={bytecode,native,viper} -- set the default code emitter
heapsize=<n> -- set the heap size for the GC (default 2097152)

[pythoncoder]

This has caused confusion before. It's not obvious how an emit=native option could possibly work without a means of specifying the target architecture/instruction set. In practice, as far as I'm aware, it doesn't work and @Roberthh's post is correct.