Freezing modules having native and viper functions

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Freezing modules having native and viper functions

Post by pythoncoder » Thu Aug 15, 2019 10:13 am

I thought this was now implemented. The Pyboard D builds the code happily but the outcome is a crash. Other frozen modules work - the fail is with the one which uses those decorators. The offending module runs fine as a .py file.

Has anyone else had problems with this? Or is freezing such code not yet a stable feature?
Peter Hinch
Index to my micropython libraries.

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Freezing modules having native and viper functions

Post by Roberthh » Thu Aug 15, 2019 12:07 pm

Same here, both for PYBD and PYBV11. The -march flag is properly set.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Freezing modules having native and viper functions

Post by pythoncoder » Thu Aug 15, 2019 12:44 pm

Should we raise an issue or is the feature not yet fully implemented?
Peter Hinch
Index to my micropython libraries.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Freezing modules having native and viper functions

Post by jimmo » Thu Aug 15, 2019 1:06 pm

Yeah I'm pretty sure this is supposed to work. Can you please raise a bug. Some things to try though: does it work as an .mpy on the filesystem, rather than freezing? And are you using git master or at the v1.11 tag? And you've done "make" in mpy-cross?

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Freezing modules having native and viper functions

Post by Roberthh » Thu Aug 15, 2019 4:57 pm

Some things to try though: does it work as an .mpy on the filesystem, rather than freezing?
@jimmo: mpy files with native code work fine.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Freezing modules having native and viper functions

Post by pythoncoder » Thu Aug 15, 2019 5:16 pm

In my case I'm using git master. The Makefile is standard and has

Code: Select all

MPY_CROSS_FLAGS += -march=armv7m
I rebuild the cross compiler whenever I pull an update from GitHub.

Running all files in the filesystem, if I cross compile the problematic file (with -march=armv7m) and replace the .py file with the resultant .mpy everything works. The entire project also works with everything frozen except for the problematic file, which I put in the filesystem as a .py file. The only unusual thing about this file is its use of Viper and Native. The other files are pure MicroPython.

@Roberthh perhaps you have a simple test case? Mine isn't ideal as it's a device driver which expects to see hardware and is quite large. So you reckon the issue is with Viper?
Peter Hinch
Index to my micropython libraries.

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Freezing modules having native and viper functions

Post by Roberthh » Thu Aug 15, 2019 6:35 pm

I just picked a file from the tests directory, because they are small. I used tests/micropython/native_misc.py. In the tests/perf_bench there are a few viper & native files, like viper_call2b.py, which use both native and viper code.
So you reckon the issue is with Viper?
The test I made failed with frozen native code. The comment I made above was a response to @jimmo's question, whether mpy files from the file system work. But you tested that already.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Freezing modules having native and viper functions

Post by jimmo » Fri Aug 16, 2019 6:34 am

I think I've found the bug and why it's different between .mpy and frozen. It's because of QSTR handling, and the additional optimisation that the freezing process uses to share the QSTR database with the main build. I'll write up a github bug (or fix if the solution becomes obvious while writing it up!).

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Freezing modules having native and viper functions

Post by jimmo » Fri Aug 16, 2019 2:44 pm

Not actually QSTR-related, but much simpler in the end. Fix in https://github.com/micropython/micropython/pull/5014

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Freezing modules having native and viper functions

Post by pythoncoder » Sat Aug 17, 2019 10:52 am

This definitely fixes a problem, but I'm still experiencing instability.

Testing was done on an SF2W with all modules frozen. The draw_glyph method is Native and calls a Viper function. If the module containing the method (driver.py) is not frozen, everything works. If it is cross compiled and put in the filesystem, everything works.

The oddest symptom is that I can paste lines one at a time at the REPL and the code runs, but if I paste multiple lines it crashes. For example this crashes. The draw_glyph method runs once correctly but crashes: no print data appears:

Code: Select all

from time import sleep_ms
import micropython_ra8875.support.font14 as font14
from micropython_ra8875.tft_local import setup

tft = setup(True, True)
x = 0
y = 100
fmv, rows, cols = font14.get_ch('A')
sleep_ms(500)  # Just being conservative here
tft.draw_glyph(fmv, x, y, rows, cols, (0, 255, 0), (0, 0, 0))
print('gh1')
x += cols
sleep_ms(500)
tft.draw_glyph(fmv, x, y, rows, cols, (0, 255, 0), (0, 0, 0))
print('gh2')
By contrast if I paste

Code: Select all

from time import sleep_ms
import micropython_ra8875.support.font14 as font14
from micropython_ra8875.tft_local import setup

tft = setup(True, True)
x = 0
y = 100
fmv, rows, cols = font14.get_ch('A')
I can then do this until the cows come home:

Code: Select all

>>> tft.draw_glyph(fmv, x, y, rows, cols, (0, 255, 0), (0, 0, 0))
>>> x += cols
I've not previously encountered a case where pasting at the >>> prompt produces different behaviour from pasting at ===.

To add to the confusion three programs call the draw_glyph method in very similar ways. One (the actual big application) works. Two simple test programs crash (at the same point as above). :?

Code lurks here.
Peter Hinch
Index to my micropython libraries.

Post Reply