Page 1 of 2

Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 10:13 am
by pythoncoder
I thought this was now implemented. The Pyboard D builds the code happily but the outcome is a crash. Other frozen modules work - the fail is with the one which uses those decorators. The offending module runs fine as a .py file.

Has anyone else had problems with this? Or is freezing such code not yet a stable feature?

Re: Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 12:07 pm
by Roberthh
Same here, both for PYBD and PYBV11. The -march flag is properly set.

Re: Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 12:44 pm
by pythoncoder
Should we raise an issue or is the feature not yet fully implemented?

Re: Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 1:06 pm
by jimmo
Yeah I'm pretty sure this is supposed to work. Can you please raise a bug. Some things to try though: does it work as an .mpy on the filesystem, rather than freezing? And are you using git master or at the v1.11 tag? And you've done "make" in mpy-cross?

Re: Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 4:57 pm
by Roberthh
Some things to try though: does it work as an .mpy on the filesystem, rather than freezing?
@jimmo: mpy files with native code work fine.

Re: Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 5:16 pm
by pythoncoder
In my case I'm using git master. The Makefile is standard and has

Code: Select all

MPY_CROSS_FLAGS += -march=armv7m
I rebuild the cross compiler whenever I pull an update from GitHub.

Running all files in the filesystem, if I cross compile the problematic file (with -march=armv7m) and replace the .py file with the resultant .mpy everything works. The entire project also works with everything frozen except for the problematic file, which I put in the filesystem as a .py file. The only unusual thing about this file is its use of Viper and Native. The other files are pure MicroPython.

@Roberthh perhaps you have a simple test case? Mine isn't ideal as it's a device driver which expects to see hardware and is quite large. So you reckon the issue is with Viper?

Re: Freezing modules having native and viper functions

Posted: Thu Aug 15, 2019 6:35 pm
by Roberthh
I just picked a file from the tests directory, because they are small. I used tests/micropython/native_misc.py. In the tests/perf_bench there are a few viper & native files, like viper_call2b.py, which use both native and viper code.
So you reckon the issue is with Viper?
The test I made failed with frozen native code. The comment I made above was a response to @jimmo's question, whether mpy files from the file system work. But you tested that already.

Re: Freezing modules having native and viper functions

Posted: Fri Aug 16, 2019 6:34 am
by jimmo
I think I've found the bug and why it's different between .mpy and frozen. It's because of QSTR handling, and the additional optimisation that the freezing process uses to share the QSTR database with the main build. I'll write up a github bug (or fix if the solution becomes obvious while writing it up!).

Re: Freezing modules having native and viper functions

Posted: Fri Aug 16, 2019 2:44 pm
by jimmo
Not actually QSTR-related, but much simpler in the end. Fix in https://github.com/micropython/micropython/pull/5014

Re: Freezing modules having native and viper functions

Posted: Sat Aug 17, 2019 10:52 am
by pythoncoder
This definitely fixes a problem, but I'm still experiencing instability.

Testing was done on an SF2W with all modules frozen. The draw_glyph method is Native and calls a Viper function. If the module containing the method (driver.py) is not frozen, everything works. If it is cross compiled and put in the filesystem, everything works.

The oddest symptom is that I can paste lines one at a time at the REPL and the code runs, but if I paste multiple lines it crashes. For example this crashes. The draw_glyph method runs once correctly but crashes: no print data appears:

Code: Select all

from time import sleep_ms
import micropython_ra8875.support.font14 as font14
from micropython_ra8875.tft_local import setup

tft = setup(True, True)
x = 0
y = 100
fmv, rows, cols = font14.get_ch('A')
sleep_ms(500)  # Just being conservative here
tft.draw_glyph(fmv, x, y, rows, cols, (0, 255, 0), (0, 0, 0))
print('gh1')
x += cols
sleep_ms(500)
tft.draw_glyph(fmv, x, y, rows, cols, (0, 255, 0), (0, 0, 0))
print('gh2')
By contrast if I paste

Code: Select all

from time import sleep_ms
import micropython_ra8875.support.font14 as font14
from micropython_ra8875.tft_local import setup

tft = setup(True, True)
x = 0
y = 100
fmv, rows, cols = font14.get_ch('A')
I can then do this until the cows come home:

Code: Select all

>>> tft.draw_glyph(fmv, x, y, rows, cols, (0, 255, 0), (0, 0, 0))
>>> x += cols
I've not previously encountered a case where pasting at the >>> prompt produces different behaviour from pasting at ===.

To add to the confusion three programs call the draw_glyph method in very similar ways. One (the actual big application) works. Two simple test programs crash (at the same point as above). :?

Code lurks here.