@micropython.native not implemented too long
@micropython.native not implemented too long
I forget the exact error message.
Is there documentation on what triggers this or where the limits are?
Today, I took a MicroPython file that runs just fine and ran it through a minification program. It removes all comments, shortens all variable names, etc. The standard python_minifier tool.
To my surprise, when I tried to run the minified version on a Pico board I got the "too long" error message. The non-minified file runs just fine.
Any thoughts on this?
Is there documentation on what triggers this or where the limits are?
Today, I took a MicroPython file that runs just fine and ran it through a minification program. It removes all comments, shortens all variable names, etc. The standard python_minifier tool.
To my surprise, when I tried to run the minified version on a Pico board I got the "too long" error message. The non-minified file runs just fine.
Any thoughts on this?
Re: @micropython.native not implemented too long
I'll assume from the "not implemented" part that it's likely
https://github.com/micropython/micropyt ... umb.c#L540
or possibly
https://github.com/micropython/micropyt ... umb.c#L405
I don't know off the top of my head why the minifier is causing this unless it's doing inlining?
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: @micropython.native not implemented too long
As a general point I stopped using pyminify on MP programs several years ago. Cross-compiling achieves a size reduction on the order of 3:1. It has other benefits: faster loading and lower RAM use because the code is pre-compiled.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: @micropython.native not implemented too long
Playing with different workflows at the moment. That's actually on my list to explore in a day or two.pythoncoder wrote: ↑Mon Jul 11, 2022 7:58 amAs a general point I stopped using pyminify on MP programs several years ago. Cross-compiling achieves a size reduction on the order of 3:1. It has other benefits: faster loading and lower RAM use because the code is pre-compiled.
How much of an improvement in execution speed do you see with this approach?
Re: @micropython.native not implemented too long
The only performance difference is that there's no startup cost to parsing and compiling the .py file, instead you're directly loading the pre-compiled bytecode (and/or native code). From that point on though it's the same performance as it's the same compiler (just it ran on your pc instead)
Re: @micropython.native not implemented too long
Circling back to this, not sure what it might be doing that is causing the anger of MicroPython gods. I don't think I am going to pursue this path. Probably not worth the effort required to diagnose and narrow it down to a root cause at this point.
Here's the relevant portion of the minification code:
Code: Select all
import python_minifier as pm
...
result = pm.minify(sfp.read(),
remove_literal_statements=True,
rename_locals=True,
rename_globals=True,
preserve_locals=locals_to_preserve,
preserve_globals=globals_to_preserve)
Re: @micropython.native not implemented too long
Knowing the exact error you're seeing would help. But it should be pretty clear -- either your method is using a lot more locals than before, or it's just gotten longer.
Either way though, what Pete said -- use the cross-compiler instead. It's designed for this purpose.
Re: @micropython.native not implemented too long
I had trouble reproducing it because I wasn't sure where the trigger point might be. I've seen methods that threw this error before magically work just fine after a few edits and trying again. Here it is:jimmo wrote: ↑Mon Jul 11, 2022 4:12 pmKnowing the exact error you're seeing would help. But it should be pretty clear -- either your method is using a lot more locals than before, or it's just gotten longer.
Either way though, what Pete said -- use the cross-compiler instead. It's designed for this purpose.
Code: Select all
>>> %Run -c $EDITOR_CONTENT
NotImplementedError: native method too big
>>>
Being that this is my first real MicroPython project I am still on a journey of discovering best practices for optimization at every level. I've resorted to assembler where I could. Sadly, no time to write some modules in C for this one, maybe next time. I'll probably mess with cross-compilation next. I am sure that's the right path, I just haven't touched it yet. Thanks.
Re: @micropython.native not implemented too long
Just ran into this one again using mpy-cross.
What's interesting is that the source files, unmodified, run just fine on the RP2040. No warnings whatsoever.
The minute I tried to pre-compile I got:
Not complaining (how could one when it comes to amazing free open-source software), just stating what I think is a fact. I think I have probably wasted days chasing after errors that do not provide sufficient information for one to at least get close to the problem with a reasonable effort. I had one case where I got a unicode error and the only traceback offered was, effectively main(). The source file for that one was nearly 2000 lines of code. It probably took a couple of hours to find the problem.
In this case, when the code runs just fine on the processor and mpy-cross says it can't handle it. Well, not sure what to do. All I can do is remove one @micropyhon.native decorator at a time and try again. Which, of course, burns time at a rate proportional to the square of your desperation to get the job done.
Anyhow. I'll see if I can provide something more useful than what is likely indistinguishable from a rant. Not intended that way. FOSS is what it is, take it or leave it and don't complain.
What's interesting is that the source files, unmodified, run just fine on the RP2040. No warnings whatsoever.
The minute I tried to pre-compile I got:
Code: Select all
NotImplementedError: native method too big
In this case, when the code runs just fine on the processor and mpy-cross says it can't handle it. Well, not sure what to do. All I can do is remove one @micropyhon.native decorator at a time and try again. Which, of course, burns time at a rate proportional to the square of your desperation to get the job done.
Anyhow. I'll see if I can provide something more useful than what is likely indistinguishable from a rant. Not intended that way. FOSS is what it is, take it or leave it and don't complain.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: @micropython.native not implemented too long
I suggest you review your use of micropython.native. Like all optimisations it's best only to apply it where there is a true performance bottleneck.
My approach is to identify which part of the code is causing the slowdown and optimise it until it sqeals. micropython.native rarely makes a large difference. micropython.viper makes a big difference but needs some effort to use. For even better performance consider inline assembler or C modules.
My approach is to identify which part of the code is causing the slowdown and optimise it until it sqeals. micropython.native rarely makes a large difference. micropython.viper makes a big difference but needs some effort to use. For even better performance consider inline assembler or C modules.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.