@micropython.native not implemented too long

[martincho]

I forget the exact error message.

Is there documentation on what triggers this or where the limits are?

Today, I took a MicroPython file that runs just fine and ran it through a minification program. It removes all comments, shortens all variable names, etc. The standard python_minifier tool.

To my surprise, when I tried to run the minified version on a Pico board I got the "too long" error message. The non-minified file runs just fine.

Any thoughts on this?

[jimmo]

I forget the exact error message.

I'll assume from the "not implemented" part that it's likely

https://github.com/micropython/micropyt ... umb.c#L540

or possibly

https://github.com/micropython/micropyt ... umb.c#L405

I don't know off the top of my head why the minifier is causing this unless it's doing inlining?

[pythoncoder]

As a general point I stopped using pyminify on MP programs several years ago. Cross-compiling achieves a size reduction on the order of 3:1. It has other benefits: faster loading and lower RAM use because the code is pre-compiled.

[martincho]

As a general point I stopped using pyminify on MP programs several years ago. Cross-compiling achieves a size reduction on the order of 3:1. It has other benefits: faster loading and lower RAM use because the code is pre-compiled.

Playing with different workflows at the moment. That's actually on my list to explore in a day or two.

How much of an improvement in execution speed do you see with this approach?

[jimmo]

How much of an improvement in execution speed do you see with this approach

The only performance difference is that there's no startup cost to parsing and compiling the .py file, instead you're directly loading the pre-compiled bytecode (and/or native code). From that point on though it's the same performance as it's the same compiler (just it ran on your pc instead)

[martincho]

I don't know off the top of my head why the minifier is causing this unless it's doing inlining?

Circling back to this, not sure what it might be doing that is causing the anger of MicroPython gods. I don't think I am going to pursue this path. Probably not worth the effort required to diagnose and narrow it down to a root cause at this point.

Here's the relevant portion of the minification code:

Code: Select all

	import python_minifier as pm
	...
		result = pm.minify(sfp.read(),
						   remove_literal_statements=True,
						   rename_locals=True,
						   rename_globals=True,
						   preserve_locals=locals_to_preserve,
						   preserve_globals=globals_to_preserve)

[jimmo]

Circling back to this, not sure what it might be doing that is causing the anger of MicroPython gods. I don't think I am going to pursue this path. Probably not worth the effort required to diagnose and narrow it down to a root cause at this point.

Knowing the exact error you're seeing would help. But it should be pretty clear -- either your method is using a lot more locals than before, or it's just gotten longer.

Either way though, what Pete said -- use the cross-compiler instead. It's designed for this purpose.

[martincho]

Knowing the exact error you're seeing would help. But it should be pretty clear -- either your method is using a lot more locals than before, or it's just gotten longer.

Either way though, what Pete said -- use the cross-compiler instead. It's designed for this purpose.

I had trouble reproducing it because I wasn't sure where the trigger point might be. I've seen methods that threw this error before magically work just fine after a few edits and trying again. Here it is:

Code: Select all

>>> %Run -c $EDITOR_CONTENT
NotImplementedError: native method too big
>>> 

I have no idea which function triggered it (feature request?) in this case because every single function has the decorator. I would have to figure this out by a process of elimination.

Being that this is my first real MicroPython project I am still on a journey of discovering best practices for optimization at every level. I've resorted to assembler where I could. Sadly, no time to write some modules in C for this one, maybe next time. I'll probably mess with cross-compilation next. I am sure that's the right path, I just haven't touched it yet. Thanks.

[martincho]

Just ran into this one again using mpy-cross.

What's interesting is that the source files, unmodified, run just fine on the RP2040. No warnings whatsoever.

The minute I tried to pre-compile I got:

Code: Select all

NotImplementedError: native method too big

Not complaining (how could one when it comes to amazing free open-source software), just stating what I think is a fact. I think I have probably wasted days chasing after errors that do not provide sufficient information for one to at least get close to the problem with a reasonable effort. I had one case where I got a unicode error and the only traceback offered was, effectively main(). The source file for that one was nearly 2000 lines of code. It probably took a couple of hours to find the problem.

In this case, when the code runs just fine on the processor and mpy-cross says it can't handle it. Well, not sure what to do. All I can do is remove one @micropyhon.native decorator at a time and try again. Which, of course, burns time at a rate proportional to the square of your desperation to get the job done.

Anyhow. I'll see if I can provide something more useful than what is likely indistinguishable from a rant. Not intended that way. FOSS is what it is, take it or leave it and don't complain.

[pythoncoder]

I suggest you review your use of micropython.native. Like all optimisations it's best only to apply it where there is a true performance bottleneck.

My approach is to identify which part of the code is causing the slowdown and optimise it until it sqeals. micropython.native rarely makes a large difference. micropython.viper makes a big difference but needs some effort to use. For even better performance consider inline assembler or C modules.