MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
The flag "MICROPY_PY_REVERSE_SPECIAL_METHODS" seems to enabled only on Unix, Windows ports of MicroPython.
This feature is useful to allow easier implementation of operations with different types not depending on the order, like in ulab module :
>>> np.arange(10) + 1
>>> 1 + np.arange(10) // only works with "MICROPY_PY_REVERSE_SPECIAL_METHODS" enabled
MicroPython modules, like upyuncertainties module, would also profit from this feature.
I've built some Pycom MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled. The difference in firmware size is small :
- "WiPy_ulab_sp_thread_v1.20.2.rc10-g87b47d16e_2020-08-06" has 'text' with 1.411.288 bytes;
- without the flag, 'text' has 1.411.256;
- difference = 32 bytes.
So my question : why "MICROPY_PY_REVERSE_SPECIAL_METHODS" is not enabled by default for stm32, esp32, etc ? Any drawbacks ?
This feature is useful to allow easier implementation of operations with different types not depending on the order, like in ulab module :
>>> np.arange(10) + 1
>>> 1 + np.arange(10) // only works with "MICROPY_PY_REVERSE_SPECIAL_METHODS" enabled
MicroPython modules, like upyuncertainties module, would also profit from this feature.
I've built some Pycom MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled. The difference in firmware size is small :
- "WiPy_ulab_sp_thread_v1.20.2.rc10-g87b47d16e_2020-08-06" has 'text' with 1.411.288 bytes;
- without the flag, 'text' has 1.411.256;
- difference = 32 bytes.
So my question : why "MICROPY_PY_REVERSE_SPECIAL_METHODS" is not enabled by default for stm32, esp32, etc ? Any drawbacks ?
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
Have you checked code size?
The plethora of build options and their presence or absence on different platforms makes writing cross-platform code increasingly difficult. If someone asks whether ulab will run on some arbitrary platform how do you set about answering?
The plethora of build options and their presence or absence on different platforms makes writing cross-platform code increasingly difficult. If someone asks whether ulab will run on some arbitrary platform how do you set about answering?
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
Building on Pycom MicroPython non-PyBytes ulab SP version for LOPY, it shows in the end :
when the firmware size is ok.
Building Pycom MicroPython PyBytes ulab DP version for LOPY :
when the firmware size doesn't fit the firmware partition.
Code: Select all
...
text data bss dec hex filename
1470792 227600 61548 1759940 1adac4 build/LOPY/release/application.elf
...
1720592 bytes => Size OK
...
Building Pycom MicroPython PyBytes ulab DP version for LOPY :
Code: Select all
...
text data bss dec hex filename
1497072 316024 61548 1874644 1c9ad4 build-PYBYTES/LOPY/release/application.elf
...
1835296 bytes => Firmware image size exceeds avialable space on board!
...
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
Any comments about drawbacks of "MICROPY_PY_REVERSE_SPECIAL_METHODS" ?
Otherwise I'll always use "MICROPY_PY_REVERSE_SPECIAL_METHODS" in my new firmware builds for Pyboard's, ESP8266, ESP32, Pycom, etc.
Otherwise I'll always use "MICROPY_PY_REVERSE_SPECIAL_METHODS" in my new firmware builds for Pyboard's, ESP8266, ESP32, Pycom, etc.
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
Re: MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
Most features like that have flags just because the code takes up space that smaller devices might not want.
You'll generally find that these options are on by default for the unix build https://github.com/micropython/micropyt ... ort.h#L103 and the stm32 builds (although I see this particular flag isn't enabled in the stm32 builds).
You'll generally find that these options are on by default for the unix build https://github.com/micropython/micropyt ... ort.h#L103 and the stm32 builds (although I see this particular flag isn't enabled in the stm32 builds).
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
This is becoming a PITA.
I've just run into this "feature" writing a numeric (quaternion) class.
It is another instance of a general problem: it is almost impossible to write cross-platform MicroPython code because language features vary randomly between platforms in a manner which is entirely undocumented.
I raised this RFC but it has provoked little discussion.
It is another instance of a general problem: it is almost impossible to write cross-platform MicroPython code because language features vary randomly between platforms in a manner which is entirely undocumented.
I raised this RFC but it has provoked little discussion.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Reverse special methods and numeric classes
Thinking about this some more, the absence of these special methods is problematic, especially __rtruediv__. If you have a class Foo representing some numeric type, you might ask users to put other types on the RHS:
How do you avoid this:
To save users from compiling a special build, I guess you'd have to implement an inverse() method. This results in revolting syntax like
This would soon get ugly in an expression of any complexity.
Code: Select all
f = Foo()
g = f + 4 # uses normal special method
h = 3 - f # Ask users to avoid this
Code: Select all
x = 7/f # Calls __rtruediv__()
Code: Select all
y = f.inverse() * 7
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
This problem can be helped in a number of ways.rcolistete wrote: ↑Mon Aug 10, 2020 2:04 pm...
when the firmware size doesn't fit the firmware partition.
1. In ulab, you can exclude sub-modules. There are some that are somewhat obscure (e.g., approx, or filter, or compare), and might not hold general appeal. They are there, but it doesn't mean that you need them in all applications.
2. If you want, I can make exclusion customisable at the function level. E.g., vectorise contains a huge number of functions that you are almost never going to use. How many times would you need sinh, or erf on a microcontroller?
3. I am completely re-writing the code (see your PM). In the new version, you can choose the number of dimensions. 1D is quite small now. So, if flash space is an issue, you can compile with 1D arrays, which are still useful for many applications.
4. Many functions accept generic iterables as well as ndarrays as their arguments. We could think about conditionally throwing away the iterable arguments, so that
Code: Select all
import ulab
from ulab import numerical
numerical.std([1, 2, 3])
Code: Select all
import ulab
from ulab import numerical
numerical.std(ulab.array([1, 2, 3]))
5. We could conditionally remove the vectorize function https://micropython-ulab.readthedocs.io ... -functions. You only get a factor of two in speed compared to python.
6. When working on the new features, I realised that associative operators (see my reply to @pythoncoder below) allow for flash savings. https://github.com/v923z/micropython-ul ... ay.c#L1208 We could implement those in the old version, though, I would rather work on the new.
These are just a couple of possibilities.
Instead of compiling all possible combinations, I am wondering, whether one could perhaps turn this problem around, and allow users to configure their own firmware. You would need some sort of service that compiles the firmware, and you could choose, what you want to include.
Last edited by v923z on Sat Aug 15, 2020 4:49 pm, edited 5 times in total.
Re: Reverse special methods and numeric classes
I think the problem is more severe: there are only four associative operators, namely *, +, ==, and !=. All others suffer from the problem that you describe.pythoncoder wrote: ↑Fri Aug 14, 2020 5:05 pmThinking about this some more, the absence of these special methods is problematic, especially __rtruediv__. If you have a class Foo representing some numeric type, you might ask users to put other types on the RHS:How do you avoid this:Code: Select all
f = Foo() g = f + 4 # uses normal special method h = 3 - f # Ask users to avoid this
To save users from compiling a special build, I guess you'd have to implement an inverse() method. This results in revolting syntax likeCode: Select all
x = 7/f # Calls __rtruediv__()
This would soon get ugly in an expression of any complexity.Code: Select all
y = f.inverse() * 7
By exploiting the symmetries of associative operators, I could actually save some flash space in the new version.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: MicroPython firmwares with "'MICROPY_PY_REVERSE_SPECIAL_METHODS'" enabled ?
That's a very good point. It calls into question whether there is much merit in MicroPython supporting only this subset.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.