ulab for ESP8266
ulab for ESP8266
Hi,
I just got an ESP8266 and flashed micropython firmware on it. My main goal was to try out the ulab(port of numpy) before ordering a D series PyBoard.
Does anyone have any pointers on how to install ulab on ESP8266 micrpython.
Thanks
I just got an ESP8266 and flashed micropython firmware on it. My main goal was to try out the ulab(port of numpy) before ordering a D series PyBoard.
Does anyone have any pointers on how to install ulab on ESP8266 micrpython.
Thanks
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
Tried compiling ulab on ESP8266, but there are issues with ESP8266 partitions :
Need to try changing the flash layout, which is not simple at all on ESP8266.
Code: Select all
LINK build-GENERIC/firmware.elf
xtensa-lx106-elf-ld: build-GENERIC/firmware.elf section `.text' will not fit in region `iram1_0_seg'
xtensa-lx106-elf-ld: region `iram1_0_seg' overflowed by 64796 bytes
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
There are, for the moment, 3 issues when building MicroPython v1.12 + ulab on ESP8266 :
1) the one cited above, `iram1_0_seg' overflowed, I think the fix is to changing the flash layout and/or place ulab in "irom0_0_seg", i. e., changing the MicroPython source code;
2) "'out' may be used uninitialized error in function 'numerical_sort_helper' of 'numerical.c'", solved by this ulab PR#153;
3) some math functions are not visible in MicroPython on ESP8266, but they are used by ulab :
the solution seems to be modifying ".../ports/ESP8266/Makefile" to include this functions, i. e., changing the MicroPython source code.
1) the one cited above, `iram1_0_seg' overflowed, I think the fix is to changing the flash layout and/or place ulab in "irom0_0_seg", i. e., changing the MicroPython source code;
2) "'out' may be used uninitialized error in function 'numerical_sort_helper' of 'numerical.c'", solved by this ulab PR#153;
3) some math functions are not visible in MicroPython on ESP8266, but they are used by ulab :
Code: Select all
build-GENERIC/code/vectorise.o:(.text.vectorise_acosh+0x0): undefined reference to `acoshf'
build-GENERIC/code/vectorise.o:(.text.vectorise_asinh+0x0): undefined reference to `asinhf'
build-GENERIC/code/vectorise.o:(.text.vectorise_atanh+0x0): undefined reference to `atanhf'
build-GENERIC/code/vectorise.o:(.text.vectorise_erf+0x0): undefined reference to `erff'
build-GENERIC/code/vectorise.o:(.text.vectorise_erfc+0x0): undefined reference to `erfcf'
build-GENERIC/code/vectorise.o:(.text.vectorise_gamma+0x0): undefined reference to `tgammaf'
build-GENERIC/code/vectorise.o:(.text.vectorise_lgamma+0x0): undefined reference to `lgammaf'
build-GENERIC/code/vectorise.o:(.text.vectorise_log2+0x0): undefined reference to `log2f'
Last edited by rcolistete on Fri Jul 24, 2020 3:44 am, edited 2 times in total.
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
Which ESP8266 ? How much flash size ? ulab takes a lot of space (> 40 kB) for a ESP8266 module, so I think that ESP8266 with smaller flash sizes (512 kB or 1 MB) can't have ulab at all.
About ordering a Pyboard (D SF2 ?), I recommed, there are many Pyboard D features worth buying one.
It is needed to compile MicroPython source code to build a firmware with ulab included. Follow :
- MicroPython port to ESP8266 instructions, e. g., using the docker commands;
- MicroPython ulab instructions, i. e. :
* put ulab folder in the same folder where micropython folder is;
* add "USER_C_MODULES=../../../ulab all" in the end of make commands.
For example, on Linux terminal :
Code: Select all
[micropython]$ cd mpy-cross
[mpy-cross]$ docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk make
[mpy-cross]$ cd ../ports/esp8266
[esp8266]$ docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk make submodules
[esp8266]$ docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk make clean
[esp8266]$ docker run --rm -v $HOME:$HOME -u $UID -w $PWD larsks/esp-open-sdk make -j8 USER_C_MODULES=../../../ulab all
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
This issue (1) seems to be solved. Edit the file "../ports/esp8266/boards/esp8266_common.ld", and add the linercolistete wrote: ↑Thu Jul 23, 2020 10:30 pmThere are, for the moment, 3 issues when building MicroPython v1.12 + ulab on ESP8266 :
1) the one cited above, `iram1_0_seg' overflowed, I think the fix is to changing the flash layout and/or place ulab in "irom0_0_seg", i. e., changing the MicroPython source code;
Code: Select all
*code/*.o*(.literal* .text*) /* ulab module */
Code: Select all
*/frozen.o(.rodata.mp_frozen_content) /* frozen modules */
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
Solved the issue (3), succeeded building MicroPython firmware with ulab v0.54.0 for ESP8266 ! Yeah, the new version of ulab, release some hours ago.
I'll later (time to sleep...) describe the needed changes in '...esp8266/Makefile' and '...esp8266/mpconfigport.h', make a "README.md" in the repository below, etc.
Link for the firmware MicroPython daily build v1.12-652-gcaaaa2b1f, with ulab v0.54.0 + sp (single precision), for Generic ESP8266 module, 2M or more of flash.
Tried fft with 1024 and 2048 points, it works ! So preliminary benchmarks for fft of 1024 points, sp (single precision), without threads, :
- ESP8266 (WeMos D1 Mini) + MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-24 with ulab, default clock (80 MHz) :
* 57.509 ms;
- ESP8266 (WeMos D1 Mini) + MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-24 with ulab, overclock (160 MHz)
* 28.936 ms;
- Pyboard D SF2 + MicroPython v1.12-624-g57333f6ea on 2020-07-13 with ulab, default clock (120 MHz) :
* 1.873 ms;
- Pyboard D SF2 + MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-23 with ulab in QSPI flash, default clock (120 MHz) :
* 2.070 ms.
I'll later (time to sleep...) describe the needed changes in '...esp8266/Makefile' and '...esp8266/mpconfigport.h', make a "README.md" in the repository below, etc.
Link for the firmware MicroPython daily build v1.12-652-gcaaaa2b1f, with ulab v0.54.0 + sp (single precision), for Generic ESP8266 module, 2M or more of flash.
Code: Select all
MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-24; ESP module with ESP8266
Type "help()" for more information.
>>> import gc
>>> gc.collect()
>>> gc.mem_free()
36656
>>> import ulab as np
>>> np.__version__
'0.54.0'
- ESP8266 (WeMos D1 Mini) + MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-24 with ulab, default clock (80 MHz) :
* 57.509 ms;
- ESP8266 (WeMos D1 Mini) + MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-24 with ulab, overclock (160 MHz)
* 28.936 ms;
- Pyboard D SF2 + MicroPython v1.12-624-g57333f6ea on 2020-07-13 with ulab, default clock (120 MHz) :
* 1.873 ms;
- Pyboard D SF2 + MicroPython v1.12-652-gcaaaa2b1f-dirty on 2020-07-23 with ulab in QSPI flash, default clock (120 MHz) :
* 2.070 ms.
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: ulab for ESP8266
Those benchmarks are interesting, notably the minimal (~10%) penalty associated with using QSPI Flash on the Pyboard D.
I'm impressed you got it to run on an ESP8266.
I'm impressed you got it to run on an ESP8266.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: ulab for ESP8266
I am not against posting the instructions on https://github.com/v923z/micropython-ulab/. Raise an issue, if you want to do that!rcolistete wrote: ↑Fri Jul 24, 2020 3:07 amSolved the issue (3), succeeded building MicroPython firmware with ulab v0.54.0 for ESP8266 ! Yeah, the new version of ulab, release some hours ago.
I'll later (time to sleep...) describe the needed changes in '...esp8266/Makefile' and '...esp8266/mpconfigport.h', make a "README.md" in the repository below, etc.
The FFT takes place in place, so as long as you can fit your data (watch out for the imaginary part!) in RAM, you should be fine. I only use a couple of bytes for temporary variables. 4096 points might be a bit tight, though.
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
I've made a MicroPython branch (in my fork), "esp8266_compatible_ulab", with commit "ESP8266 compatible with ulab module : math functions and ulab in irom", and also submitted the MicroPython PR#6284 "ESP8266 made compatible with ulab module", which solves the issues (1) and (3) cited above, by changing files :rcolistete wrote: ↑Fri Jul 24, 2020 3:07 amSolved the issue (3), succeeded building MicroPython firmware with ulab v0.54.0 for ESP8266 ! Yeah, the new version of ulab, release some hours ago.
I'll later (time to sleep...) describe the needed changes in '...esp8266/Makefile' and '...esp8266/mpconfigport.h', make a "README.md" in the repository below, etc.
- "../ports/esp8266/boards/esp8266_common.ld" so ulab is moved from IRAM1 to IROM0, fixing the "`iram1_0_seg' overflowed" error;
- '.../esp8266/Makefile' and '.../esp8266/mpconfigport.h' to define some math functions used by ulab, fixing the building/linking errors "undefined reference to acoshf, asinhf, atanhf, erff, erfcf, tgammaf, lgammaf, log2f."
Issue (2) was merged in ulab master after PR#153, "numerical.c : fixed 'out' may be used uninitialized error in function…".
So, to compile ulab + MicroPython for ESP8266, use :
- ulab source code from July 24th 2020 onwards;
- temporary my MicroPython branch, "esp8266_compatible_ulab" while the PR#6284 isn't merged in MicroPython upstream/master.
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).
- rcolistete
- Posts: 352
- Joined: Thu Dec 31, 2015 3:12 pm
- Location: Brazil
- Contact:
Re: ulab for ESP8266
If the PR#6284 is merged, then the instructions to compile MicroPython for ESP8266 + ulab will be very simple : use ulab >= 0.54.0 & MicroPython newer than 'some date'.v923z wrote: ↑Fri Jul 24, 2020 2:16 pmI am not against posting the instructions on https://github.com/v923z/micropython-ulab/. Raise an issue, if you want to do that!
After FFT with 2048 points, the free RAM on ESP8266 (WeMos D1 Mini) & MicroPython v1.12 2020-07-24 is very small, just 2800 bytes (from 35968 bytes of free RAM after boot/restart).
So no way to run FFT with 4096 bytes on MicroPython for ESP8266.
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).