Page 1 of 2

Benchmark comparison of MicroPython boards

Posted: Wed Nov 16, 2016 11:50 am
by rcolistete
Some benchmark comparison of MicroPython boards :

- pystone_lomem (with '%g' changed to '%u' in lines 69 and 71 to be compatible with WiPy 1.0, download here), in pystones/second :
* Pyboard D SF2W with MicroPython v1.9.4 : 1,724 (120 MHz) / 2,762 (216 MHz);
* Pyboard v1.1 with MicroPython v1.8.6 : 1,754;
* Pyboard Lite v1.0 with MicroPython v1.8.6 : 1,048;
* WiPy 1.0 with MicroPython v1.8.2-103 : 659;
* ESP8266 with MicroPython v1.8.6-7 : 223 (80 MHz) / 415 (160 MHz);
* ESP32 with MicroPython v1.10 : 1,024;
* BBC Micro:bit with MicroPython v1.7.9 : not enough RAM to run;
* LoPy with MicroPython 0.9.6.b1 : 829;
* WiPy2 with MicroPython 0.9.6.b1 : 836;
* Teensy 3.2 with MicroPython v1.8.6 : 846;
* Teensy 3.5 with MicroPython v1.8.6 : 1,086;
* Teensy 3.6 with MicroPython v1.8.6 : 2,212;
* MAiX BiT with MicroPython v0.1.1 (@ 403 MHz) : 5,102;

- PerformanceTest using 'time.ticks_ms' (download here versions with 'pyb.millis', 'time.ticks_ms' and 'microbit.running_time()'), in number of times in 10s :
* Pyboard D SF2W @ 120 MHz with MicroPython v1.9.4, using 'pyb.millis' : 2,961,396 / 5,112,725 (native) / 7,201,474 (viper);
* Pyboard D SF2W @ 120 MHz with MicroPython v1.9.4 : 2,960,096;
* Pyboard D SF2W @ 216 MHz with MicroPython v1.9.4, using 'pyb.millis' : 4,717,962 (216 MHz) / 7,473,484 (native) / 11,930,233 (viper);
* Pyboard D SF2W @ 216 MHz with MicroPython v1.9.4 : 4,712,068;
* Pyboard v1.1 with MicroPython v1.8.6, using 'pyb.millis' : 2,815,970 / 4,610,390 (native) / 8,771,699 (viper);
* Pyboard v1.1 with MicroPython v1.8.6 : 2,783,122;
* Pyboard Lite v1.0 with MicroPython v1.8.6, using 'pyb.millis' : 1,738,503 / 2,686,262 (native) / 5,392,659 (viper);
* Pyboard Lite v1.0 with MicroPython v1.8.6 : 1,732,370;
* WiPy 1.0 with MicroPython v1.8.2-103 : 918,031;
* ESP8266 with MicroPython v1.8.6-7 : 171,145 (80 MHz) / 341,224 (160 MHz);
* BBC Micro:bit with MicroPython v1.7.9 : 168,826;
* LoPy with MicroPython 0.9.6.b1 : 651,394;
* WiPy2 with MicroPython 0.9.6.b1 : 680,488;
* Teensy 3.2 with MicroPython v1.8.6 : 1,041,445;
* Teensy 3.5 with MicroPython v1.8.6 : 1,204,193
* Teensy 3.6 with MicroPython v1.8.6 : 3,451,931;

- hsquare.py (type Ia supernova calculation with float point for 50,000 points), mean time for each calculation in us :
* Pyboard D SF2W with MicroPython v1.9.4 : 76 (120 MHz) / 46 (216 MHz);
* Pyboard v1.1 with MicroPython v1.8.6 : 83;
* Pyboard Lite v1.0 with MicroPython v1.8.6 : 139;
* WiPy 1.0 with MicroPython v1.8.2-103 : - (no float point support);
* ESP8266 with MicroPython v1.8.6-7 : 460 (80 MHz) / 239 (160 MHz);
* BBC Micro:bit with MicroPython v1.7.9 : 1,589;
* LoPy with MicroPython 0.9.6.b1 : 107;
* WiPy2 with MicroPython 0.9.6.b1 : 143;
* Teensy 3.2 with MicroPython v1.8.6 : 278;
* Teensy 3.5 with MicroPython v1.8.6 : 138
* Teensy 3.6 with MicroPython v1.8.6 : 73.

MicroPython release 0.9.6.b1 (for LoPy and WiPy 2) has default ESP32 frequency set as 160 MHz, instead of 80 MHz as before.

(21/11/2016 : added Teensy 3.x)
(27/11/2016 : updated LoPy and WiPy 2 with firmware 0.9.6.b1)
(11/03/2019) updated with Pyboard D, ESP32 and MAiX BiT

Re: Benchmark comparison of MicroPython boards

Posted: Thu Nov 17, 2016 8:17 am
by pythoncoder
Very useful information. But this "PerformanceTest" figure looks surprising
LoPy with MicroPython 0.9.4.b1 : 91,810

In other tests the LoPy is 6-7 times slower than the Pyboard 1.1 (standard code emitter), but in that test the factor is slower by a factor of 30. I wonder if there is some specific issue here.

Re: Benchmark comparison of MicroPython boards

Posted: Thu Nov 17, 2016 12:29 pm
by rcolistete
pythoncoder wrote:Very useful information. But this "PerformanceTest" figure looks surprising
LoPy with MicroPython 0.9.4.b1 : 91,810

In other tests the LoPy is 6-7 times slower than the Pyboard 1.1 (standard code emitter), but in that test the factor is slower by a factor of 30. I wonder if there is some specific issue here.
Pystone is based on Dhrystone, it is considered useful to measure the efficiency of the (Micro)Python implementation (e.g., if Python 2.7.2 is faster than Python 2.7.1), but it is not meant to exactly compare different hardwares.
PerformanceTest is very, very simple test. But is cited in many comparisons in our MicroPython community.
So it is natural to see different ratios (e.g. ESP8266/Pyboard 1.1) in speed of both benchmarks.

Re: Benchmark comparison of MicroPython boards

Posted: Sun Nov 20, 2016 6:36 pm
by rcolistete
Added BBC Micro:bit. Only 'pystone_lowmem' doesn't run due to lack of RAM. I've tried without comments and lower number of 'LOOPS', with no success.

Re: Benchmark comparison of MicroPython boards

Posted: Mon Nov 21, 2016 6:38 am
by rcolistete
Added Teensy 3.2, 3.5 and 3.6 benchmarks.

Re: Benchmark comparison of MicroPython boards

Posted: Sun Nov 27, 2016 4:10 am
by rcolistete
Updated the benchmark on LoPy and WiPy 2 using firmware 0.9.6.b1.

See historic evolution of these benchmarks for LoPy and WiPy 2.

Re: Benchmark comparison of MicroPython boards

Posted: Sat Nov 04, 2017 4:17 am
by cagiva
Sorry for reviving this old thread, but I was wondering where does the OpenMV M7 board rank within this list?

https://openmv.io/

Re: Benchmark comparison of MicroPython boards

Posted: Sat Nov 04, 2017 10:01 am
by fpp
Could be interesting to see the new Circuit Playground Express + CircuitPython 2.1.0 in there too !

Re: Benchmark comparison of MicroPython boards

Posted: Fri Dec 21, 2018 1:13 pm
by gpson
anyone tried this on esp32?

Re: Benchmark comparison of MicroPython boards

Posted: Sun Feb 10, 2019 2:56 pm
by ckuehnel
PyStone result for ESP32_Core_board_V2 w/ MicroPython v1.10:

>>> os.uname()
(sysname='esp32', nodename='esp32', release='1.10.0', version='v1.10-54-g43a894fb4 on 2019-02-07', machine='ESP32 module with ESP32')
>>> exec(open('pystone_lowmem.py').read())
Pystone(1.2) time for 500 passes = 488 ms
This machine benchmarks at 1024 pystones/sec