MicroPython version 1.19 released

Announcements and news related to MicroPython.
Post Reply
Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

MicroPython version 1.19 released

Post by Damien » Thu Jun 16, 2022 6:29 am

Dear community,

After a bit of a delay, we are pleased to announce the next release of MicroPython, v1.19! There were 617 commits since the last release on 17th January 2022, so there were about 4.1 commits per day since then (it was 2.5 commits per day for the previous release).

Source code and firmware can be downloaded from https://micropython.org/download/

The full change log can be found at https://micropython.org/resources/micro ... ngeLog.txt

A summary of the changes follows.

Enjoy!

v1.19: New mpy 6 file format with improved bytecode, addition of renesas-ra port

In this release of MicroPython the .mpy file format has been significantly reworked and bumped to version 6. The new format now contains data and bytecode organised to be mostly static and ready to run in-place. To achieve this, qstr references from bytecode (and native machine code) now go through a lookup table, to convert from local qstr number in the module to global qstr number in the runtime that the module executes within. This means bytecode (and most native code) does not need rewriting when it is imported. Along with this change, the jump opcodes have been optimised to emit 1-byte jump offsets when possible, which gives a good reduction in bytecode size, saving RAM and reducing the size of .mpy files.

The change to the bytecode means that code constants (like long strings, bytes objects and floats) are now shared and de-duplicated across the whole module, rather than each code object having a separate constant table. The parser can also now turn a tuple of constants into a constant object, so the tuple doesn't need to be built at runtime each time it is used. Furthermore, any constant object can now be used in "X = const(obj)" statements.

With these bytecode changes, .mpy files see around a 10% reduction in size and frozen .mpy code sees about a 5% size reduction. Heap memory use after import is also down by around 5% (more savings are seen if the code uses lots of constants). Due to the qstr indirection performance of the VM is slightly reduced on some benchmarks by a few percent, but the benefits of smaller code and reduced RAM usage are deemed to outweigh this. See commit f2040bfc7ee033e48acef9f289790f3b4e6b74e5 for more details.

In the runtime, support has been added for multiple *args and **args in a function call, following PEP 448. Built-in modules can now have (optional) mutable attributes, and this is used to implement sys.ps1/sys.ps2 which allow the user to customise the REPL prompt. This can be useful, for example, to configure a different REPL for different boards so they can be easily distinguished. The sys module has also had the MicroPython git version and build date appended to sys.version, sys.implementation._machine has been added, and sys.implementation.mpy renamed to sys.implementation._mpy. The sys module can now be used to fully identify the MicroPython version and target it is running on (and the platform module provides additional information when it's available).

The MP_REGISTER_MODULE macro is now used to register all built-in C modules and the third argument to this macro has been removed. Any user-C-modules will need to be updated. Also support for MICROPY_PORT_BUILTIN_MODULES is removed because this is no longer needed, and should be replaced by usage of MP_REGISTER_MODULE (see commit 0a92469c10542d57ef0ea6b6cf2ed0cae9491dbe for example).

The MP_REGISTER_MODULE macro is now used to register all built-in C modules and the third argument to this macro has been removed. Also support for MICROPY_PORT_BUILTIN_MODULES is removed because this is no longer needed.

mpy-cross now supports the -march=armv6m option, which can be used with the rp2 port. The -mno-unicode has been removed.

The littlefs2 library has been updated to v2.5.0 which includes some performance improvements and minor fixes.

At the REPL, a tab will now insert an indent when it follows whitespace, and auto-indent is automatically disabled if a space or tab follows an auto-indent. This means that Python code can be pasted into the standard REPL without messing up the indentation.

mpremote is now at version 0.3.0, with a progress indicator shown when copying large files, and recursive copy fixed on Windows. It also has the following commands added: --help, --version, version, resume, soft-reset, umount.

The esp32 port has a new machine.ADCBlock class and updated ADC class. Fixes have been made to ADC2 channel mapping, low PWM frequencies, UART initialisation of unspecified parameters, and RMT channel on S2/S3/C3 variants. The Pin.PULL_HOLD constant has been replaced by a "hold" keyword argument to the Pic constructor, and it's now possible to configure pin drive strength. I2C write-then-read transfers have been optimised to be faster. WLAN.config() now supports setting/getting the txpower. New boards include LOLIN C3 MINI ESP32-C3, LilyGO LoRa32, and UM ESP32-S3 based boards.

The mimxrt port has updated the nxp_driver to v2.10 and fixed USB CDC RX handling to not block. Pin drive constants have been renamed to DRIVE_x to match other ports, and the machine.I2S class has been added. New boards include Olimex RT1010 and MIMXRT1015_EVK.

The nrf port now includes uasyncio in the default board manifest, and additional features have been enabled on nrf52840 and nrf9160 boards to match the features on stm32, rp2 and esp32. This includes enabling the json, re, zlib and framebuf modules.

There is a new renesas-ra port which targets Renesas RA4M1, RA4W1, RA6M1 and RA6M2 MCUs, with board definitions for evaluation kits for these MCUs, along with RA4M1 clicker. This port currently supports Pin, ADC, SoftI2C, I2C, SoftSPI, SPI, UART, Timer and RTC classes in the machine module, along with the VFS and an internal flash filesystem.

The rp2 port now has optional USB MSC support, and optional dupterm support. The machine.UART class has init and deinit methods added, and the ucryptolib module is now available. The lwIP network stack has been integrated, along with mbedtls for the ssl module, and these can be used with the new Wiznet driver to provide Ethernet support. The Wiznet W5100S-EVB-Pico board has been added which makes use of this driver. The day-of-week value reported by time.localtime() has been fixed, memory corruption when a thread is created on core1 has been fixed, and the UART now uses a mutex when reading/writing to prevent character duplication in code that uses both cores.

The stm32 port has added support for G0, G4 and WL MCUs. The new boards for these MCUs are NUCLEO_G0B1RE, NUCLEO_G474RE and NUCLEO_WL55. The ARDUINO_PORTENTA_H7 board has also been added, with full WiFi and BLE support. The rfcore on WB55 MCUs has been improved by removing an extra layer of buffering, and the restriction on the use of address resolution has been removed, meaning that the rfcore WS firmware should be updated to v1.12.0 or later; see commit 26b1d31eda292d9cd5cfc68f14ce8055256fade8. The board-level configuration of mboot has been significantly improved to allow more control over system clocks, more hooks into the start up process, and a hook for state change. And the CRC32 of DFU files is now verified before writing them.

The pyb.CAN class has been improved to support FD frames and buses with mixed classic/extended-ID nodes, allowing extended IDs to be specified on a per-message basis. CAN.recv also now returns a value indicating if the received message had an extended ID or not. And CAN.initfilterbanks has been removed, replaced with the num_filter_banks keyword argument to CAN.init. These are all breaking changes to the pyb.CAN class API and code will need to be updated accordingly; see commit 5cdf9645711cc12b45e602ef5795c33895116fc4 for details.

The unix port now shows compiler information in the REPL banner to make it consistent with how bare-metal ports work, and the "-X realtime" command-line option has been added for macOS to get improved timing. All variants have switched to use the VFS subsystem, and use the VfsPosix class for mounting the host's filesystem. The os module has been consolidated with all other ports and the minimal and standard variants gain the unlink, chdir, getcwd and listdir functions.

The windows port has also switched to the VFS subsystem and VfsPosix. It adds support for micropython.schedule and uasyncio on the dev variant. And BCryptGenRandom has been used to implement os.urandom.

The zephyr port upgraded Zephyr to v3.0.0.

The change in code size since the previous release for various ports is (absolute and percentage change in the text section):

Code: Select all

       bare-arm:   +792  +1.394%
    minimal x86:  +2091  +1.286%
       unix x64: +10231  +1.995%
    unix nanbox:  +9777  +2.162%
          stm32:   +184  +0.047% PYBV10
         cc3200:   +904  +0.493%
        esp8266:  -1240  -0.177% GENERIC
          esp32:  +6464  +0.425% GENERIC
            nrf: +11288  +6.472% pca10040
            rp2: +14432  +2.929% PICO
           samd:  +1352  +0.970% ADAFRUIT_ITSYBITSY_M4_EXPRESS
The changes that dominate these numbers are:
  • bare-arm, minimal, cc3200, samd: reworked bytecode and mpy format, optimisation of jump opcodes to use 1-byte offsets, de-duplication of constants at the module level, and PEP 448 support
  • unix: switching to use VFS, and new mpy format and related changes
  • esp8266: removal of esp.info and machine.WDT.deinit
  • esp32: addition of machine.ADCBlock, and new mpy format changes
  • nrf: enabling uasyncio, enabling MICROPY_PY_IO
  • rp2: enabling ucryptolib with mbedtls
Note that the stm32 port's size remained roughly unchanged. This is because it has a decent amount of frozen Python code and the reduction in frozen code size due to the reworked bytecode has offset the increase in runtime/VM size due to this change. Ports like bare-arm, minimal, cc3200 and samd do not have any frozen code so only see the increase in runtime/VM size. Production-level projects that have a significant amount of frozen code have seen around a 4% overall reduction in their firmware size.

Thanks to everyone who contributed to this release: Algy Tynan, Andrew Leech, Andrew Scheller, Artyom Skrobov, Asensio Lorenzo Sempere, Bradley Wogsland, Cem Eliguzel, Chris Wilson, Christian Decker, Christian Zietz, Christophe Priouzeau, Clayton Mills, Damien George, Daniel Jour, Daniël van de Giessen, Dave Hylands, David Lechner, Emil Kondayan, Herwin Grobben, iabdalkader, IhorNehrutsa, Jan, Jared Hancock, jason, Jeff Epler, Jeremy Herbert, Jim Mussared, Jon Bjarni Bjarnason, Jonathan Hogg, Jos Verlinde, Kattni Rembor, Lars Kellogg-Stedman, Luiz Brandao, marcidy, Martin Fischer, Maureen Helm, Meriç SARIIŞIK, Michael Himing, Michael O'Cleirigh, MikeTeachman, Peter D. Gray, Peter Hinch, Peter Züger, Phil Howard, Philipp Ebensberger, Rob Knegjens, robert-hh, Sean Coates, Seon Rozenblum, Stephane Smith, stijn, Takeo Takahashi, ubi de feo, Waterlens, wemos, YoungJoon Chun, Yukai Li.

Guustaaf
Posts: 1
Joined: Tue Jun 07, 2022 11:32 pm

Re: MicroPython version 1.19 released

Post by Guustaaf » Thu Jun 16, 2022 7:11 am

Thank you so much everyone who contributed to this!

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: MicroPython version 1.19 released

Post by Damien » Fri Jun 17, 2022 3:09 am

A bug was found in v1.19 on esp32, the machine.SoftI2C.readfrom_mem() method does not work properly.

A new release v1.19.1 has now been made with a fix for this. Please use v1.19.1 instead of v1.19 (but it only matters for esp32 boards).

kindmartin
Posts: 20
Joined: Sun Apr 26, 2020 7:30 am

Re: MicroPython version 1.19 released

Post by kindmartin » Mon Jun 20, 2022 5:58 am

Happy to read how live is this project, it had passed a time from that kickstarter days and in the while I got into basic micropython. thanks to all the team that makes our days more easy with each update.

M

User avatar
RobH
Posts: 88
Joined: Fri Mar 23, 2018 3:37 pm
Location: Netherlands
Contact:

Re: MicroPython version 1.19 released

Post by RobH » Tue Jun 21, 2022 8:22 am

@Damien. Since the renaming from 1.19 to 1.19.1 I see no more nightly builds...

User avatar
jimmo
Posts: 2645
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: MicroPython version 1.19 released

Post by jimmo » Tue Jun 21, 2022 9:34 am

RobH wrote:
Tue Jun 21, 2022 8:22 am
@Damien. Since the renaming from 1.19 to 1.19.1 I see no more nightly builds...
This was due to investigating an issue with the esp32 build with idf 4.4, which is now resolved. The nightlies should start appearing again soon. Thanks for letting us know!

User avatar
RobH
Posts: 88
Joined: Fri Mar 23, 2018 3:37 pm
Location: Netherlands
Contact:

Re: MicroPython version 1.19 released

Post by RobH » Tue Jun 21, 2022 2:37 pm

@jimmo: Thanks, found the unstables again!

User avatar
RobH
Posts: 88
Joined: Fri Mar 23, 2018 3:37 pm
Location: Netherlands
Contact:

Re: MicroPython version 1.19 released

Post by RobH » Sun Jun 26, 2022 6:02 pm

I'm afraid there is still (or again) something wrong with the nightly build procedure. I've seen several source updates since 1.19.1-33, but no more nightly builds for the boards I regularly refresh (a.o. ESP32, PYB11).

User avatar
jimmo
Posts: 2645
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: MicroPython version 1.19 released

Post by jimmo » Mon Jun 27, 2022 12:53 pm

RobH wrote:
Sun Jun 26, 2022 6:02 pm
I'm afraid there is still (or again) something wrong with the nightly build procedure. I've seen several source updates since 1.19.1-33, but no more nightly builds for the boards I regularly refresh (a.o. ESP32, PYB11).
Thanks, will investigate ASAP.

User avatar
RobH
Posts: 88
Joined: Fri Mar 23, 2018 3:37 pm
Location: Netherlands
Contact:

Re: MicroPython version 1.19 released

Post by RobH » Mon Jun 27, 2022 5:55 pm

Thnks, found 1.19.1-61 builds!

Post Reply