micropython-lib project: CPython backports

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

micropython-lib project: CPython backports

Post by pfalcon » Sat Oct 13, 2018 11:15 am

Recently, with the increasing number of packages which work both on MicroPython and CPython, I figured it might be a good idea to provide "MicroPython compatibility layer" for CPython. Put simply, you could "import utime" on CPython and use MicroPython's extended functions available in this module (and vice-versa, wouldn't be able to use functions not available in MicroPython).

The usecases enabled by this are:
  • "MicroPython first" module development. I.e. modules can be developed and optimized for MicroPython, the efficient Python language implementation, while still work on bloated implementations like CPython.
  • Reuse of the wealth of tools available for CPython, like debuggers, syntax/style checkers, coverage, etc.
  • Porting large MicroPython libs/apps to CPython. For example, uasyncio used to work on CPython by monkey-patching CPython's asyncio, but as it evolved, it became impractical/inefficient to do that. Implementing uasyncio dependencies, like utime, uselect, etc. is a better way.
  • Overall proof that despite differences in standard libraries, MicroPython and CPython are members of one big Python ecosystem.
For more information, see https://github.com/pfalcon/micropython- ... -backports

An example:

Code: Select all

$ pip3 install --user micropython-cpython-uhashlib
...
$ python3
...
>>> import uhashlib
>>> uhashlib.sha1(b"test").hexdigest()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'sha1' object has no attribute 'hexdigest'
# MicroPython's uhashlib doesn't have hexdigest(), use ubinascii.hexlify(.digest())
>>> uhashlib.sha1(b"test").digest()
b'\xa9J\x8f\xe5\xcc\xb1\x9b\xa6\x1cL\x08s\xd3\x91\xe9\x87\x98/\xbb\xd3'

To remind, the master micropython-lib repository is https://github.com/pfalcon/micropython-lib . Issues reports and patches adhering to Contributor Guidelines are welcome.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

User avatar
mattyt
Posts: 410
Joined: Mon Jan 23, 2017 6:39 am

Re: micropython-lib project: CPython backports

Post by mattyt » Sat Oct 13, 2018 11:32 pm

This is a very helpful contribution, thanks! I see it as being especially useful for unit testing where it can be much easier to run tests (at least first) on a PC.

Note that there's a similar effort in the CircuitPython space - adafruit_blinka. It's a little different since, as this guide explains, the main purpose is to allow CircuitPython code to run on Raspberry Pi, BeagleBone Blacks and generally Linux - but with support for GPIO, SPI and I2C. Maybe there's something in there that you might find useful?

In any case, thanks again @pfalcon!

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib project: CPython backports

Post by pfalcon » Fri Oct 19, 2018 6:17 am

mattyt wrote:
Sat Oct 13, 2018 11:32 pm
This is a very helpful contribution, thanks! I see it as being especially useful for unit testing where it can be much easier to run tests (at least first) on a PC.
Right. But actually, it's possible for quite some to do unit testing using MicroPython itself (with Unix port if doing on a PC). The corresponding module (unittest) is in micropython-lib: https://github.com/pfalcon/micropython- ... r/unittest . It surely doesn't have all the features CPython's counterpart has, but a few people contributed to it, which is a sign they actually find it useful.

For more complex tools, like code coverage or debuggers, running the code using CPython may be the most "low-hanging" way indeed.
Note that there's a similar effort in the CircuitPython space - adafruit_blinka. It's a little different since, as this guide explains, the main purpose is to allow CircuitPython code to run on Raspberry Pi, BeagleBone Blacks and generally Linux - but with support for GPIO, SPI and I2C. Maybe there's something in there that you might find useful?
Sure, it's great to see more MicroPython related projects around!
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib project: CPython backports

Post by pfalcon » Sun Dec 16, 2018 9:18 pm

There's noticeable progress with CPython module backports - now it's possible to run the "upip" package manager on CPython using them:

Code: Select all

$ pip3 install --user micropython-cpython-upip
Collecting micropython-cpython-upip
  Downloading https://files.pythonhosted.org/packages/23/86/7a98c5b0840d65ffbe6e71a0a9bd2ab93f5c938542b84a3c4824d2fc1553/micropython-cpython-upip-1.2.6.tar.gz
Collecting micropython-cpython-uerrno (from micropython-cpython-upip)
  Using cached https://files.pythonhosted.org/packages/55/65/06f724152cc383704b44a92a2d4dc8f62159f64c6bbb84e21f83b62aeb28/micropython-cpython-uerrno-0.1.tar.gz
Collecting micropython-cpython-ujson (from micropython-cpython-upip)
...
Successfully installed micropython-cpython-micropython-0.2 micropython-cpython-uerrno-0.1 micropython-cpython-uio-0.2 micropython-cpython-ujson-0.1 micropython-cpython-uos-0.1 micropython-cpython-upip-1.2.6 micropython-cpython-usocket-0.1 micropython-cpython-ussl-0.1 micropython-cpython-uzlib-0.1

$ python3 -m upip install -p install_dir notes-pico
Installing to: install_dir/
Warning: pypi.org SSL certificate is not validated
Installing notes-pico 0.8.9 from https://files.pythonhosted.org/packages/93/d3/f15081396c45b77709c9346215a45836130ae0367e801d17bb69a19751a3/notes-pico-0.8.9.tar.gz
Installing picoweb 1.5.1 from https://files.pythonhosted.org/packages/56/5c/4540e02d8be5b4dc3cc705a3047fb54c1d79e98f49b115290d7a2e736f5d/picoweb-1.5.1.tar.gz
Installing utemplate 1.2 from https://files.pythonhosted.org/packages/ce/c3/6ef0cc5c256c2192d8f6ff0175a6b6c3dff727fdb6ec1eb432fc0a2a97e9/utemplate-1.2.tar.gz
Installing micropython-logging 0.4.1 from https://files.pythonhosted.org/packages/46/93/7ce7c6632c59a47908b068d047d7b76c4505c6bf2b2dfa53b540fb1881bc/micropython-logging-0.4.1.tar.gz
Installing micropython-pkg_resources 0.2.1 from https://files.pythonhosted.org/packages/58/87/5acc262e0f642186891ef8d944bd727989000ce8d9263514f73b8feb7fdc/micropython-pkg_resources-0.2.1.tar.gz
Installing micropython-btreedb 0.3 from https://files.pythonhosted.org/packages/c4/03/b60fdda40235da6322ce11948a2576da7a2d8c4fe56d31f4440388e49a59/micropython-btreedb-0.3.tar.gz
Installing micropython-uasyncio 2.1.1 from https://files.pythonhosted.org/packages/0c/33/cbd1ee245621dab9952ebd1d3be840e6acd66e847b33ea88a7f015d2b952/micropython-uasyncio-2.1.1.tar.gz
Installing micropython-os 0.7.1 from https://files.pythonhosted.org/packages/a7/dc/9956b7ea7e4cc46e3d2a326225a2ea48fe66bcbea03f7ac0c84519445c4d/micropython-os-0.7.1.tar.gz
Installing micropython-uasyncio.core 2.0.1 from https://files.pythonhosted.org/packages/39/9f/619bbe7454419b72073e61b3d9abbc5dfeb54cc4062c2a2d21ad4984db85/micropython-uasyncio.core-2.0.1.tar.gz
Installing micropython-ffilib 0.1.3 from https://files.pythonhosted.org/packages/81/8a/5c3e6188384fbb57800a188d24582dc1a16b4fc8b538e0ba58aa443e3573/micropython-ffilib-0.1.3.tar.gz
Installing micropython-errno 0.1.5.1 from https://files.pythonhosted.org/packages/c5/bc/ac0b6e27a37f985ac5237ad1a6633a89a646cb23152c27fecca368abe811/micropython-errno-0.1.5.1.tar.gz
Installing micropython-stat 0.5.1 from https://files.pythonhosted.org/packages/cf/cf/fde35e109408fb94a5c00725e53608dada21a6b3eaa04dfd4189a8def2c1/micropython-stat-0.5.1.tar.gz

$ MICROPYPATH=install_dir micropython -m notes_pico.__main__
* Running on http://127.0.0.1:8081/
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib project: CPython backports

Post by pfalcon » Fri Feb 22, 2019 5:36 am

Initial backports of "ffi" and "uctypes" modules are available. Functionality is enough to e.g. run ffi_example.py from MicroPython distro:

Code: Select all

$ pip3 install --user micropython-cpython-ffi
$ pip3 install --user micropython-cpython-uctypes
$ cd micropython/examples/unix
$ python3 ffi_example.py 
libc: <ffi.DynMod object at 0x7f8d8992f0f0>

time: <ffi.Func object at 0x7f8d8992f978>
UNIX time is: 1550813462

perror before error: Success
errno object: <ffi.Var object at 0x7f8d8991d278>
errno value: 0
perror after error: Success

callback: <CFunctionType object at 0x7f8d8b22db38>
org string: bytearray(b'foobar')
cmp: b'o' b'o'
cmp: b'f' b'o'
cmp: b'a' b'r'
cmp: b'b' b'a'
cmp: b'b' b'r'
cmp: b'f' b'a'
cmp: b'f' b'b'
cmp: b'f' b'r'
cmp: b'o' b'r'
cmp: b'o' b'r'
qsort'ed string: bytearray(b'abfoor')
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib project: CPython backports

Post by pfalcon » Fri May 10, 2019 10:11 pm

The project gradually progress. The latest module released is https://pypi.org/project/micropython-cpython-ure/ (very bare, yeah).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

Post Reply