micropython-lib - "distributed" uPython standard library

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
pfalcon
Posts: 968
Joined: Fri Feb 28, 2014 2:05 pm

micropython-lib - "distributed" uPython standard library

Post by pfalcon » Sat Apr 26, 2014 1:40 pm

micropython-lib is a community project to build a non-monolithic and lean standard Python library for MicroPython.

Following aims and requirements are set forth:
  1. To coordinate efforts for implementing standard library modules for MicroPython.
  2. Be testbed and proof of concept for implementing Python library not as a big monolithic blob unalienably installed with the interpreter, but based on the idea of separately distributed and installable individual modules - just in the same way as 3rd-party modules are distributed and used.
  3. To bootstrap prototyping support for using standard Python package infrastructure with MicroPython, like pip package manager, PyPI package repository, etc.
  4. To further develop "unix" port to be useful, and be testbed for idea of implementing as much as possible system integration modules in Python (using FFI (Foreign Function Interface) module), instead of C.
  5. To be implemented in such way as to be usable for MCU/baremetal ports of MicroPython too.
Initial RFC was in the ticket #405. Since then, number of modules were published on PyPi, from where they are installable using pip-micropython tool from the tools/ folder of the main MicroPython repository. Here's the list of currently published packages: https://pypi.python.org/pypi?%3Aaction= ... mit=search
Last edited by pfalcon on Sat Apr 26, 2014 1:55 pm, edited 2 times in total.

pfalcon
Posts: 968
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib - "distributed" uPython standard library

Post by pfalcon » Sat Apr 26, 2014 1:41 pm

(reserved message)

pfalcon
Posts: 968
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib - "distributed" uPython standard library

Post by pfalcon » Sat Apr 26, 2014 1:56 pm

(reserved message)

pfalcon
Posts: 968
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib - "distributed" uPython standard library

Post by pfalcon » Sat May 24, 2014 2:51 pm

Some update report: There's now 230 commits in https://github.com/micropython/micropython-lib . 60 packaged published on PyPI: https://pypi.python.org/pypi?%3Aaction= ... mit=search
(many are dummy packages just to get imports not fail).

I now try to concentrate on making "bloated" CPython packages run on MicroPython with minimal changes. The bloat they bring is not exciting at all (for example, http.client requires 600K heap just to import, so no chance it'll run on MCU), but it's very important to get "real" Python code to run, to be able that we provide *Python* implementation, and not some weird Python-like language instead.

Testing of published packages is very much needed, consider helping with that.
Last edited by pfalcon on Mon Jun 09, 2014 8:04 pm, edited 1 time in total.

User avatar
kfricke
Posts: 334
Joined: Mon May 05, 2014 9:13 am
Location: Germany

Re: micropython-lib - "distributed" uPython standard library

Post by kfricke » Sun May 25, 2014 1:29 pm

Hi! I am personally interested in the hashlib module, which is a dummy for some good reasons. Knowing that this topic and all it's usages and dependencies is a big one, does anyone have a good hint on how to feasibly implement at least the very basic hashing functions (whith focus on sha256 and the other remaining secure hashing algorithms)? I did start to use the SHA256 module from PyPy (linked here: http://stackoverflow.com/questions/7321 ... -in-python) and slowsha (https://code.google.com/p/slowsha/) also did cause a hard error on my pyboard :-(

Any hints towards another good starting point are appreciated, thanks!

pfalcon
Posts: 968
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib - "distributed" uPython standard library

Post by pfalcon » Wed May 28, 2014 8:06 pm

which is a dummy for some good reasons
Yeah, and those reasons are that nobody needed it yet - at all, or that much as to work on it ;-). And then, because it's not clear now to approach it - common sense says that hashing should be implemented in C, but besides finding good license-compatible code, that means also bloating the uPy binaries with functions which only fractions of users will need (especially on MCU). Those are not unsolvable problems, but require more than one head to even decide on approach.

So all in all, I support an idea to start with Python-only implementation. And to get that running, I always suggest using unix port first, because it's more comfortable to use than a board.

I had a look at https://code.google.com/p/slowsha/ , it surely cannot run directly, I was able to workaround few smaller issues, but then I hit https://github.com/micropython/micropython/issues/627 . As can be seen, @dpgeorge fixed it today, so interested parties can start looking for further issues ;-).

pfalcon
Posts: 968
Joined: Fri Feb 28, 2014 2:05 pm

Re: micropython-lib - "distributed" uPython standard library

Post by pfalcon » Fri Jun 06, 2014 8:43 pm

After few attempts on how to manage metadata for so many modules, I wrote a script to generate setup.py from concise description: https://github.com/micropython/micropyt ... etadata.py . And now it runs with MicroPython too! It took porting glob module from CPython and implementing "%(key)s" % {} formatting.

salimfadhley
Posts: 22
Joined: Thu Jun 19, 2014 10:18 pm

Re: micropython-lib - "distributed" uPython standard library

Post by salimfadhley » Sun Oct 19, 2014 1:36 am

Where do I find the ffi module which is required by the _libc package?

blmorris
Posts: 344
Joined: Fri May 02, 2014 3:43 pm
Location: Massachusetts, USA

Re: micropython-lib - "distributed" uPython standard library

Post by blmorris » Sun Oct 19, 2014 2:12 am

salimfadhley wrote:Where do I find the ffi module which is required by the _libc package?
Depending on your OS / distribution you probably need to install it. On OSX I needed to install 'libffi', I assume that it will be called something similar in any given Linux distro.
-Bryan

Turbinenreiter
Posts: 269
Joined: Sun May 04, 2014 8:54 am

Re: micropython-lib - "distributed" uPython standard library

Post by Turbinenreiter » Sat Oct 25, 2014 9:58 pm

How should we go about lib for the pyboard? A lot of the modules don't work on the pyboard, just as an example, pystone doesn't work, but it's an easy fix - I just replaced time.clock with pyb.millis/1000.

btw, the pyboard apparently does 1688 pysteons/second. whatever that means.

Code: Select all

Pystone(1.1) time for 50000 passes = 29.6147
This machine benchmarks at 1688.35 pystones/second

Post Reply