asyncio-like cooperative multitasking framework for uPy

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Thu Dec 04, 2014 12:30 am

fma wrote:uasyncio.core relies on the fact that (unix) poller uses a file descriptor, and so calls polled object fileno() method. But UART, which is said to be pollable, does not have such method. So, I guess I need to overwrite the entire run_forever() method, replacing arg.fileno() with just arg. Am I write?

But then, what if I want to poll both standard objects (socket, file...) which do have fileno() method and needs to be polled on their fd, and specific pyboard objects, like UART, which don't have fd?


Doing such analysis and proposing (concurrent) solution(s) to the situation is the essence of programming.

So, based on the analysis above, I can propose different ways to solve it:

1. We implement function inlining in uPy compiler so we're not afraid of functional abstraction. (Because currently, function calls are expensive, and that's why uasyncio main loop written as pretty big flat code). This is the most productive solution, especially if someone volunteers to implement it.
2. We add "if" or exception handling to call .fileno() only of it exists, and use arg as is otherwise. To this, I don't agree, as it hurts performance ;-).
3. We say that UART should have .fileno() method and that it's essentially part of stream interface, so any object which can be used as stream should implement it.
4. We change function signature of .add_reader() and friends - instead of accepting "file descriptor", as CPython asyncio mandates, it would accept "file-like object", and .add_reader() can decide what to do with it (it will be likely overridden for particular implementation anyway).


4 is the most compromise variant, which would be good choice for a winner. But it adds another discrepancy with CPython asyncio ;-). So, I'd like to have better discussion of this, in particular consider burdening @dpgeorge with choice 3 ;-). And as he doesn't appear here too often, that would warrant a github ticket with reference back to this post.

nvbn
Posts: 1
Joined: Sun Dec 21, 2014 1:12 am

Re: asyncio-like cooperative multitasking framework for uPy

Postby nvbn » Mon Dec 22, 2014 6:48 pm

Hi all, I wrote a little article about using uasyncio on pyboard with csp, producer/consumer and coroutines - https://nvbn.github.io/2014/12/21/pyboard-uasyncio-csp/
I think it could be interesting for whom who in this thread.

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Fri Jan 02, 2015 10:55 pm

nvbn, thanks much for this article, trying uasyncio on PyBoard and contributing the queue code! The usage you show is pretty advanced, I'm glad that both PyBoard and uasyncio are up to it.

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Fri Jan 02, 2015 11:02 pm

Some updates from me: I was looking into validating socket i/o handling, and improving compatibility with CPython asyncio implementation, and providing CPython extension module for features where uasyncio deviates from asyncio. The latter is here: https://github.com/micropython/micropyt ... n-uasyncio

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Fri Apr 17, 2015 10:44 pm

FYI, a PEP was posted to add async/await to Python: https://mail.python.org/pipermail/pytho ... 33007.html . So far, response was positive.

Damien
Site Admin
Posts: 502
Joined: Mon Dec 09, 2013 5:02 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby Damien » Sat Apr 18, 2015 10:19 pm

pfalcon wrote:FYI, a PEP was posted to add async/await to Python: https://mail.python.org/pipermail/pytho ... 33007.html . So far, response was positive.

This is interesting, especially since it comes with a change to the grammar. Initially I was skeptical but perhaps it's a good step forward. You've got to progress like this to stay competitive in the language arena.

So does it partially replace asyncio module? Will they keep yield from (what would be the point of it)?

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Mon Apr 20, 2015 10:11 am

Yes, as PEP says, having "async def" and "await" is just a syntactic sugar, underlying implementation is still based on generators and yield from, which are of course stay as is. It doesn't replace asyncio either, because it's a coroutine scheduler, you'll still need it to schedule "async def" coroutines (but otherwise, great share of asyncio is stuff not related to coroutines, but deal with other cooperative concurrency patterns, and those are "replaced" by normal coroutines, and uasyncio for example don't implement all those Future's and stuff).

As PEP says, the idea behind this is to make coroutine writing is more obvious, clean, and less error-prone. That's indeed good and most of the changes reside in compiler. Unfortunately, they also affect runtime type model by adding "async" vs "non-async" special method dichotomy. The PEP admits it was possible to avoid that, but follows with saying that its topic is to make async programming obvious and easy, and then it's simple rule of thumb: if you use non-async machinery, use __*__ methods, if you use async - __a*__. Unfortunately, for us it means more bloat than needed.

Damien
Site Admin
Posts: 502
Joined: Mon Dec 09, 2013 5:02 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby Damien » Mon Apr 20, 2015 11:48 am

I'll have to read the PEP fully.

Maybe we can implement the basics of this PEP as it stands so we can give quality feedback and perhaps drive it to a more lean standard?

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Mon Apr 20, 2015 3:31 pm

We definitely can start with "reading" part ;-). And worth keeping an eye on it, but first day there was no "don't do that!" response, and BDFL is favorable of it, so I guess it'll be accepted, and apparently even into 3.5. Implementing it would be nice, if would you have time for that soon - it's mostly parser/compiler work, at least for the 1st part, adding "async def"/"await". "async for"/"async with" is 2nd part, where majority of newly added special methods lie, and after reading PEP, I don't think it would be a productive argument "you should reuse existing special methods, because adding more of them makes it problematic to run it on microcontrollers".

So, implementing it, or at least thinking how yo make it possible, would be definitely a good thing. My immediate priority (and long backlogged :-( ) would be to "announce" uasyncio as it is now - I kinda gave up on approach "make it as complete re: asyncio compatibility first" approach.

altasoul
Posts: 1
Joined: Sat May 30, 2015 12:34 am

Re: asyncio-like cooperative multitasking framework for uPy

Postby altasoul » Mon Jul 27, 2015 9:42 pm

BDFL accepts PEP 492 (async/await):
In order to save myself a major headache I'm hereby accepting PEP 492.

I've been following Yury's efforts carefully and I am fully confident that
we're doing the right thing here. There is only so much effort we can put
into clarifying terminology and explaining coroutines.

https://mail.python.org/pipermail/python-dev/2015-May/139844.html


Return to “Programs, Libraries and Tools”

Who is online

Users browsing this forum: No registered users and 1 guest