asyncio-like cooperative multitasking framework for uPy

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Thu Nov 13, 2014 7:29 pm

fma wrote:As only uasyncio.core works under pyb, for now, how would you add support for simple non-blocking reading on a pyb serial line, for example?

The goal would be to have a coroutine monitoring a serial line used to send commands to pyboad, while it is doing some work in background...


Well, pyboard port also has "select" module now which Poll object as needed for this to work. So, it shouldn't be overly hard to port existing Linux port version to pyboard. But what I'd aim for is to try to support both with a single platform-independent uasyncio code, and that would be harder apparently, as surely there're a lot of differencies how Poll works in Linux and on PyBoard (for example, I recently switched it to one-short epoll support, which appears to be most efficient and robust way to implement needed semantics, but one-shot mode is not available in pyboard Poll object).

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Thu Nov 13, 2014 7:34 pm

fma wrote:Ok, I successfully subclassed EventLoop, and overrided time() and wait() methods:

Code: Select all

class MyEventLoop(asyncio.EventLoop):
    def time(self):
        return pyb.millis()

    def wait(self, delay):
        start = pyb.millis()
        while pyb.elapsed_millis(start) < delay:
            pyb.delay(10)





FYI, the way I was thinking about it was to use .millis() internally, but still provide external interface of (fractional) seconds. Otherwise, it automatically becomes incompatible with standard asyncio interface. But that requires working with floating-point values, and while pyboard can do that efficiently, some other boards won't. So, I'm not sure what to do about it, need more people look into that.

User avatar
dhylands
Posts: 2180
Joined: Mon Jan 06, 2014 6:08 pm
Location: Shuswap, BC, Canada
Contact:

Re: asyncio-like cooperative multitasking framework for uPy

Postby dhylands » Fri Nov 14, 2014 12:58 am

I think that you have to draw a line somewhere.

You either need to decide to say that 32-bit float is the minimum needed or break something.

I think that if your processor is so "small" that it can't support floats, then you're probably doing something really custom and probably not using the std libs anyways since its so small.

Just my thoughts...

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Sat Nov 15, 2014 7:57 pm

dhylands wrote:I think that you have to draw a line somewhere.

You either need to decide to say that 32-bit float is the minimum needed or break something. I think that if your processor is so "small" that it can't support floats, then you're probably doing something really custom and probably not using the std libs anyways since its so small.



CPython people thought something like that (or didn't think in enough detail), and with current CPython stdlib API essentially said "64-bit float is the minimum". It doesn't make sense to repeat that and place arbitrary restrictions. Much better would be to design APIs for *Python* (not just our implementation) which supports "graceful degradation" or purpose-focused operation.


That said, I guess short-term approach should be indeed to make uasyncio.pyboard and override there implementation for PyBoard specifically, and leave other ports for when they will really exist.

User avatar
dhylands
Posts: 2180
Joined: Mon Jan 06, 2014 6:08 pm
Location: Shuswap, BC, Canada
Contact:

Re: asyncio-like cooperative multitasking framework for uPy

Postby dhylands » Sun Nov 16, 2014 3:29 am

Well, you have to decide if you want to do it the "right" way and provide something more pythonic, or do it the CPython way and maintain compatability with cpython.

You might be able to do it both ways (although it probably depends on the exact API in question).

For example, you could have the existing API use float and return float, and perhaps be useful on micropython.

You could then introduce a new API which used some more appropriate type, and works well on micropython. As long as the new API can be supported in CPython by say creating a wrapper then I think that would be the best way to move forward.

User avatar
fma
Posts: 156
Joined: Wed Jan 01, 2014 5:38 pm
Location: France

Re: asyncio-like cooperative multitasking framework for uPy

Postby fma » Fri Nov 21, 2014 1:04 pm

As micropython IO object like Uart, I2C, SPI... does not have a fileno() method, to be used by the EventLoop class when using IORead/IOWrite, and as select/poller is not avaible, I'm trying to adapt uasyncio. But it is not clear how this should be done.

Any suggestion?
Frédéric

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Sat Nov 22, 2014 10:31 am

fma wrote:As micropython IO object like Uart, I2C, SPI... does not have a fileno() method, to be used by the EventLoop class when using IORead/IOWrite, and as select/poller is not avaible


Why not available? http://docs.micropython.org/en/latest/l ... elect.html

As I mentioned (elsewhere?), the right way would be to start uasyncio.pyboard module, subclass uasyncio.core event loop, and implement needed methods using API provided by PyBoard. I these months don't even have PyBoard on my hands (and still busy with unix port), so can't help much. But I tried to get more people interested who usually work with PyBoard (dhylands, pythoncoder). Here's another recent discovery: https://github.com/nvbn/microasync/issues/1

pfalcon
Posts: 893
Joined: Fri Feb 28, 2014 2:05 pm

Re: asyncio-like cooperative multitasking framework for uPy

Postby pfalcon » Sat Nov 22, 2014 10:48 am

And well, this little "announcement" got pretty backlogged, but let me do it now:

We have a small win in uasyncio-to-asyncio compatibility: CPython 3.4.2 added loop.create_task() method for straightforward (and thus efficient) scheduling of (new) coroutines. I used to criticize asyncio for not having such straightforward API and instead having it done in arguably confusing way (see python-tulip thread linked from the first post), so glad the improvements are coming.

I actually discovered that when I finished compatibility implementation of original asyncio calls to do scheduling (top-level asyncio.Task() and asyncio.async()), so they're implemented too, but we probably can drop them now to save on bytecode ;-).

Here's the thread with my "thanks" in upstream mailing list: https://groups.google.com/forum/#!topic ... mU4_qyPVQM . It also smoothly moves into my trying to argue adding even more natural form of coroutine scheduling: just yielding a new corouitine - the event loop then just adds it to run queue and it starts running. I got immediate "No" from BDFL on that, but I keep thinking it's rather intuitive and pythonic, and don't have intention to drop it from uasyncio.

What do you think - is Guido right that using "yield" for that may be a source of (user-confusion) errors? I argue that there may be bigger sources of confusion, and that again, "yield coro()" to start a new coroutine is pretty intuitive - see the thread above for details.

But if you're conservative, just use loop.create_task() and your code will be compatible with CPython asyncio. (But you'll need to pass around event loop instance, or make it globally available, which is the reason I think "yield" approach is better).

User avatar
fma
Posts: 156
Joined: Wed Jan 01, 2014 5:38 pm
Location: France

Re: asyncio-like cooperative multitasking framework for uPy

Postby fma » Sat Nov 22, 2014 12:04 pm

pfalcon wrote:
fma wrote:As micropython IO object like Uart, I2C, SPI... does not have a fileno() method, to be used by the EventLoop class when using IORead/IOWrite, and as select/poller is not avaible


Why not available? http://docs.micropython.org/en/latest/l ... elect.html

Ouch, I missed that! Thanks for pointing.
Frédéric

User avatar
fma
Posts: 156
Joined: Wed Jan 01, 2014 5:38 pm
Location: France

Re: asyncio-like cooperative multitasking framework for uPy

Postby fma » Mon Nov 24, 2014 8:08 am

pfalcon wrote:As I mentioned (elsewhere?), the right way would be to start uasyncio.pyboard module, subclass uasyncio.core event loop, and implement needed methods using API provided by PyBoard.

Ok, I started to implement uasyncio.pyboard, but I'm a bit lost.

uasyncio.core relies on the fact that (unix) poller uses a file descriptor, and so calls polled object fileno() method. But UART, which is said to be pollable, does not have such method. So, I guess I need to overwrite the entire run_forever() method, replacing arg.fileno() with just arg. Am I write?

But then, what if I want to poll both standard objects (socket, file...) which do have fileno() method and needs to be polled on their fd, and specific pyboard objects, like UART, which don't have fd?

Also, I think that common part of uasyncio should be in the __init__.py file, and specific code should be in something like unix.py, pyb.py...
Frédéric


Return to “Programs, Libraries and Tools”

Who is online

Users browsing this forum: No registered users and 1 guest