uasyncio - asyncio-like cooperative multitasking framework for uPy
Re: asyncio-like cooperative multitasking framework for uPy
This may be a silly question, but what's the best way to install uasyncio.core (or indeed any micropython-lib module) on the pyboard?
Re: asyncio-like cooperative multitasking framework for uPy
You could use rshell or you could copy the files over usb mass storage.
https://github.com/dhylands/upy-shell/t ... ter/rshell
https://github.com/dhylands/upy-shell/t ... ter/rshell
Re: asyncio-like cooperative multitasking framework for uPy
On unix, using uasyncio I am getting strange behaviour for the following code:
This snippet acts as expected for the first input, but then stops echoing. Is this a bug or am I missing something? The equivalent code runs fine on regular python.
Edit: On inspection I suspect this is an issue with epoll, which is doing something strange with the callback. On a related note, a 'uasyncio.pyboard' module would need to do something about the fact that the pyboard select.poll().register does not provide any way to register a callback. Any thoughts on what would be a good way to approach this problem?
Code: Select all
import sys
import uasyncio
def echo_input():
print('Received this'+ sys.stdin.readline() )
event_loop = uasyncio.get_event_loop()
event_loop.add_reader(sys.stdin.fileno(), echo_input)
event_loop.run_forever()
Edit: On inspection I suspect this is an issue with epoll, which is doing something strange with the callback. On a related note, a 'uasyncio.pyboard' module would need to do something about the fact that the pyboard select.poll().register does not provide any way to register a callback. Any thoughts on what would be a good way to approach this problem?
-
- Posts: 463
- Joined: Wed Apr 08, 2015 5:19 am
Re: asyncio-like cooperative multitasking framework for uPy
Was that ticket ever created? Is it #1550?pfalcon wrote:So, based on the analysis above, I can propose different ways to solve it:
1. [...]
4. We change function signature of .add_reader() and friends - instead of accepting "file descriptor", as CPython asyncio mandates, it would accept "file-like object", and .add_reader() can decide what to do with it (it will be likely overridden for particular implementation anyway).
4 is the most compromise variant, which would be good choice for a winner. But it adds another discrepancy with CPython asyncio . [...] that would warrant a github ticket with reference back to this post.
IHMO, it's ok to abandon CPython compatibilty, so I favour 4). uasyncio is not a drop-in replacement for asyncio anyway, i.e. existing software using asyncio would have to be adapted to run under uasyncio. The current situation makes it impossible to write a custom event loop, which uses something other than objects with fileno() without ripping out uasyncio's guts.
Re: asyncio-like cooperative multitasking framework for uPy
Apparently no, there was little community involvement in uasyncio for baremetal ports so far. But that ticket approaches other issues of unified support, and would be best place to revive this issue.SpotlightKid wrote: Was that ticket ever created? Is it #1550?
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Re: asyncio-like cooperative multitasking framework for uPy
It doesn't do something strange, it works in one-shot mode witch is the natural mode for coroutines, as explained in https://github.com/micropython/micropyt ... -158636465 . Fairly speaking, when writing that comment (and in turn working on porting uasyncio to poll() instead of epoll()), I wondered how CPython handled that, and quickly skimmed thru asyncio code and didn't find anything about one-shot mode. And you report confirms that it's not handled on add_readed/writer level, so then it must be handled on coroutine dispatcher level. But that's too expensive for uasyncio.jjlong wrote: This snippet acts as expected for the first input, but then stops echoing. Is this a bug or am I missing something? The equivalent code runs fine on regular python.
Edit: On inspection I suspect this is an issue with epoll, which is doing something strange with the callback. On a related note, a 'uasyncio.pyboard' module would need to do something about the fact that the pyboard select.poll().register does not provide any way to register a callback. Any thoughts on what would be a good way to approach this problem?
You're welcome to argue the point that uasyncio should be compatible with asyncio there (using good usecases), but... As previous poster pointed, uasyncio is already not compatible with asyncio. And the difference, to remind is that uasyncio is asynchonous coroutine framework, with only rudimentary callback support, whereas asyncio is semi-synchronous multiparadigm framework.
asyncio's synchronous writes is where there can't be compromise - the whole idea is based on "unlimited memory" illusion of big systems, and even there it fails regularly, making apps crash due to OOM errors or thrash swap with system not being responsive for minutes. And for deeply embedded systems, such illusion can't be applied at all. So, uasyncio will stay incompatible with asyncio (because asyncio isn't interested to cover uasyncio's usecases on its side either), and then I'd rather optimize it for its primary usage, rather than try to maintain random heritage of non-coroutine asyncio API.
Also, to remind, while uasyncio can't run asyncio apps, asyncio can be quite easily made to run uasyncio apps. Monkey-patching module for that is provided in micropython-lib. (Non monkey-patching solution is of course also possible.)
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Re: asyncio-like cooperative multitasking framework for uPy
And just to state the obvious: it would be possible to implement synchronous write support on top of what uasyncio currently offers. Doing so is left as an exercise to a reader, but the point is that breaking compatibility in anger may be not the best choice after all...
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Re: asyncio-like cooperative multitasking framework for uPy
First of all nice asyncio. I have some suggestions.
1. Change dispatching mechanic in uasyncio.coro. Since repeat if/elif is bad as log(N)(?) for cpu. And this is already have some overhead:
And result on unix build:
2. Also, I think there should be wait_for function in core, which is just:
or as loop method.
Why this should be there? Cause almost allways there is no polling on boards, and even if they have interrupts how will you add some custom interrupt to your poll? The straight way is flags (interrupt set flag, you check it), so to check flags we need something like wait_for, which deals with non-blocking/non-io but io relevant operations.
My version of asyncio deals only with timeouts and all checks are done as wait_for. This cause latency or high cpu, but I have no choice for this (since my board have no poll/select). But it really not so critical for embedded, since almost all embedded are made as infinite loop with same flag checks and interrupts.
3. There is no cancel method/function. How to stop scheduled coroutine from another?
1. Change dispatching mechanic in uasyncio.coro. Since repeat if/elif is bad as log(N)(?) for cpu. And this is already have some overhead:
Code: Select all
class SysCall:
def __init__(self, *args):
self.args = args
def handle(self):
raise NotImplementedError
# Optimized syscall with 1 arg
class SysCall1(SysCall):
def __init__(self, arg):
self.arg = arg
class Sleep(SysCall1):
pass
class StopLoop(SysCall1):
pass
class IORead(SysCall1):
pass
class IOWrite(SysCall1):
pass
class IOReadDone(SysCall1):
pass
class IOWriteDone(SysCall1):
pass
def isinstance_dispatcher(ret):
if isinstance(ret, Sleep):
pass
elif isinstance(ret, IORead):
pass
elif isinstance(ret, IOWrite):
pass
elif isinstance(ret, IOReadDone):
pass
elif isinstance(ret, IOWriteDone):
pass
elif isinstance(ret, StopLoop):
pass
def action():
pass
foos = {
Sleep: action,
IORead: action,
IOWrite: action,
IOReadDone: action,
IOWriteDone: action,
StopLoop: action,
}
def dict_dispathcer(ret):
foos[type(ret)]()
def test_this(foo, count=10000):
start = utime.time()
for _ in range(count):
for o in classes:
foo(o)
return utime.time() - start
import utime
classes = (Sleep(None), IORead(None), IOWrite(None), IOReadDone(None), IOWriteDone(None), StopLoop(None))
print("isinstance", test_this(ii))
print("dictionary", test_this(di))
Code: Select all
$ ./micropython test.py
isinstance 0.09972596168518066
dictionary 0.0411829948425293
Code: Select all
def wait_for(condition, pause=0, *, loop=None):
while not condition():
yield from sleep(pause, loop=loop)
Why this should be there? Cause almost allways there is no polling on boards, and even if they have interrupts how will you add some custom interrupt to your poll? The straight way is flags (interrupt set flag, you check it), so to check flags we need something like wait_for, which deals with non-blocking/non-io but io relevant operations.
My version of asyncio deals only with timeouts and all checks are done as wait_for. This cause latency or high cpu, but I have no choice for this (since my board have no poll/select). But it really not so critical for embedded, since almost all embedded are made as infinite loop with same flag checks and interrupts.
3. There is no cancel method/function. How to stop scheduled coroutine from another?
Re: asyncio-like cooperative multitasking framework for uPy
As a general response, please try to be more detailed when posing questions, many of them below lack a lot of context. I tried to do a bit of lookup trying to find an interpretation for your words below, but I'm not sure I do that right (and I don't keep all that stuff in my head, so otherwise it may take quite a long time before I'll hack on it again, remember all internal details and will be able to respond).pohmelie wrote:First of all nice asyncio. I have some suggestions.
What is uasyncio.coro? Do you mean uasyncio.core module?1. Change dispatching mechanic in uasyncio.coro.
Not sure what you mean here. O(log(N)) complexity is a bless. O(N) is "good baseline", O(N*log(N)) is "baseline". Anything else "may not scale, but still may be OK in controlled conditions".Since repeat if/elif is bad as log(N)(?) for cpu.
There's no such code in uasyncio.core. Ok, I looked into the source, trying to decipher what you mean. You're saying that that code has O(N) complexity, where N is number of choices in "switch". You say that it can be done in O(1) instead. But as soon as say it like that and look at the code, you'll immediately grasp why I did it like: it's *optimization*. Original idea was of course to have the operations to be in classes' virtual methods, and I optimized it to be a short static switch instead.And this is already have some overhead:Code: Select all
class SysCall: def isinstance_dispatcher(ret): if isinstance(ret, Sleep): pass elif isinstance(ret, IORead): pass elif isinstance(ret, IOWrite): pass elif isinstance(ret, IOReadDone): pass elif isinstance(ret, IOWriteDone): pass elif isinstance(ret, StopLoop): pass
And, so what - you posted a typical purposeless micro-benchmark. Did you try to run it with native codegen? With viper codegen? Note that to answer "yes", you would need to finish implementation of them first. But until you did that, it makes no sense to micro-optimize it. And looking into that, you're still fixed on "performance". But MicroPython's is not about raw speed, it's about memory efficiency. Have you measured memory impact of changing code above? If it increased memory pressure, it's not ok.
And result on unix build:Code: Select all
$ ./micropython test.py isinstance 0.09972596168518066 dictionary 0.0411829948425293
Here, you want to post link to asyncio documentation. Unless there's proof that such function exists, there's almost zero chance to have it there.2. Also, I think there should be wait_for function in core, which is just:Code: Select all
def wait_for(condition, pause=0, *, loop=None):
Arguments like that are still not good enough. I wrote web framework on top of uasyncio (https://github.com/pfalcon/picoweb), then took an existing 3rd-party webapp and ported it to this web framework (https://github.com/pfalcon/notes-pico) - I never needed wait_for(). So, I'd like to see real-world (and community) useful apps which need this functionality - assuming you want to contribute to uasyncio development. Otherwise, it's still very useful feedback, I'll keep it in mind, but wouldn't plan to add anything, because current direction of usasyncio development is optimizing it to work on PyBoard and other bare-metal boards, and adding more stuff conflicts with this direction, so first aim is to get it running there, then will consider adding more stuff, while controlling how it affects uasyncio resource usage.Why this should be there? Cause almost allways there is no polling on boards, and even if they have interrupts how will you add some custom interrupt to your poll?
See above - I never needed to cancel a coroutine, while doing more or less non-trivial things. You obviously can do that by setting some flag and letting target coro check it. That's especially useful as the target coro may very well need to clean up its state. Again, I can easily agree that it would be nice to have such functionality, but that can be said about any feature which full asyncio has, and uasyncio doesn't have. So, as usual, there should be real-world example why it's ~ unavoidable to have it, and analysis of its impact on uasyncio portability to the smallest of boards.3. There is no cancel method/function. How to stop scheduled coroutine from another?
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/
Re: asyncio-like cooperative multitasking framework for uPy
Yep, it's typo. uasyncio.core of course.pfalcon wrote:What is uasyncio.coro? Do you mean uasyncio.core module?
Ah, yes. My fault. Can't start to think about memorypfalcon wrote:And looking into that, you're still fixed on "performance"
So this question is off, since I can't and don't want to dig into memory efficience.
There is no such function in asyncio. But I don't get why there is "almost zero chance", since some things in uasyncio was made before (loop.create_task) and some without such asyncio api (loop.wait). But of course, you are author and you decide what to do with it.pfalcon wrote:Here, you want to post link to asyncio documentation. Unless there's proof that such function exists, there's almost zero chance to have it there.
This framework is based on sockets. If you try to run on almost any kind of hardware (except pyboard if it have socket polling) you will fail. Almost all real boards are made without polling, they have blocking or/and flag/interrupt api. And this example is more like proof of concept, that micropython can do on PC. Of course if you use powerful board with linux everything is ok, but in this case you can use cpython.pfalcon wrote:Arguments like that are still not good enough. I wrote web framework on top of uasyncio (https://github.com/pfalcon/picoweb), then took an existing 3rd-party webapp and ported it to this web framework (https://github.com/pfalcon/notes-pico)
I think "arguments like that are still not good enough".pfalcon wrote:I never needed wait_for()
As I told above, this is for "flag"/"interrupt" based api's. If board have no api for polling you must do it yourself and there is to ways. First one is to make polling on C level, it's harder, but you will see "nice" poll() on python level (even if inside is same infinite loop with repeat checks). Second one is to make polling on python level. First one requires recompile if you want to add new type of objects to wait to, second — only one more condition for wait_for. My primary work is mostly with embedded boards and I can't remember even one board, which have polling (except some, which can be used with linux/qnx). And of course, most of embedded things are closed source and are not very popular. So, I can't show you good project, which will imagine you.pfalcon wrote:I'd like to see real-world (and community) useful apps which need this functionality
Uasyncio is extremly small (and this is very good). So, I think most of development is done at this point. And everyone can write his own asyncio. I just want to discuss a little bit, cause maybe I missed something That's why I'm not on github, but here.pfalcon wrote:assuming you want to contribute to uasyncio development
Clean up is done via CancelledError exception, which interrupted coroutine can catch and finalize. For complex coros, which are nested you should check stop flag after every await in every coro in stack, and even in this case you can't say, that it will be cancelled right after loop will take flow control.pfalcon wrote:I never needed to cancel a coroutine, while doing more or less non-trivial things. You obviously can do that by setting some flag and letting target coro check it. That's especially useful as the target coro may very well need to clean up its state. Again, I can easily agree that it would be nice to have such functionality, but that can be said about any feature which full asyncio has, and uasyncio doesn't have. So, as usual, there should be real-world example why it's ~ unavoidable to have it, and analysis of its impact on uasyncio portability to the smallest of boards.