Avoiding "async def" creep

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
User avatar
jcw
Posts: 37
Joined: Sat Dec 21, 2019 12:08 pm
Location: CET/CEST

Re: Avoiding "async def" creep

Post by jcw » Sat Jun 06, 2020 3:09 pm

You can't make a silk purse out of a sow's ear
Heh.

Yes, all good points. Note that block I/O speed can also depend on response latency, i.e. when reading a block on an SD card, there's a delay before actual data comes in (in the order of few ms, I think). There's a 100x retry loop here:

https://github.com/micropython/micropyt ... rd.py#L154

I've written DMA-enabled SD card I/O code for STM32 and it might be nice to port that to µPy, but I'm not sure it would solve anything. I think my expectation was that async I/O would be more Unix-like, i.e. a process which sometimes suspends while the request it makes gets handled. So you'd have a Task as the unit of execution competing for the CPU with other Task instances, switching only where there is an "await". With open/read/write implemented as thin Python wrappers around the actual drivers (Python or C primitives), suspending the "enclosing" task context as needed.

The key benefit I see in having "await" is that it lets you precisely specify where cooperative (i.e. voluntary) task switching is allowed to happen. I'm sure the consensus is that this is superior to dealing with preemptive threads.

My confusion appears to be that Python treats every await and yield as a sign that the enclosing routine must be turned into a coroutine (with different activation semantics, which the obligatory "async" keyword then clarifies). I had assumed that "await" merely signals a suspend point for the current task (which can be nested several calls deep). IOW, in Python, it's "coroutines all the way down". Compare this to Go, where an outer "go" identifies the suspend context, and anything that wants to suspend, regardless how deeply nested at that point (blocking I/O, channel get/put), wil suspend it all.

Anyway - by now I may have bored enough people with my pedantic posts. Still, I hope we can at least agree that the current situation is unfortunate: we have slow blocking code which can't be made concurrent (open/read/write/etc) and we have coroutines, wrapped in tasks to allow scheduling and cancelling, with associated async/await/yield specifiers.
Last edited by jcw on Sat Jun 06, 2020 4:54 pm, edited 1 time in total.

User avatar
tve
Posts: 216
Joined: Wed Jan 01, 2020 10:12 pm
Location: Santa Barbara, CA
Contact:

Re: Avoiding "async def" creep

Post by tve » Sat Jun 06, 2020 3:44 pm

jcw wrote:
Sat Jun 06, 2020 3:09 pm
You can't make a silk purse out of a sow's ear
Anyway - by now I may have bored enough people with my pedantic posts. Still, I hope we can at least agree that the current situation is unfortunate: we have slow blocking code which can't be made concurrent (open/read/write/etc) and we have coroutines, wrapped in tasks to allow scheduling and cancelling, with associated async/await/yield specifiers.
I think you're hitting the spot where the creep is really, really painful: libraries. Did you just write an async sd card driver? Congrats, now you get to make a copy of the filesystem code and sprinkle async/await into it. Got async driver for Uart? Congrats, now write an "aprint" so you can `await aprint(1000000*"x")`. Etc.

Post Reply