Avoiding "async def" creep

[xt6master]

Reset when the network slows down mmm..okay. Remote services you don't control hitch, TCP buffers grow and shrink - relying on it to be empty enough and large enough for a message on the UI thread (your only thread) is... May your wifi be strong, your dependencies flawless and your latencies ever green that you never need learn network realities.

If you put time.sleep(5) in conn.process() you will fall behind the Hello %d producer rather than processing the requests in parallel. If you put `await asyncio.sleep(5)` in there, however, you would actually be eligible for Little's Law and reap benefits of async/await and concurrent coroutines. As it is, your application gains nothing from the complexity though you could at least do other things on the main thread while waiting for the 1 byte. (so it'll only hitch when it receives a message woo)

So it's all single-threaded, with several tasks, i.e. coroutiness, running cooperatively. Lots of async/await activity (i.e. task switching) may be going on. But the bulk of my application is written as running the show without awaits.

It appears that you have misunderstood your own concurrency model. In fact you have only 2 long-running tasks: 1 that publishes "Hello %d" every 3ish seconds and 1 that awaits 1 byte to kick off a stack that consumes your cpu until the request is completely materialized, handled and responded to, then waits for another byte. It is single threaded you are very right there, more right that I suspect you intended.
There are 5 switches to the main task to do a blocking publish and 5 switches to the recvTrigger task to pass 1 materialized byte into a blocking handler.
There is not a lot of task switching and not a lot of asynchrony. You have 1 degree of handler parallelism here due to your blocking handler loop. Sprinkle in some time.sleep() to convince yourself of this. If you are actually shooting for 1 degree of parallelism, why are you even using asyncio? Just use a thread, block it to your heart's content and plop messages onto a queue that your message handler loop consumes from. That way you at least get a little pressure relief architecturally and other people who look at your code will understand it because it uses 1 consistent paradigm.

Sure, and they are. In moderation.

Good luck to you. I sincerely hope others are not led into fear of async by this. It is easy stuff; and like class member methods vs. module static methods, the calling convention is a little different. It's so straightforward though!


Right, so tl;dr: Python's async/await is pythonic, fine and generally the right way to do responsive apps in 2020 (some cpu-bound apps may require threading to be responsive, but Python isn't usually the best choice for heavily cpu-bound applications either). Fighting against it while trying to benefit from it is subtle and will most likely confuse even the author.
If you want a responsive python app, you probably want to use async libraries and idioms. If you want a serial python app (as you have here), you probably want to use serial libraries and idioms (as you have not done here).

Bonus tl;dr: Answering the topic's question of how to avoid async def, there are 2 ways:
A. Do not do anything that involves waiting. (network activity, DMA send/receive, file access, etc.) Then you won't be tempted to make your app responsive because it'll be responsive already.
B. Do not use async. Figure out a different way to keep your app responsive.

[jcw]

This sounds a bit defensive to me: don't do this because it's not right.

Note that there are no sleep calls in my application code (again: it's message/dataflow based) and there is no UI.

I'm a bit surprised by the pushback. But it did point out a flaw in my thinking: the callbacks I'm using mean that there is no "application task" at the moment, the app "hops around" on top of whichever task currently triggers. It could (and maybe should) be done through a coroutine yield instead. That doesn't change the model in any way: my app code is synchronously working through incoming messages, with a single suspend point (or return, same thing).

[tve]

Code: Select all

Good luck to you. I sincerely hope others are not led into fear of async by this. It is easy stuff;

LOL. Async is anything but easy. There are lots of issues around atomicity, about deadlock, and about resource consumption (to name 3 topics) that are anything but simple.

You also say that asyncio is pythonic, that's just saying that the python designers decided this is the standard way to do something in python. That's a good argument to say that unless you know better you should do it that way 'cause it has the highest chance of continuing to work in the future and the highest chance of being recognized and understood by other python programmers. IMHO that's an important argument, but that doesn't necessarily make it the right way or the best way.

One place where I find that asyncio sucks is when you write a framework or library that has some form of callback. Such pieces of code end up calling some arbitrary code plugged in by a different programmer later on. And one has to make the decision up-front whether the code that is being plugged is allowed to block or not and not only that, one forces that decision on all future uses.

One example I've experienced my self is the `subs_cb` callback in Peter's mqtt_as (that's the callback the library makes into the app when a message arrives so the app can handle the message). He decided that the callback is a `def` and not an `async def`. Why? It's very inconvenient at times, for example, if the message handler wants to send a reply message. In this specific case, I believe Peter made the right decision given the internals of his library. However, I wrote a compatible library that functions differently internally and had the same interface and in my library's case making the callback `async def` makes more sense. One of the reasons still not to make it an async def is that doing so forces "async def" on all callbacks yet most make no use of that. Ugly: instead of thinking about what the deep effects of suspending in the handler or not are we end up debating about the syntax.

What jcw is trying to do makes a ton of sense to me. For many years (and many years ago) I wrote a lot of async C code in a unix context. Basically an outer loop that calls select (later changed to poll/epoll/...) and then dispatches to the appropriate handler which isn't allowed to block. That structure has a lot of positive attributes, however it requires that all asynchronous events be selectable. On unix that's mostly a given, but in MP it's not, and even on unix it makes it awkward to mix-in internally generated events. The way I see what jcw is doing is very similar, except that he's using a small number of asyncio coroutines to replace select, which allows him to have more flexibility when it comes to event sources. The main app code then remains non-blocking "cpu-only" type of code and not tainted by async def.

[pythoncoder]

...
One example I've experienced my self is the `subs_cb` callback in Peter's mqtt_as (that's the callback the library makes into the app when a message arrives so the app can handle the message). He decided that the callback is a `def` and not an `async def`. Why? It's very inconvenient at times, for example, if the message handler wants to send a reply message. In this specific case, I believe Peter made the right decision given the internals of his library...

Is this a real problem? A callback can always issue

Code: Select all

asyncio.create_task(foo())

It's also easy to make a method accept either a coro or a callback using my launch function

Code: Select all

async def _g():
    pass
type_coro = type(_g())

# If a callback is passed, run it and return.
# If a coro is passed initiate it and return.
# coros are passed by name i.e. not using function call syntax.
def launch(func, tup_args):
    res = func(*tup_args)
    if isinstance(res, type_coro):
        asyncio.create_task(res)

I accept this is a workround for the red/blue problem and that a better async paradigm would make this redundant. But in terms of writing practical code, I'm afraid I see the red/blue thing as a non-issue. I've yet to encounter a problem which couldn't easily be resolved.

[tve]

...
One example I've experienced my self is the `subs_cb` callback in Peter's mqtt_as (that's the callback the library makes into the app when a message arrives so the app can handle the message). He decided that the callback is a `def` and not an `async def`. Why? It's very inconvenient at times, for example, if the message handler wants to send a reply message. In this specific case, I believe Peter made the right decision given the internals of his library...

Is this a real problem? A callback can always issue

Code: Select all

asyncio.create_task(foo())

That's not the same thing. If the incoming message was QoS=1 the ack now gets sent before the handler event starts (typically), never mind completes, so it no longer ACKs handling of the message. It also has very different flow-control behavior: if there is an incoming stream of messages you can end up creating a gazillion tasks before any of them even runs, resulting in an uncontrollable memory allocation situation. I'm not saying that making the handler a coro is an automatic solution to all this, I only gave an example where the red/blue coloring forces a choice on the library designer that is unfortunate and has a tendency to draw the attention away from the deeper issues.

[kevinkk525]

If the incoming message was QoS=1 the ack now gets sent before the handler event starts (typically), never mind completes, so it no longer ACKs handling of the message. It also has very different flow-control behavior: if there is an incoming stream of messages you can end up creating a gazillion tasks before any of them even runs, resulting in an uncontrollable memory allocation situation.

Using the ACK message to acknowledge a message handling could be problematic and uncomfortable. For example if an incoming message starts a pump for 5 minutes and then switches it off again. If the mqtt client ACKs the incoming message before starting a task that processes this message, the task can easily activate the pump, wait 5 minutes and turn it off again.
If you expect an ACK after message processing, the pump task would need to return after activating the pump and start a new task scheduled in 5 minutes to turn the pump off again.
In this scenario you would win absolutely nothing an make integrating libraries into mqtt very difficult and uncomfortable.

As for flow-control: I was considering the different variants. Both variants will have problems if too many messages arrive. Callbacks might result in a memory error resetting the device. (this could be prevented by implementing a concurrency limit in the client, similar to how the async variant would work).
Processing one message after the other without putting them on a queue (tasks) can result in a huge message backlog on the broker side (maybe even timeouts? not sure how mqtt brokers work in that regard).

Ultimately my choice to stick with callbacks was mainly due coroutines being dangerous in this scenario. If an incoming message triggers a badly written coroutine that doesn't exit within the next 5 minutes, you won't receive any messages during that time. And if your "callback coroutine" creates a new task for further processing and not keeping the message flow stuck, then there is no difference anymore from using callbacks directly.

[pythoncoder]

@tve I take your point. I hadn't appreciated that you were considering the case where a coro is awaited at that point in the code, prior to sending the ACK. Starting a task in the callback is not the same thing.

It would be easy to detect the type of the callback and await it if it's a coro. But this would lead to the potential problem which @kevinkk525 raised. It's quite likely that a user might write a slow coro. Writing a time consuming callback in a uasyncio application would require a user of some, er, perversity

As Kevin says, choices made here are critical for flow control. When we were developing this and the iot library we had extensive discussions on this subject. In general there is a tradeoff between throughput and RAM usage. We concluded that you can design a "conservative" API which minimises the risk of RAM exhaustion or an "aggressive" one which gives the user a higher degree of control over concurrency whose exploitation requires a thorough grasp of these issues. An "aggressive" design also requires the user to consider potential concurrency issues.

On a philosophical note, say we had an async framework which solved the red/blue function problem so that a coroutine and a callback were indistinguishable. A case like we are discussing would put the onus on the user programmer to ensure that the "callback" terminated quickly. With these objects being separate, developers can give user a "nudge" towards fast termination by specifying a callback.

[tve]

A case like we are discussing would put the onus on the user programmer to ensure that the "callback" terminated quickly. With these objects being separate, developers can give user a "nudge" towards fast termination by specifying a callback.

Thanks for the thoughts. I went back and forth in my library for exactly the reasons you mention. And I think I'm going to switch yet again. Sigh. A real-world although extreme example is OTA. The OTA process consists of just under 1000x 1400-byte messages sent to the esp32. The message handler needs to write them to flash (more precisely, it needs to accumulate a 4KB block and then write that to flash). Given that the writes to flash are blocking there is no conflict: the handler is a plain function and the writes to flash are system-blocking to the tune of an ~80ms hiccup IIRC.

Suppose there was an async flash interface available, it would be impossible to use as far as I can tell. To use it, the handler would have to `create_task` but the messages are coming in as fast as the TCP connection allows and it often only takes a very small number of messages to run out of memory.

If the handler is allowed to be a coro it can use the async flash interface and block until the write completes. Yes, that blocks further incoming messages, but that's the point: the device just can't handle them anyway and it's better for them to pause in the broker and network pipe.

The big issue I see with allowing a coro handler is deadlock: if the coro does a publish with QoS=1 then it has to wait for an ACK, and that requires processing incoming messages, but that is not possible. It's a poor property of MQTTv3 that a client has to be able to receive an unbounded number of incoming messages before it gets an ACK. I saw that MQTTv5 can limit the number of messages in the pipe but I haven't looked whether it says anything about giving ACKs priority. Maybe that's implicit.

On a philosophical note, say we had an async framework which solved the red/blue function problem so that a coroutine and a callback were indistinguishable. A case like we are discussing would put the onus on the user programmer to ensure that the "callback" terminated quickly. With these objects being separate, developers can give user a "nudge" towards fast termination by specifying a callback.

Yes, but oddly in my numerous years of using Go I don't recall hitting this type of issue. (Go does async without red/blue.)

[jcw]

When I write something like "with open('blah','r') as f: ...", am I correct in assuming that this will block for the duration of the open, and then again for the subsequent reads?

I'm asking because I get the impression that flash and sd-card reads can be slow (usng one of the pyboard variants). Writes perhaps even slower.

At a basic level, this is to be expected when the block drivers use h/w polling of the SPI bus, for example. But even if these were to be augmented with DMA and interrupts on completion, I suspect no speedup will be possible, because the absence of "async" cq "await" keywords imples that by design, the code cannot relinquish control to anything else.

The same holds for a (contrived) "print(10000*'x')" when sent through a serial port (e.g. 1 s @ 115200 baud).

So in essence, we have code which is either: normal, slow, or suspendable. Where one type of I/O will be unavoidably slow and the other relies on adding async/await keywords all the way up the call chain to the event loop.

I'm sorry, with sincere respect for people who have thought a lot about this and worked hard on it, but (if the above is correct) it feels like a bad joke.

[pythoncoder]

The issue is whether blocking occurs at the device driver level. In the case of UARTS it does not because the underlying device driver maintains circular buffers by means of interrupts. You can therefore use the StreamReader and StreamWriter classes to do asynchronous I/O to a UART. As you probably know you can do asynchronous I/O with sockets by using nonblocking sockets or select/poll.

Unfortunately MicroPython file and Flash I/O is synchronous, and Flash I/O can block for some time when a sector erase is required.

You can't make a silk purse out of a sow's ear. For nonblocking file I/O you'd need to find - or write - an asynchronous device driver. In general writing an efficient nonblocking driver depends on hardware support for interrupts and/or DMA. The MicroPython SPI driver does block, but typically for periods which are very short in uasyncio terms: baud rates of 20MHz can be practical.

Given an asynchronous I/O device driver compatible with the uasyncio stream protocol, uasyncio does allow asynchronous I/O. I have written drivers supporting this. There are plans to enhance uasyncio to offer an option to prioritise I/O streams, further enhancing performance.