asyncio wait for task in a group

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
angel0ne
Posts: 7
Joined: Sun Mar 21, 2021 4:33 pm

asyncio wait for task in a group

Post by angel0ne » Mon Apr 05, 2021 6:49 am

Hi all,

I'm new to Asyncio (and MicroPython as well) and I'm writing a firmware for an ESP32.
I need to update an LCD display every time one of multiple events is triggered. Let me give an example just to clarify:
I have a task that reads from a DHT22 sensors (temperature and humidity) every two seconds, every time it returns I need to write the new values on the LCD display (this is quite easy using the Event mechanism).

I also have two push button to increase/decrease the target temperature and, every time one of this button is pressed I need to update the display with the new selected temperature.

Is there a way to await for ONE of multiple task and return when just the first one returns?
I studied Barrier and Gather but it is useful when you have to wait the completion of all tasks that belongs to the pool.

Many thanks to everyone would help me.

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: asyncio wait for task in a group

Post by kevinkk525 » Mon Apr 05, 2021 7:48 am

You can poll all events like:

Code: Select all

a=[event1,event2,event3]

for event in a:
    if event.is_set():
        # process LCD
        event.clear()
await asyncio.sleep_ms(20)
The downside is obviously that it checks all the events every 20ms and is therefore less efficient than awaiting a single event directly. Also it might take 20ms until an event is recognized, whereas awaiting a single event will work "immediately".
But it's probably the easiest workaround.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

angel0ne
Posts: 7
Joined: Sun Mar 21, 2021 4:33 pm

Re: asyncio wait for task in a group

Post by angel0ne » Mon Apr 05, 2021 8:57 am

Thanks Kevin, I thought to do something similar but the problem is the continuous polling and the minimum amount of time to receive the next event (in this case 20ms).

Is there another way to make the same stuff without polling?
kevinkk525 wrote:
Mon Apr 05, 2021 7:48 am
You can poll all events like:

Code: Select all

a=[event1,event2,event3]

for event in a:
    if event.is_set():
        # process LCD
        event.clear()
await asyncio.sleep_ms(20)
The downside is obviously that it checks all the events every 20ms and is therefore less efficient than awaiting a single event directly. Also it might take 20ms until an event is recognized, whereas awaiting a single event will work "immediately".
But it's probably the easiest workaround.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: asyncio wait for task in a group

Post by pythoncoder » Mon Apr 05, 2021 9:07 am

Am I missing something here? It seems to me that you want one event, triggered by multiple tasks. This would read data from five sources, triggering the display when any changed significantly:

Code: Select all

import uasyncio as asyncio
data = [0]*5
sources = []  # Somehow create a list of data sources
evt = aysncio.Event()
async def get_temp(n):
    while True:
        temp = read_temperature(sources[n])
        if abs(temp - data[n]) > 4:
            data[n] = temp
            evt.set()
            evt.clear()
        await asyncio.sleep(2)

async def main():
    for n in range(5):
        asyncio.create_task(get_temp(n))
    while True:
        await evt.wait()
        # Output data to LCD
        
Peter Hinch
Index to my micropython libraries.

angel0ne
Posts: 7
Joined: Sun Mar 21, 2021 4:33 pm

Re: asyncio wait for task in a group

Post by angel0ne » Mon Apr 05, 2021 12:24 pm

Thanks Peter, it is exactly what I looking for. I didn't realized that multiple tasks can trigger the same event.
I've just have a little problem:

Code: Select all


async def dht_measure(d: DHT22):
    while True:
        global temp, hum
        d.measure()
        temp = d.temperature()
        hum = d.humidity()
        event.set()
        event.clear()
        await asyncio.sleep(5)
        
async def update_display(display):
    while True:
        await event.wait()
        print('Update display')
        display.fill(0)
        display.text('Temp. {:.1f} C'.format(temp), 0, 0)
        display.text('Humidity. {:.2f}'.format(hum), 0, 10)
        display.text('Target Temp {:.1f}'.format(targetTemp), 0, 21)
        display.show()
        event.clear()
        
async def main():
    global dht
    deviceList = i2c.scan()
    if len(deviceList) == 0:
        raise Exception('No device found on I2C bus, please check the connection wires')

    display = Display(i2c, deviceList[0])
    dht = dht.DHT22(Pin(23))
    
    asyncio.create_task(dht_measure(dht))
    asyncio.create_task(update_display(display))
    
    #This won't trigger the update_display task, maybe because it take some time to be scheduleted
    event.set()
    event.clear()
As you can see dht_measure runs every 2 seconds and update_display wait for someone that triggers the event (I've omitted the code for the pushbutton that triggers the same event).
At the startup, the display is off until 2 seconds expires (the first time that dht_measure trigger the event), is there a way to trigger the event manually? I tried to call

Code: Select all

event.set()
event.clear()
after the create_task but it won't work.

Many thanks for the support

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: asyncio wait for task in a group

Post by kevinkk525 » Mon Apr 05, 2021 12:40 pm

Don't clear the event in the reading method, let the main loop do that.

As for the 2 seconds delay, you can change that by turning your main loop around a bit:

Code: Select all

async def update_display(display):
    # if needed wait a few ms on startup
    while True:
        print('Update display')
        display.fill(0)
        display.text('Temp. {:.1f} C'.format(temp), 0, 0)
        display.text('Humidity. {:.2f}'.format(hum), 0, 10)
        display.text('Target Temp {:.1f}'.format(targetTemp), 0, 21)
        display.show()
        await event.wait()
        event.clear()
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: asyncio wait for task in a group

Post by pythoncoder » Mon Apr 05, 2021 3:53 pm

I'm puzzled. There seems little point in populating the display until a reading has been taken. So you don't want

Code: Select all

    event.set()
    event.clear()
in main().

It seems to me that, without those lines, your code as written, will populate the display with valid data almost immediately. The dht_measure() task will immediately take a reading and set the event and update_display() will detect the event and fill the display.

If this isn't happening I can only suggest putting in some print statements to figure out what is going on. The only minor point I would raise is that there is no need for event.clear() in update_display() as it has already been cleared in dht_measure()

@kevinkk525
I used to advocate issuing event.set() in the producer task and event.clear() in the waiting task(s) because - as I recall - that was how events worked in V2. In V3 you can always clear down an event immediately after setting it: I think this is normally the best way to use the class as it keeps the set/clear logic in one place. This avoids risks of an event being set and not cleared.
Peter Hinch
Index to my micropython libraries.

angel0ne
Posts: 7
Joined: Sun Mar 21, 2021 4:33 pm

Re: asyncio wait for task in a group

Post by angel0ne » Mon Apr 05, 2021 4:14 pm

Thanks for the support. I figured out the problem was the order of task scheduling:

main

Code: Select all

asyncio.create_task(dht_measure(dht))
asyncio.create_task(update_display(display))
In this order the first event triggered from dht_measure was missed by update_display, maybe the scheduling require some time.

If I switch the tasks orders:

main

Code: Select all

asyncio.create_task(update_display(display))
asyncio.create_task(dht_measure(dht))
The display is update at the first run of dht_measure.
I though that an event should be cleared in the waiting task, you said in the V3 you can clear an event immediately after setting it.
What exactly is the aim of clearing an event? Doesn't clear automatically after every waiters have read it?

Thanks for the support, I really like the uasyncio library you've done a great job

kevinkk525
Posts: 969
Joined: Sat Feb 03, 2018 7:02 pm

Re: asyncio wait for task in a group

Post by kevinkk525 » Mon Apr 05, 2021 6:12 pm

pythoncoder wrote:
Mon Apr 05, 2021 3:53 pm
In V3 you can always clear down an event immediately after setting it: I think this is normally the best way to use the class as it keeps the set/clear logic in one place. This avoids risks of an event being set and not cleared.
I would highly recommend not to do that.
It's not a matter of how V3 works vs V2. It can lead to difficult to debug race conditions between consumer and producers if the consumer is not a strictly synchronous function (except for the awaiting of the event).
In the current project with the current version of "update_display" it wouldn't matter. But imagine the lcd update functions to be asynchronous with some wait time in it or add some mqtt publish function into that loop and you'll create a race condition that could lead to the consumer loop missing Event triggers. While the consumer is awaiting something else (like the lcd update or an mqtt publish), a producer might trigger the event. However, no task is waiting on the event so if the producer now immediately clears the event, the consumer loop will not notice that the event was set for a brief amount of time and will just await the event until the producer triggers it again. Now you have data loss that will be difficult to hunt down as it is a race condition.
Conclusion: Always clear the event in the consumer loop/task, never in the producer task.
(at least if you have n producers and 1 consumer, everything else will be more complicated)
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: asyncio wait for task in a group

Post by pythoncoder » Tue Apr 06, 2021 6:23 am

kevinkk525 wrote:
Mon Apr 05, 2021 6:12 pm
...
Conclusion: Always clear the event in the consumer loop/task, never in the producer task.
(at least if you have n producers and 1 consumer, everything else will be more complicated)
I had a feeling you might have a reason for doing this. ;) You are right - I'll have to correct my tutorial.

The notion that multiple tasks can wait on an Event (as advocated by @Damien)) is true, but only on the proviso that all are paused on the event at the time it is triggered. If it is possible that any one of them may not yet have reached the Event it is indeed complicated. :(

Thank you for pointing out this case.
Peter Hinch
Index to my micropython libraries.

Post Reply