uasyncio StreamReader read question

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
stephanelsmith
Posts: 9
Joined: Mon Oct 18, 2021 7:03 pm

uasyncio StreamReader read question

Post by stephanelsmith » Tue Oct 26, 2021 9:22 pm

I'm likely misunderstanding this, but thought I'd post anyways.

Something I'm noticing in my application, in Uasyncio StreamReader docs, I'm expecting StreamReader.read to return as many characters up to value n.
https://github.com/peterhinch/micropyth ... UTORIAL.md
"read(n) Return as many characters as are available but no more than n."

In practice, I'm finding streamreader.read reads exactly 'n' bytes before returning form await. Maybe that's expected. Here's my snippit, I'd expect to echo each character as I'm typing, however it waits until 10 characters are queued up before returning.

Code: Select all

import sys
import gc
import uasyncio as asyncio

async def main():
    sreader = asyncio.StreamReader(sys.stdin)
    while True:
        try:
            r = await sreader.read(10)
            sys.stdout.write(r)
        except asyncio.CancelledError:
            return
        except Exception as err:
            sys.print_exception(err)
            return

try:
    asyncio.run(main())
except KeyboardInterrupt:
    pass
except Exception as err:
    sys.print_exception(err)
finally:
    asyncio.new_event_loop()
    gc.collect()

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uasyncio StreamReader read question

Post by pythoncoder » Wed Oct 27, 2021 9:23 am

I wonder if this is some kind of oddity with stdin. I have just tried the following script on a Pyboard and it works as expected, with the read returning the number of characters available:

Code: Select all

import uasyncio as asyncio
from machine import UART
uart = UART(4, 9600)

async def sender():
    swriter = asyncio.StreamWriter(uart, {})
    while True:
        swriter.write('Hello\n')
        await swriter.drain()
        await asyncio.sleep(5)

async def receiver():
    sreader = asyncio.StreamReader(uart)
    while True:
        res = await sreader.read(10)
        print('Recieved', res)

async def main():
    asyncio.create_task(sender())
    asyncio.create_task(receiver())
    while True:
        await asyncio.sleep(1)

def test():
    try:
        asyncio.run(main())
    except KeyboardInterrupt:
        print('Interrupted')
    finally:
        asyncio.new_event_loop()
        print('as_demos.auart.test() to run again.')

test()
The output was as follows, showing that read(10) returns with fewer than 10 characters each time. The readexactly(n) method is provided to pause until n characters are available.

Code: Select all

>>> import rats102
Recieved b'Hello\n'
Recieved b'Hello\n'
Recieved b'Hello\n'
Recieved b'Hello\n'
Interrupted
Peter Hinch
Index to my micropython libraries.

peter9477
Posts: 2
Joined: Mon Nov 15, 2021 1:23 am

Re: uasyncio StreamReader read question

Post by peter9477 » Mon Nov 15, 2021 1:34 am

I can confirm I see the same behaviour on the RP Pico port. Just spent an hour experimenting to characterize this and attempt to find workarounds.

It would be nice if sys.stdin could act like a regular UART, so the USB connection could be treated mostly like a third UART interface, but for now it appears possibly one has no choice but to use StreamReader.read(1) and process the characters one by one.

It's actually a bit surprising that read(1) even works without blocking asyncio, but I'll take what I can get in this case...

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uasyncio StreamReader read question

Post by pythoncoder » Mon Nov 15, 2021 9:23 am

peter9477 wrote:
Mon Nov 15, 2021 1:34 am
...
It's actually a bit surprising that read(1) even works without blocking asyncio, but I'll take what I can get in this case...
StreamReader.read() should never block asyncio: the select.poll mechanism is used to ensure that uasyncio tasks continue to be scheduled until data is available.

So your workround seems to make sense: if StreamReader.read(N) pauses until N characters are available (while letting other tasks be scheduled), setting N to 1 can be expected to work as you describe.
Peter Hinch
Index to my micropython libraries.

peter9477
Posts: 2
Joined: Mon Nov 15, 2021 1:23 am

Re: uasyncio StreamReader read question

Post by peter9477 » Tue Nov 16, 2021 4:07 pm

pythoncoder wrote:
Mon Nov 15, 2021 9:23 am
peter9477 wrote:
Mon Nov 15, 2021 1:34 am
...
It's actually a bit surprising that read(1) even works without blocking asyncio, but I'll take what I can get in this case...
StreamReader.read() should never block asyncio: the select.poll mechanism is used to ensure that uasyncio tasks continue to be scheduled until data is available.

So your workround seems to make sense: if StreamReader.read(N) pauses until N characters are available (while letting other tasks be scheduled), setting N to 1 can be expected to work as you describe.
I don't quite follow your conclusion, Peter.

If StreamReader.read(N) blocks the whole thread when N>1 even though it should never block, why should anyone expect that it doesn't block when N==1 ?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uasyncio StreamReader read question

Post by pythoncoder » Wed Nov 17, 2021 6:24 am

I think we're talking at cross purposes here with the term "blocking". None of the StreamReader methods should block uasyncio as a whole. But some methods cause the running task to pause. For example .readexactly(N) will pause that task only until N characters are available.

So what is the behaviour when you issue read(1) and no character is available? Does that task continue or does it pause until a character arrives?
Peter Hinch
Index to my micropython libraries.

Post Reply