uasyncio task freezing

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
coayer
Posts: 4
Joined: Sat Oct 02, 2021 7:40 pm

uasyncio task freezing

Post by coayer » Sat Oct 02, 2021 7:54 pm

Hi everyone,

I have a uasyncio task running on an ESP8266 which sends DNS lookups (UDP). I'm using uasyncio.StreamReader (and writer) on the socket object to do this asynchronously. Most requests are successful, but sometimes the task indefinitely hangs on "await reader.read(128)", which is reading the response from the server. There's a timeout set on the socket.

I have a main() task with its own infinite loop, and I can see that when the DNS read gets stuck the main loop keeps running just fine.

If I press CTRL-C, I get:

Code: Select all

Traceback (most recent call last):
  File "main.py", line 290, in <module>
  File "uasyncio/core.py", line 1, in run
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in wait_io_event
KeyboardInterrupt: 
Am I missing something or is this a bug I should report?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uasyncio task freezing

Post by pythoncoder » Sun Oct 03, 2021 8:29 am

If you are using socket.getaddrinfo() this is a blocking call. This is a known issue which hasn't yet been fixed.
Peter Hinch
Index to my micropython libraries.

coayer
Posts: 4
Joined: Sat Oct 02, 2021 7:40 pm

Re: uasyncio task freezing

Post by coayer » Sun Oct 03, 2021 9:42 am

I am using getaddrinfo, but directly on an IP address (1.1.1.1) so that I can create a socket to the DNS server — I'm doing the DNS lookup manually. That's not the line that is freezing though, it's the "await reader.read(8)" after a "writer.write" and "await writer.drain".

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uasyncio task freezing

Post by pythoncoder » Mon Oct 04, 2021 9:18 am

My MicroPython work has been with nonblocking sockets so I'm not entirely sure about the behaviour of blocking sockets. I would imagine that, in the absence of the requested number of characters, read(n) would terminate on timeout with as much data as was available. It would seem this isn't the case.

One option would be to investigate this and find out how read(n) behaves on CPython. Another way would be to impose a timeout on the read() using a uasyncio timeout. I think this is probably the orthodox uasyncio approach, rather than having a socket timeout.
Peter Hinch
Index to my micropython libraries.

coayer
Posts: 4
Joined: Sat Oct 02, 2021 7:40 pm

Re: uasyncio task freezing

Post by coayer » Wed Oct 13, 2021 6:46 pm

Thanks for the advice — I've swapped over to using

Code: Select all

await asyncio.wait_for(reader.read(X), Y)
but it isn't raising a TimeoutError even if the socket is connected to an unreachable IP (the program hangs on that line). Is this the appropriate way to use timeouts on nonblocking sockets with asyncio?

On the Python docs, it says
Changed in version 3.7: When aw is cancelled due to a timeout, wait_for waits for aw to be cancelled. Previously, it raised asyncio.TimeoutError immediately.
which sounds like the problem.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uasyncio task freezing

Post by pythoncoder » Thu Oct 14, 2021 8:38 am

coayer wrote:
Wed Oct 13, 2021 6:46 pm
Is this the appropriate way to use timeouts on nonblocking sockets with asyncio?...
I thought you were using blocking sockets. You can't use timeouts with nonblocking sockets as read(n) always returns immediately. With nonblocking sockets you need to explicitly code timeouts. The code here is an example.
On the Python docs, it says
Changed in version 3.7: When aw is cancelled due to a timeout, wait_for waits for aw to be cancelled. Previously, it raised asyncio.TimeoutError immediately.
which sounds like the problem.
MicroPython behaves as described here.
Peter Hinch
Index to my micropython libraries.

coayer
Posts: 4
Joined: Sat Oct 02, 2021 7:40 pm

Re: uasyncio task freezing

Post by coayer » Thu Oct 14, 2021 11:41 am

pythoncoder wrote:
Thu Oct 14, 2021 8:38 am
coayer wrote:
Wed Oct 13, 2021 6:46 pm
Is this the appropriate way to use timeouts on nonblocking sockets with asyncio?...
I thought you were using blocking sockets. You can't use timeouts with nonblocking sockets as read(n) always returns immediately. With nonblocking sockets you need to explicitly code timeouts. The code here is an example.
On the Python docs, it says
Changed in version 3.7: When aw is cancelled due to a timeout, wait_for waits for aw to be cancelled. Previously, it raised asyncio.TimeoutError immediately.
which sounds like the problem.
MicroPython behaves as described here.
Ah, I see. Thanks for your help ­— this is my first time working with async programming so it's still a bit confusing!

ozzkuma
Posts: 2
Joined: Thu Jul 01, 2021 10:22 am

Re: uasyncio task freezing

Post by ozzkuma » Fri Oct 15, 2021 10:02 am

coayer wrote:
Sat Oct 02, 2021 7:54 pm
Hi everyone,

I have a uasyncio task running on an ESP8266 which sends DNS lookups (UDP). I'm using uasyncio.StreamReader (and writer) on the socket object to do this asynchronously. Most requests are successful, but sometimes the task indefinitely hangs on "await reader.read(128)", which is reading the response from the server VPower 777 APK. There's a timeout set on the socket.

I have a main() task with its own infinite loop, and I can see that when the DNS read gets stuck the main loop keeps running just fine.

If I press CTRL-C, I get:

Code: Select all

Traceback (most recent call last):
  File "main.py", line 290, in <module>
  File "uasyncio/core.py", line 1, in run
  File "uasyncio/core.py", line 1, in run_until_complete
  File "uasyncio/core.py", line 1, in wait_io_event
KeyboardInterrupt: 
Am I missing something or is this a bug I should report?
I am using getaddrinfo and doing DNS lookup manually.
Last edited by ozzkuma on Sat Sep 30, 2023 7:00 am, edited 1 time in total.

AntoniaUnderwood
Posts: 2
Joined: Fri Aug 19, 2022 3:18 am

Re: uasyncio task freezing

Post by AntoniaUnderwood » Fri Aug 19, 2022 4:30 am

Do you know any way to DNS automatically?

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: uasyncio task freezing

Post by jimmo » Fri Aug 19, 2022 4:45 am

AntoniaUnderwood wrote:
Fri Aug 19, 2022 4:30 am
Do you know any way to DNS automatically?
Not sure what you mean by automatically, but it's straightforward to use "getaddrinfo"

Code: Select all

addr_info = socket.getaddrinfo("micropython.org", 80)
If you're using a higher-level library like urequests, then it happens automatically

Code: Select all

response = urequests.get("http://micropython.org/")

Post Reply