Page 1 of 2
how to make urequests asynchronous?
Posted: Thu Jan 27, 2022 11:49 am
by kenjican
I used to post json by urequests module. Now I need it to be none blocking for efficiency.
I tried uaiohttpclient ,but it seems method get only, no post method. And wrap urequest.post in generator , it doesn't work either. Still blocked.
If ascyncio could make urequests to be async?
Re: how to make urequests asynchronous?
Posted: Thu Jan 27, 2022 8:29 pm
by KJM
urequests normally supplies output via return, if you make it non blocking what are your expectations for how to receive it's output? I've been able to time out urequests (so my main script doesn't hang waiting for it) using either _thread or uasyncio but both required some mods to the urequests lib to replace the return with an alternative.
Re: how to make urequests asynchronous?
Posted: Fri Jan 28, 2022 5:18 am
by kenjican
My expectation is fire and forget. Response is not necessary. Post method of urequests module doesn't have timeout option. Async is also a good alternative.
Could you please show me how to modify urequests.py ?
Re: how to make urequests asynchronous?
Posted: Fri Jan 28, 2022 5:51 am
by KJM
If you don't need the response from request there is no need to mod it I guess. If you're doing posts only, we just need to mod the calling function underneath it in the lib.
Code: Select all
def post(url, **kw): import _thread; _thread.start_new_thread(request, ("POST", url, data))
alternatively you could try
Code: Select all
def post(url, **kw): import uasyncio as asyncio; asyncio.run(request("POST", url, data))
then edit
Code: Select all
def request(method, url, data=None)
to
Code: Select all
async def request(method, url, data=None)
I run a modified urequests as a function in my main.py, I'm not sure if these suggestions will work on urequests imported as a lib. If it doesn't work let me know & we can try something different.
Re: how to make urequests asynchronous?
Posted: Sat Jan 29, 2022 5:30 am
by kenjican
I've tried both.
1: _thread. It is blocking also. I think urequests is block in the new thread.
2: asyncio. I test it in REPL, it throws out message: "coroutine expected"
Now, I'm trying to mod urequests . My idea is to make s.read() awaitble...
Re: how to make urequests asynchronous?
Posted: Sat Jan 29, 2022 7:22 am
by KJM
Blocking your main program form inside a thread is weird, are you running 1.17 upython on an ESP32?
Re: how to make urequests asynchronous?
Posted: Sat Jan 29, 2022 8:46 am
by kenjican
I'm running 1.18 on ESP32 Wrover. As I know, threads run in turns. So,,,blocking should reasonable.
Now, I'm trying uayncio.open_connection, it would return awaitable socket. Therefor, We should modify urequests.py to make sockets awaitable.
Re: how to make urequests asynchronous?
Posted: Sat Jan 29, 2022 2:29 pm
by kenjican
Let's make a asyncio socket awaitable, see this
https://docs.python.org/3/library/asyncio-stream.html in python docs
- First step: mock a tcp server: netcat -lk -p 8888 , we could see if ESP32 send message
- Second step: chekc if socket established: watch 'ss -tp| grep 100.235' 192.168.100.235 is IP of ESP32
- Third step: run asyncio socket in ESP32
mock tcp server ip is : 192.168.100.230
Code: Select all
import uasyncio as asyncio
async def test(ip):
sReader,sWriter = await asyncio.open_connection(ip,8888)
sWriter.write("test async socket write data")
asyncio.run(test("192.168.100.230"))
In this command : watch 'ss -tp| grep 100.235' , we could see the socket established
But in netcat , there is no message comes in.
I'm not sure whether the code is wrong or it's a bug of uasyncio
Re: how to make urequests asynchronous?
Posted: Sat Jan 29, 2022 5:32 pm
by curt
Have you considered UDP? It is non-blocking and much faster than TCP.
Curt
Re: how to make urequests asynchronous?
Posted: Sat Jan 29, 2022 7:37 pm
by marcidy
kenjican wrote: ↑Sat Jan 29, 2022 2:29 pm
Let's make a asyncio socket awaitable, see this
https://docs.python.org/3/library/asyncio-stream.html in python docs
- First step: mock a tcp server: netcat -lk -p 8888 , we could see if ESP32 send message
- Second step: chekc if socket established: watch 'ss -tp| grep 100.235' 192.168.100.235 is IP of ESP32
- Third step: run asyncio socket in ESP32
mock tcp server ip is : 192.168.100.230
Code: Select all
import uasyncio as asyncio
async def test(ip):
sReader,sWriter = await asyncio.open_connection(ip,8888)
sWriter.write("test async socket write data")
asyncio.run(test("192.168.100.230"))
In this command : watch 'ss -tp| grep 100.235' , we could see the socket established
But in netcat , there is no message comes in.
I'm not sure whether the code is wrong or it's a bug of uasyncio
You need to drain the stream after writing to actually send. write adds to the internal buffer, stream sends the buffer over the socket.
I have a few examples here which may be helpful:
https://github.com/marcidy/micropython- ... webexample
The http examples are a server but the code may help you along.