ESP32 DOITv1 urequests.post issue

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
Post Reply
novaproejct
Posts: 3
Joined: Sun Feb 16, 2020 6:08 pm

ESP32 DOITv1 urequests.post issue

Post by novaproejct » Sun Feb 16, 2020 6:26 pm

Hello guys :)
I have a very strange issue. I have searched a lot about this, but I haven't found any working solution on this.
Now, let me explain. I have an ESP32 DoIT devkit v1 board with micropython esp32-idf3-20180511-v1.9.4 firmware.
The issue I have is with this:

Code: Select all

import urequests as rq
import ujson as json

def read_sensor():
  global temp, hum
  temp = hum = 0
  try:
    sensor.measure()
    temp = sensor.temperature()
    hum = sensor.humidity()
    if (isinstance(temp, float) and isinstance(hum, float)) or (isinstance(temp, int) and isinstance(hum, int)):
      msg = (b'{0:3.1f},{1:3.1f}'.format(temp, hum))
      hum = round(hum, 2)
      return(msg)
    else:
      return('Invalid sensor readings.')
  except OSError as e:
    return('Failed to read sensor.')
    
def collect():
  data = json.dumps({"device":"device01","temp":str(temp),"hum":str(hum)})
  return data
 
while True:
	read_sensor()
	measures = collect()
	response = rq.post("http://blabla:9000/hooks/collector", data = measures.encode('utf-8'), headers = { "content-type": "application/json" })
	time.sleep(30)
The problem is that this code actually works, but the while loop executes successfully only a couple of times (around 9) and then fails.
I am also managing the target server and I can confirm that during the first couple of calls, there is a successful data transfer. But then something happens and it fails with:

Code: Select all

Traceback (most recent call last):
  File "main.py", line 48, in <module>
  File "rq.py", line 115, in post
  File "rq.py", line 56, in request
OSError: 23
MicroPython v1.9.4 on 2018-05-11; ESP32 module with ESP32
Previously I tried to post my measures payload as string, and acted with the same behavior. Couple of calls were fine, but then crashes.
When I try parsing measures payload as dict object, it fails instantly.

EDIT: Regardless how many times I try, it always fails on the 9th post request...

Can you help on this? I am literally looping around and can't find a solution...
Thank you!

novaproejct
Posts: 3
Joined: Sun Feb 16, 2020 6:08 pm

Re: ESP32 DOITv1 urequests.post issue

Post by novaproejct » Sun Feb 16, 2020 9:31 pm

Tried with the newest firmware, same story:

Traceback (most recent call last):
File "main.py", line 47, in <module>
File "urequests.py", line 111, in post
File "urequests.py", line 56, in request
OSError: 23
MicroPython v1.12-169-gbaf11f237 on 2020-02-16; ESP32 module with ESP32

User avatar
jimmo
Posts: 1449
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: ESP32 DOITv1 urequests.post issue

Post by jimmo » Sun Feb 16, 2020 11:38 pm

This won't fix the issue, but it will help with memory usage -- try changing the while loop to:

Code: Select all

def read_and_send():
        read_sensor()
	measures = collect()
	response = rq.post("http://blabla:9000/hooks/collector", data = measures.encode('utf-8'), headers = { "content-type": "application/json" })

while True:
        read_and_send()
	time.sleep(30)
But I suspect your actual issue is that you never do anything with "response", so the socket never gets closed (this leads to OSError(23) which is that there's a limit on the number of open sockets).

So either add

Code: Select all

response.close()
or if you access any of the properties on response (e.g. response.content, response.text, or response.json() ) then it'll read the data from the scoket at close it at the end.

(This is a surprising feature of urequests that I hadn't noticed before!!)

User avatar
jimmo
Posts: 1449
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: ESP32 DOITv1 urequests.post issue

Post by jimmo » Sun Feb 16, 2020 11:39 pm

novaproejct wrote:
Sun Feb 16, 2020 9:31 pm
Tried with the newest firmware, same story:
Cool, thanks for re-posting to confirm this (I was going to suggest that otherwise).

I think the fact that it's OSError(23) "'Too many open files in system'" combined with the fact that it happens exactly on the 9th request every time, makes the explanation above pretty likely.

novaproejct
Posts: 3
Joined: Sun Feb 16, 2020 6:08 pm

Re: ESP32 DOITv1 urequests.post issue

Post by novaproejct » Mon Feb 17, 2020 8:47 am

jimmo wrote:
Sun Feb 16, 2020 11:39 pm
novaproejct wrote:
Sun Feb 16, 2020 9:31 pm
Tried with the newest firmware, same story:
Cool, thanks for re-posting to confirm this (I was going to suggest that otherwise).

I think the fact that it's OSError(23) "'Too many open files in system'" combined with the fact that it happens exactly on the 9th request every time, makes the explanation above pretty likely.
Hey Jimmo! THank you mate! response.close() solved the issue...
I haven't noticed that behaviour as well. I have other post requests and they are not failing, but they indeed have data processing of the response... :)
And thanks for the memory advise, I will optimize my code right away.

Post Reply