persistent wifi/mqtt connections or not

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
Post Reply
janol
Posts: 12
Joined: Wed Oct 02, 2019 7:52 am

persistent wifi/mqtt connections or not

Post by janol » Sat Nov 16, 2019 3:30 pm

I have a few esp32 boards with bme280 and ds1820 sensors scattered around a farm to monitor temperatures and humidity. Each esp32 has a oled screen showing the data picked up by the sensors on the local esp board. Currently the oled is on all the time and updated every 5 seconds. The esp32:s are connected via wifi and sends update on their data via mqtt every 5 minutes. Then I have a pi-board acting as a mqtt broker and also running a mqtt client that reads all mqtt data and puts the data from all esp-boards in a central data base.

But I have problems with the reliabilty of the esp32:s and this is probably due to my coding -- I am new to both esp32 and python.
I'm speculating that it could be unreliable wifi- and/or mqtt-connecctions that makes my esp32 hang. Given that I send mqtt messages only every 5 minutes, I wonder if I should close the wifi after each message and then reconnect next time around. Is there a rule of thumb that below a certain frequency there is no point of trying to have a persistent connection?

Same question for my mqtt connection: Should I repeat the mqtt connect this every time I want to publish a mqtt message (currently every 5 min)? Any rule of thumb/best practice?

Thanks for any guidance on this.

/Fredrik

kevinkk525
Posts: 654
Joined: Sat Feb 03, 2018 7:02 pm

Re: persistent wifi/mqtt connections or not

Post by kevinkk525 » Sat Nov 16, 2019 4:01 pm

You should check out the asynchronous mqtt library from Peter Hinch: https://github.com/peterhinch/micropython-mqtt
This will keep your esp32 connected reliably.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

janol
Posts: 12
Joined: Wed Oct 02, 2019 7:52 am

Re: persistent wifi/mqtt connections or not

Post by janol » Sat Nov 16, 2019 7:32 pm

Thanks for the the reference. Actually I had started to look at Peter's mqtt library. My conclusion so far is that this library is where I should end up but it feels like a too big step for my coding skills right now. Also, will this library just help me cover up bad coding practices? Anyway good to get another push to wrap my head around some new things.

kevinkk525
Posts: 654
Joined: Sat Feb 03, 2018 7:02 pm

Re: persistent wifi/mqtt connections or not

Post by kevinkk525 » Sat Nov 16, 2019 8:30 pm

This library won't cover up bad coding practices. Maintaining a resilient mqtt connection is not easy so using this library is actually the best way to have a resilient mqtt connection.
A smaller step would be to use umqtt.robust if you are not yet familiar with asynchronous programming.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

janol
Posts: 12
Joined: Wed Oct 02, 2019 7:52 am

Re: persistent wifi/mqtt connections or not

Post by janol » Sat Nov 16, 2019 9:05 pm

Thanks again. Will have a look. A smaller step sounds right for me!

/Fredrik

User avatar
pythoncoder
Posts: 4037
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: persistent wifi/mqtt connections or not

Post by pythoncoder » Sun Nov 17, 2019 8:49 am

I wrote the resilient MQTT library because the official solutions do not reliably cope with WiFi outages. Such outages occur from time to time, especially if your devices are a long way from the access point (because the radio signal is weak). Even close to the AP they do occasionally occur, perhaps caused by sporadic radio interference.

In answer to your original question, my resilient MQTT library aims to keep the WiFi connection open all the time. This is because applications may subscribe, and messages may arrive at any time. If you only ever publish using qos==0, you could close the connection after each publication. This would be a bad idea with qos==1 because the library needs to handle retransmissions and this process may continue until a PUBACK has successfully been received.

In general I can see no particular merit in closing the connection after each message.
Peter Hinch

janol
Posts: 12
Joined: Wed Oct 02, 2019 7:52 am

Re: persistent wifi/mqtt connections or not

Post by janol » Mon Nov 18, 2019 8:21 pm

Thanks for input and guidance. What I'm trying to do is very simple: reading some temp and humidity sensors at various places and store these in a data base for plotting graphs/further analysis. I can definitely live with qos==0. If I miss a reading there is another one coming in a few minutes. But I definitely want the code to continue to run so that dataflows continue once the connection (or what it might be) is back. I'm currently trying to track down what causes the freeze (oled screen, wifi connection, mqtt connection, or...) that sometime comes after 1h and some other time after 2 days... But even after a freeze I can still see the esp32 connected to my wifi network and given that the wifi is good where I'm now testing I suspect that the problem is elsewhere.

/Fredrik

User avatar
pythoncoder
Posts: 4037
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: persistent wifi/mqtt connections or not

Post by pythoncoder » Tue Nov 19, 2019 8:06 am

Are you sure your source of power is reliable? Some USB wall warts are OK for charging cellphones but crap for running hardware.
Peter Hinch

janol
Posts: 12
Joined: Wed Oct 02, 2019 7:52 am

Re: persistent wifi/mqtt connections or not

Post by janol » Tue Nov 19, 2019 7:43 pm

Sure is a big word for me but the board with which I have experienced most reliability problem has been powered with something that looks ok and with rated output 5V 2A so should be OK, no? Last night (24 hours ago) I did change the code to do the MQTTclient.connect and disconnect just before and after each mqtt publish statement (i.e. about every 5 minutes). It has worked for 24h which is unusally long but powered via the RPi 3B. I will now move it back to the basement (slightly worse wifi and with the initial power supply described) to see if it still continues to run.

/Fredrik

User avatar
pythoncoder
Posts: 4037
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: persistent wifi/mqtt connections or not

Post by pythoncoder » Wed Nov 20, 2019 9:13 am

The rating of the wall wart tells you little. The problem is reliability. You can charge a cellphone battery even if the PSU has occasional glitches, perhaps caused by electrical noise on the mains. Hardware devices will crash. I find wall warts designed for continuous running are best - e.g. the ones sold to power Raspberry Pi's.
Peter Hinch

Post Reply