Page 1 of 3

persistent wifi/mqtt connections or not

Posted: Sat Nov 16, 2019 3:30 pm
by janol
I have a few esp32 boards with bme280 and ds1820 sensors scattered around a farm to monitor temperatures and humidity. Each esp32 has a oled screen showing the data picked up by the sensors on the local esp board. Currently the oled is on all the time and updated every 5 seconds. The esp32:s are connected via wifi and sends update on their data via mqtt every 5 minutes. Then I have a pi-board acting as a mqtt broker and also running a mqtt client that reads all mqtt data and puts the data from all esp-boards in a central data base.

But I have problems with the reliabilty of the esp32:s and this is probably due to my coding -- I am new to both esp32 and python.
I'm speculating that it could be unreliable wifi- and/or mqtt-connecctions that makes my esp32 hang. Given that I send mqtt messages only every 5 minutes, I wonder if I should close the wifi after each message and then reconnect next time around. Is there a rule of thumb that below a certain frequency there is no point of trying to have a persistent connection?

Same question for my mqtt connection: Should I repeat the mqtt connect this every time I want to publish a mqtt message (currently every 5 min)? Any rule of thumb/best practice?

Thanks for any guidance on this.

/Fredrik

Re: persistent wifi/mqtt connections or not

Posted: Sat Nov 16, 2019 4:01 pm
by kevinkk525
You should check out the asynchronous mqtt library from Peter Hinch: https://github.com/peterhinch/micropython-mqtt
This will keep your esp32 connected reliably.

Re: persistent wifi/mqtt connections or not

Posted: Sat Nov 16, 2019 7:32 pm
by janol
Thanks for the the reference. Actually I had started to look at Peter's mqtt library. My conclusion so far is that this library is where I should end up but it feels like a too big step for my coding skills right now. Also, will this library just help me cover up bad coding practices? Anyway good to get another push to wrap my head around some new things.

Re: persistent wifi/mqtt connections or not

Posted: Sat Nov 16, 2019 8:30 pm
by kevinkk525
This library won't cover up bad coding practices. Maintaining a resilient mqtt connection is not easy so using this library is actually the best way to have a resilient mqtt connection.
A smaller step would be to use umqtt.robust if you are not yet familiar with asynchronous programming.

Re: persistent wifi/mqtt connections or not

Posted: Sat Nov 16, 2019 9:05 pm
by janol
Thanks again. Will have a look. A smaller step sounds right for me!

/Fredrik

Re: persistent wifi/mqtt connections or not

Posted: Sun Nov 17, 2019 8:49 am
by pythoncoder
I wrote the resilient MQTT library because the official solutions do not reliably cope with WiFi outages. Such outages occur from time to time, especially if your devices are a long way from the access point (because the radio signal is weak). Even close to the AP they do occasionally occur, perhaps caused by sporadic radio interference.

In answer to your original question, my resilient MQTT library aims to keep the WiFi connection open all the time. This is because applications may subscribe, and messages may arrive at any time. If you only ever publish using qos==0, you could close the connection after each publication. This would be a bad idea with qos==1 because the library needs to handle retransmissions and this process may continue until a PUBACK has successfully been received.

In general I can see no particular merit in closing the connection after each message.

Re: persistent wifi/mqtt connections or not

Posted: Mon Nov 18, 2019 8:21 pm
by janol
Thanks for input and guidance. What I'm trying to do is very simple: reading some temp and humidity sensors at various places and store these in a data base for plotting graphs/further analysis. I can definitely live with qos==0. If I miss a reading there is another one coming in a few minutes. But I definitely want the code to continue to run so that dataflows continue once the connection (or what it might be) is back. I'm currently trying to track down what causes the freeze (oled screen, wifi connection, mqtt connection, or...) that sometime comes after 1h and some other time after 2 days... But even after a freeze I can still see the esp32 connected to my wifi network and given that the wifi is good where I'm now testing I suspect that the problem is elsewhere.

/Fredrik

Re: persistent wifi/mqtt connections or not

Posted: Tue Nov 19, 2019 8:06 am
by pythoncoder
Are you sure your source of power is reliable? Some USB wall warts are OK for charging cellphones but crap for running hardware.

Re: persistent wifi/mqtt connections or not

Posted: Tue Nov 19, 2019 7:43 pm
by janol
Sure is a big word for me but the board with which I have experienced most reliability problem has been powered with something that looks ok and with rated output 5V 2A so should be OK, no? Last night (24 hours ago) I did change the code to do the MQTTclient.connect and disconnect just before and after each mqtt publish statement (i.e. about every 5 minutes). It has worked for 24h which is unusally long but powered via the RPi 3B. I will now move it back to the basement (slightly worse wifi and with the initial power supply described) to see if it still continues to run.

/Fredrik

Re: persistent wifi/mqtt connections or not

Posted: Wed Nov 20, 2019 9:13 am
by pythoncoder
The rating of the wall wart tells you little. The problem is reliability. You can charge a cellphone battery even if the PSU has occasional glitches, perhaps caused by electrical noise on the mains. Hardware devices will crash. I find wall warts designed for continuous running are best - e.g. the ones sold to power Raspberry Pi's.