Reliability test results: Pyboard D and MQTT

The official PYBD running MicroPython, and its accessories.
Target audience: Users with a PYBD
Post Reply
User avatar
pythoncoder
Posts: 4173
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Reliability test results: Pyboard D and MQTT

Post by pythoncoder » Fri Nov 01, 2019 5:41 pm

tl;dr For serious reliability the Pyboard D seems to be the best platform for WiFi applications.

Using the asynchronous resilient MQTT mqtt_as.py in this repo I am running long term tests. To date these have accumulated 30 days running in one location with RSSI -56dB and 14 days in another with RSSI -83dB - in our garden shed far from the AP.

These tests are still running without failure. There is no watchdog timer in use.

The test script exchanges messages every 5s with qos==1. The library performs re-publications in the event of WiFi disruption and recovers from WiFi outages with no qos==1 data loss. The test script reports outages as seen by the Pyboard and also maintains and reports the repub count.

At the -56dB location in 30 days two outages occurred with four repubs - in over 500K publications. At the -83dB location in 14 days it saw one outage and 21 repubs.

I have no comparable quantitative data for ESP8266 or ESP32 but I have done a good deal of testing with them. The ESP8266 in particular experiences outages much more frequently. I believe it occasionally crashes, necessitating a WDT, but intend to re-test to confirm this.

If anyone wishes to replicate these tests it is vital to have a high quality power supply. Some USB wall-warts are poor and cause crashes on any platform.
Peter Hinch

kevinkk525
Posts: 711
Joined: Sat Feb 03, 2018 7:02 pm

Re: Reliability test results: Pyboard D and MQTT

Post by kevinkk525 » Sun Nov 03, 2019 1:15 pm

That sounds promising. -83dBm is a quite weak signal.
Thanks for all your testing efforts and patience!
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode

User avatar
tve
Posts: 203
Joined: Wed Jan 01, 2020 10:12 pm
Location: Santa Barbara, CA
Contact:

Re: Reliability test results: Pyboard D and MQTT

Post by tve » Sun Jan 05, 2020 12:38 am

Is your code available somewhere? I'd be interested in testing using an esp32...

User avatar
pythoncoder
Posts: 4173
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Reliability test results: Pyboard D and MQTT

Post by pythoncoder » Sun Jan 05, 2020 10:28 am

The resilient MQTT repo is here. It has been tested on ESP32 for resilience, but the long term tests have so far included only Pyboard D and ESP8266.
Peter Hinch

User avatar
tve
Posts: 203
Joined: Wed Jan 01, 2020 10:12 pm
Location: Santa Barbara, CA
Contact:

Re: Reliability test results: Pyboard D and MQTT

Post by tve » Sun Jan 05, 2020 4:09 pm

I'm aware of the repo, but when you write "The test script exchanges messages every 5s with qos==1" which script do you specifically run?

User avatar
pythoncoder
Posts: 4173
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Reliability test results: Pyboard D and MQTT

Post by pythoncoder » Sun Jan 05, 2020 4:51 pm

I run this one on the unit under test. The mosquitto broker runs on a Raspberry Pi, and I run mosquitto_sub on the Pi to display the results. Another terminal on the Pi runs this script to provide messages for the unit's subscription.

The Pi is just a Linux box which is economical to run 24/7 - any networked Linux box will do.
Peter Hinch

Post Reply