Microhomie - MicroPython implementation of the Homie convention

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
Rafael
Posts: 3
Joined: Mon Jan 22, 2018 8:58 pm

Microhomie - MicroPython implementation of the Homie convention

Post by Rafael » Sat Feb 10, 2018 10:50 am

Hello MicroPython community!

I want to announce a small project we call Microhomie, a framework to easily setup devices following the Homie (https://github.com/marvinroger/homie) lightweight MQTT convention for IoT. We also have some example nodes which can be used out of the box to publish data, i.e. for the DHT22 or the SDS011 sensors.

Website: https://www.microhomie.com
Github: https://github.com/microhomie/micropython-homie

And also on PyPi: https://pypi.python.org/pypi/micropython-homie

We are new into the exciting MicroPython cosmos and still learning, feedback is very appreciated!

User avatar
pythoncoder
Posts: 3053
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Microhomie - MicroPython implementation of the Homie convention

Post by pythoncoder » Sun Feb 11, 2018 8:14 am

An interesting approach. I don't know how much testing you've done but I found that getting reliable results from MQTT over WiFi was non-trivial. The problem is that packet delivery on a wireless network is not guaranteed, and the official umqtt library ignores this. For example a publish with qos==1 will hang forever if the PUBACK is not received. There are other shortcomings of the official library. It doesn't ping the broker - this should be done periodically if the connection is to be kept open, especially in subscribe-only systems or those which publish rarely.

One instructive test is to move a system to, and beyond, the limits of WiFi communication and see if it recovers after it returns within range. In my testing this was a more demanding test than temporarily downing the AP. Even if physical devices are in fixed locations, connectivity can still be disrupted by RF interference. In my testing, if the code can survive repeatedly being taken out of range it seemed resilient over days and weeks of normal operation.

You might like to look at my efforts on this.
Peter Hinch

Rafael
Posts: 3
Joined: Mon Jan 22, 2018 8:58 pm

Re: Microhomie - MicroPython implementation of the Homie convention

Post by Rafael » Sun Feb 11, 2018 11:07 am

Hi Peter,

thank you very much for your input.

[quote=pythoncoder post_id=25365 time=1518336869 user_id=265]
An interesting approach. I don't know how much testing you've done but I found that getting reliable results from MQTT over WiFi was non-trivial. The problem is that packet delivery on a wireless network is not guaranteed, and the official umqtt library ignores this. For example a publish with qos==1 will hang forever if the PUBACK is not received. There are other shortcomings of the official library. It doesn't ping the broker - this should be done periodically if the connection is to be kept open, especially in subscribe-only systems or those which publish rarely.
[/quote]

We try to catch MQTT exceptions and re-connect if we can not send mqtt pakage to the broker. The Homie convention use a '$stats/uptime' topic for the device uptime. Every minute the uptime will be published to the broker. I hope this is good enough for a periodically ping.

The main-loop should trigger wifi setup, which re-connect to wifi if wlan.isconnected() return False.

But as you mentioned, all this workarounds fail if qos==1 hang forever. The only thing we had in mind to workaround is to setup a watchdog timer, and feed it from the main-loop.

[quote=pythoncoder post_id=25365 time=1518336869 user_id=265]
You might like to look at [url=https://github.com/peterhinch/micropython-mqtt] my efforts on this[/url].
[/quote]

I already follow your work on your mqtt implementation and find it very interesting! We'll definitely try your implementation with Microhomie. For the first release we wanted to stay with the official library.

User avatar
pythoncoder
Posts: 3053
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Microhomie - MicroPython implementation of the Homie convention

Post by pythoncoder » Sun Feb 11, 2018 1:23 pm

When you refer to a watchdog is this hardware or firmware based? Publishing once a minute should be fine for keeping the broker alive.

The acid test for all of this is testing. I'd be very interested to hear the outcome of a few runs of a WiFi range test. Also the outcome of long-term (many days) running within range.
Peter Hinch

Rafael
Posts: 3
Joined: Mon Jan 22, 2018 8:58 pm

Re: Microhomie - MicroPython implementation of the Homie convention

Post by Rafael » Tue Feb 13, 2018 8:14 am

I thought about the firmware watch dog but I overlooked the availability of this class is only for pyboard and WiPy as mentioned in the documentation for the esp8266.

Yesterday I started to monitor the device uptime and signal from two esp8266 devices. One device is indoor, the other one is in our garage. I've attached a first screenshot. The outage from the Catroom device was a manual device reset.
Attachments
Screenshot-2018-2-13 Grafana - Homie.png
Screenshot-2018-2-13 Grafana - Homie.png (145.1 KiB) Viewed 619 times

User avatar
marfis
Posts: 166
Joined: Fri Oct 31, 2014 10:29 am
Location: Zurich / Switzerland

Re: Microhomie - MicroPython implementation of the Homie convention

Post by marfis » Tue Feb 13, 2018 8:14 pm

interesting project!

Regarding watchdog functionality across platforms. You could try a periodic timer irq callback. Countdown a global counter there and if it reaches 0, reset the cpu.

Timers and periodic irqs should be available on pretty much every platform.

In the foreground process you can periodically set the counter to the initial value.

User avatar
pythoncoder
Posts: 3053
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Microhomie - MicroPython implementation of the Homie convention

Post by pythoncoder » Wed Feb 14, 2018 10:55 am

A firmware watchdog can never be as bomb-proof as a hardware one. For example if the machine crashes and executes gibberish it could disable interrupts.
Peter Hinch

Post Reply