Bringing MQTT to generic MicroPython boards

Discuss development of drivers for external hardware and components, such as LCD screens, sensors, motor drivers, etc.
Target audience: Users and developers of drivers.
Post Reply
User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Bringing MQTT to generic MicroPython boards

Post by pythoncoder » Mon Jun 19, 2017 7:56 am

MQTT is a simple networking protocol aimed at internet of things devices and is ideal for monitoring and control applications. Some notes on MQTT: viewtopic.php?f=16&t=2239.

Project URL: https://github.com/peterhinch/micropython-mqtt.git.

This project uses a cheap ESP8266 board to bring MQTT to a MicroPython board lacking WiFi connectivity. The repo includes a binary firmware build for the ESP8266 so no ESP8266 toolchain, compilation or know-how is required. The host board runs the supplied MicroPython driver. It is designed to work with any MicroPython host having five spare GPIO lines. No other hardware is assumed.

I started this project over a year ago but shelved it owing to instability of the ESP8266 firmware. This now works well, so I've revived the project. It uses uasyncio so some familiarity with asynchronous (event driven) programming is worthwhile. It has substantial provision for error recovery including issuing a hardware reboot to the ESP8266 if it crashes or becomes unresponsive.

All source code is provided so those wishing to build or modify the ESP8266 code can do so. The UARTs on the ESP8266 are unused leaving them available for debug or other purposes.
Peter Hinch
Index to my micropython libraries.

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Bringing MQTT to generic MicroPython boards

Post by deshipu » Mon Jun 19, 2017 8:59 am

Five pins! This is an interesting protocol, with two clock lines. What is it? Is there any particular reason for choosing that over the standard SPI or I2C buses? Can those pins be shared with the SPI bus?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Bringing MQTT to generic MicroPython boards

Post by pythoncoder » Mon Jun 19, 2017 1:06 pm

The protocol is described here https://github.com/peterhinch/micropyth ... /syncom_as. The pins can't be shared but can be reassigned: the pins used on the host are entirely arbitrary and specified in the user code. Those on the ESP8266 are also arbitrary but if pin 15 is re-assigned a pulldown is required. Anyone reassigning these would be changing the code and building firmware.

tl;dr summary. The ESP8266 UART was to be left free for debugging or other uses. I wanted a protocol which was asynchronous at the software level but synchronous at the hardware level. The latter enables it to work regardless of processor speed. It was to be pure Python and to be device independent. The host was to be able to detect that the ESP8266 has stopped responding and physically reset it (hence the fifth line). Operation was to be full-duplex.

The protocol is trivial. Data bits are clocked out on each edge of a unit's clock out. The clock in acts as an acknowledge for the clock out. Once initialised and synchronised it is entirely symmetrical both at a hardware and software level. At a hardware level it runs continuously sending null data when there is nothing to send. This enables either end to respond to unscheduled data.

I considered and rejected several more obvious solutions before inventing my own. In each case the objection was the excruciating latency of the ESP8266 interrupt mechanism, which I believe makes Python solutions impossible. It may even make reliable C solutions impossible. Those considered and rejected were I2C and SPI slave modes and a soft UART.

The obvious drawback is that it's not particularly fast. However in the context of MQTT there are many sources of delays and I consider its advantages outweigh this rather theoretical drawback. It would be relatively easy to replace it with hardware UARTs and a reset line.
Peter Hinch
Index to my micropython libraries.

User avatar
TravisT
Posts: 72
Joined: Sun Feb 23, 2014 2:31 pm
Location: Portland, OR
Contact:

Re: Bringing MQTT to generic MicroPython boards

Post by TravisT » Mon Jun 19, 2017 1:10 pm

This is interesting to me, I like making the ESP8266 as a simple MQTT "Peripheral" and let the main processor do the fancy stuff. With how crazy cheap the ESP8266 is, but how limited it is in IO, this makes a lot of sense.
_______________
Travis Travelstead

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Bringing MQTT to generic MicroPython boards

Post by pythoncoder » Mon Jun 19, 2017 4:05 pm

Quite. Another factor is performance. The Pyboard's Python code responds to a hard interrupt in a few μs. The ESP8266 can take many hundreds. I haven't tested it under heavy load but its worst-case latency is unknown; ms? It's frankly poor for hardware interfacing. Its ADC is inaccurate and its one bidirectional UART spews garbage on reset necessitating a hardware hack to actually use it for anything other than debugging. But (at long last) it does one thing really well: WiFi access. And it's cheap.

The Pyboard has far more hardware peripherals. It has an RTC which keeps decent time out of the box and can be calibrated to timepiece accuracy. It has plenty of hardware timers with serious capabilities, multiple accurate ADC's, DAC's and more besides... It's in a different ballpark and is an amazing piece of kit.

Horses for courses.
Peter Hinch
Index to my micropython libraries.

User avatar
marfis
Posts: 215
Joined: Fri Oct 31, 2014 10:29 am
Location: Zurich / Switzerland

Re: Bringing MQTT to generic MicroPython boards

Post by marfis » Mon Jun 19, 2017 6:54 pm

interesting read, thanks for sharing this.

Post Reply