Designing resilient IOT applications

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Post Reply
User avatar
pythoncoder
Posts: 3450
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Designing resilient IOT applications

Post by pythoncoder » Sun Dec 09, 2018 7:47 pm

The work I've done on enabling ESP8266 devices to work for long periods has convinced me of the following:
  • Assuming good quality hardware it is possible to write applications that never crash or produce error messages.
  • Such applications can survive WiFi outages without loss of program state.
  • Brief WiFi outages occur frequently. On bare-metal devices these require explicit handling in code.
  • Achieving this requires a protocol supported by both ends of the interface.
  • Running web applications on the ESP8266 is probably ill-advised if reliability is a priority. This is for a variety of reasons, not least because the protocols don't meet the requirement above.
This repository suggests an alternative way to write internet IOT applications, and also applications which are purely local. This overcomes many of the drawbacks of running internet libraries on ESP8266 devices and implements a protocol capable of detecting and recovering from outages.

The code in the repo has run for long periods on multiple ESP8266 devices without incident.

Comments are welcome.
Peter Hinch

bvwelch
Posts: 1
Joined: Sun Dec 09, 2018 9:12 pm

Re: Designing resilient IOT applications

Post by bvwelch » Sun Dec 09, 2018 9:17 pm

Greetings,
Thanks very much for taking the time to study this and write-up your findings - very timely for me! I will be trying out your approach asap!
William

bitninja
Posts: 115
Joined: Thu Sep 15, 2016 4:09 pm
Location: Spring, Texas

Re: Designing resilient IOT applications

Post by bitninja » Wed Dec 12, 2018 4:55 pm

Impressive! Thank you for an inspiring design!

While you leave the actual message content up to the implementation, have you thought about formulating an additional layer to the protocol that standardizes common data exchange messages that occurs for IOT applications? I'm just thinking that another layer of abstraction may be useful to simplify things further. For one, you could perhaps make all the end-points homogeneous except for the local.py.

Just a thought.

kevinkk525
Posts: 407
Joined: Sat Feb 03, 2018 7:02 pm

Re: Designing resilient IOT applications

Post by kevinkk525 » Wed Dec 12, 2018 5:39 pm

I'm currently working on exactly that. A higher layer for the client to easily hook different apps to it. This will also require a higher level in the server.

I'm also planning to build a mqtt app using that higher layer. Then I'll compare it against my current mqtt client implementation and hope that it will be a lot more RAM friendly.

User avatar
pythoncoder
Posts: 3450
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Designing resilient IOT applications

Post by pythoncoder » Thu Dec 13, 2018 9:17 am

bitninja wrote:
Wed Dec 12, 2018 4:55 pm
...have you thought about formulating an additional layer to the protocol that standardizes common data exchange messages that occurs for IOT applications?...
My aims for this project, in order of importance, were
  • Resilience (obviously).
  • Minimal RAM use. I wanted it to be usable on an ESP8266 without recourse to frozen bytecode or cross-compilation.
  • Generality: to provide an interface not tied to any particular protocol.
My aim was to provide a socket-like interface supporting full-duplex streaming communication which can be used for any purpose (a "socket" which persists through outages).

There was some discussion on GitHub regarding enhancements. Some, like guaranteed message delivery, can readily be accomplished at application level as per my doc/demos. Other enhancements are probably best achieved by subclassing to avoid compromising RAM usage for people wanting to use the base class.

In the applications I envisage, the data being exchanged between the server and the ESP8266 nodes comprises JSON-encoded Python objects. These would contain very low-level information such as sensor readings or required Pin states or PWM values. In consequence the user code on the ESP8266 nodes would be minimal (barring device drivers).

I envisage all the clever internet stuff being done on the server.

This view may be a reflection of my background in hardware/firmware rather than web development. I'll be interested to see what others (e.g. @kevinkk525) come up with.
Peter Hinch

User avatar
pythoncoder
Posts: 3450
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

V0.2 release

Post by pythoncoder » Mon Feb 18, 2019 7:02 pm

This project evolved since my first post and became a collaboration between myself and Kevin Köck. Key features:
  • The API is similar for client and server applications.
  • It is cross-platform supporting ESP8266 and Pyboard D clients.
  • It supports guaranteed message delivery: a message will be delivered exactly once even if a WiFi or server outage takes place (obviously message delivery will be delayed until the outage has cleared).
  • It provides an optional watchdog timer (software on ESP8266, hardware on Pyboard D).
  • There is a means of using an ESP8266 with a fixed firmware build to extend this interface to a Pyboard 1.x. Connectivity between the Pyboard and the ESP8266 is via I2C.
The API has changed since the original release. No further changes are planned and, barring bugfixes, this project is essentially complete.

The ESP32 remains incapable of recovering from outages. It "works" but the interface is not resilient which renders it basically useless. I haven't tested the Loboris port. Any feedback on ESP32 issues is welcome.
Peter Hinch

newb
Posts: 28
Joined: Wed Jan 10, 2018 8:19 pm
Location: Bulgaria

Re: Designing resilient IOT applications

Post by newb » Tue Feb 19, 2019 7:24 am

Hi Peter and Kevin, great stuff and documentation as usual! Have you tried to make the server run on esp8266 as well?

User avatar
pythoncoder
Posts: 3450
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Designing resilient IOT applications

Post by pythoncoder » Tue Feb 19, 2019 12:37 pm

No. It would require significant changes. Worse, it would defeat the object of the design. The aim is to have internet facing applications running on a device with an OS, with an internet connection which is under OS control. Hence the inherent unreliability of a WiFi link affects just one interface and so is relatively easy to manage. The README provides a more detailed justification of the design.
Peter Hinch

kevinkk525
Posts: 407
Joined: Sat Feb 03, 2018 7:02 pm

Re: Designing resilient IOT applications

Post by kevinkk525 » Tue Feb 19, 2019 1:33 pm

As the server is designed to run on micropython too, it should be possible to run the server on the es8266. You would however have to make sure that the connection to the Wifi is handled. I could imagine that it works if you run the client application on it too but connected to the server running on raspberry pi (so that this connection and client library ensures the wifi is connected correctly). Another possibility would be to build a frame like the client library to ensure that the server is started after wifi is connected etc but you can't reconnect the wifi each time a client disconnects.

So although I see possibilities for it, it is not tested nor designed to be used like that. I'm sure you could get it working with some efforts.

Post Reply