Config via MQTT

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
david_syd_au
Posts: 2
Joined: Thu Dec 13, 2018 10:04 pm

Config via MQTT

Post by david_syd_au » Thu Dec 13, 2018 10:46 pm

As part of deploying a set of ESP8266 based devices (Wemo D1 mini) for my home automation system, I wanted a way to be able to re-configure them as needed without having to physically connect to each device.
Given they would be connected to a wireless access point and communicating via MQTT, I thought it worthwhile investigating managing their configurations via MQTT.
Below are some code snippets I have produced, along with explanations.
Your feedback on this approach and the code snippets is welcome.

I have created a class JsonStore which can save and load class variables to a JSON file. The class variables are accessed using the "__dict__" attribute, and as I didn't want the JsonStore variables themselves to be visible, ie "file_path", I have effectively hidden it in a private class. Here is the simplified code for JsonStore (ie no error handling):
import json
class JsonStore:
class vars:
file_path = None
_vars = vars()

def __init__(self, file_path):
self._vars.file_path = file_path

def update(self, params):
for i in params:
setattr(self, i, params[i])

def load(self):
file = open(self._vars.file_path)
self.update(json.load(file))
file.close()

def save(self):
file = open(self._vars.file_path, "w")
json.dump(self.__dict__, file)
file.close()

JsonStore is then inherited by the actual app configuration class:

class Config(JsonStore):
ID = hexlify(machine.unique_id()).format("08X")
SERIAL = 0
REBOOT = False
CONFIG_IN_TOPIC = "home/config/" + ID + "/set"
CONFIG_OUT_TOPIC = "home/config/" + ID + "/status"

Further configuration data is stored in a JSON file. Here are the contents of file "config.json" with '*' in place of data for obvious reasons :-):

"WLAN_ESSID": "**********************",
"WLAN_PASSWORD": "**************",
"MQTT_BROKER": "******************",
"MQTT_PORT": 1833,
"MQTT_USER": "**********",
"MQTT_PASSWORD": "******************"

The intiialisation code in my micropython app does:

config = Config("config.json")
config.load()

Once that is done, the variables defined in "config" are enough for the app to connect to a wireless access point and then to the chosen mqtt broker.
A MQTT message handler is registered which handles the MQTT configuration message so:

def mqtt_callback(topic, msg):
s_topic = bytes.decode(topic)
if (config.CONFIG_IN_TOPIC == s_topic):
mqtt_config(msg)
...

def mqtt_config(msg):
params = json.loads(msg)
if 'SERIAL' in params:
if params['SERIAL'] > config.SERIAL or (params['SERIAL'] == 0 and params['SERIAL'] != config.SERIAL):
config.REBOOT = False
config.update(params)
config.save()
mqtt.publish(config.CONFIG_OUT_TOPIC, '{"SERIAL":'+str(config.SERIAL)+'}')
if config.REBOOT:
time.sleep(5) # needed so mqtt publish can complete
mqtt.disconnect()
machine.reset()

An MQTT subscription is done for the config topic, and if the SERIAL value has the default value of 0, the app then waits for an initial MQTT configuration message:

mqtt.subscribe(config.CONFIG_IN_TOPIC)
if config.SERIAL == 0:
print("Waiting for initial configuration")
while config.SERIAL == 0:
mqtt.check_msg()
time.sleep(1)

That MQTT initialisation message can redefine any of the existing config values and/or add new values required for the specific device operation.
For instance the MQTT message JSON formatted content could be:

{
"SERIAL": 1,
"TIMEZONE": 11,
"NTP_HOST": "pool.ntp.org",
"RTC_UPDATE_PERIOD": 3600,
"IN_TOPIC": "home/test/switch/set",
"OUT_TOPIC": "home/test/switch/status"
}

This provides the additional configuration parameters required in this device case to allow it to manage it's RTC, and to provide an MQTT interface for a connected switch. The new values will be saved to the JSON file.
Note the serial number is set to 1, so that the configuration message handler will update the device configuration.

If a little later I decided for instance to change the timezone, I could then send the MQTT message:
{
"SERIAL": 2,
"TIMEZONE": 10,
"REBOOT": true
}

This would update the config TIMEZONE value, which would be saved to the "config.json" file, and then reboot the device to allow the new configuration to take effect. I could have added more code to identify the changed value and use that new value without requiring a reboot, but in this case a reboot is acceptable.

Variables defined in the Config class and not updated via MQTT will not be stored to the JSON file. However if one of them is updated, for example "config.CONFIG_IN_TOPIC", that will be written to the JSON config file and will then over-ride the default value as a result of the "config.load()" call.

Thoughts please?

kevinkk525
Posts: 358
Joined: Sat Feb 03, 2018 7:02 pm

Re: Config via MQTT

Post by kevinkk525 » Fri Dec 14, 2018 10:17 am

If you don't need updating configuration during runtime after the initial configuration has been received (as this would be very difficult as you have to unload the initial configuration first), you can have a look at my smarthome framework:
https://github.com/kevinkk525/pysmartnode
Changing configuration during runtime could be possible if the device just resets itself to start again with a new configuration but then you'd have a mqtt topic that will reset your devices but that could be ok.

It does exactly what you are asking for. On boot the microcontroller connects to mqtt and with a configuration server running in the network it will receive its configuration and then import modules, create classes and start functions accordingly.

It is written with uasyncio completely (unlike your code) but the documentation should make it pretty easy to add your custom components. Or maybe you will just see some code that gives you new ideas to improve your approach.

Post Reply