Using mqtt broker status messages as connection alive checker
-
- Posts: 67
- Joined: Fri Sep 04, 2020 9:27 am
- Location: Hanko, Finland
- Contact:
Using mqtt broker status messages as connection alive checker
I would like to implement some sort of status checking for the mqtt-connection.
I implemented simple check, where:
...
client.subscribe("$SYS/broker/bytes/#")
and in case:
if (utime.ticks_diff(utime.ticks_ms(), mqtt_last_seen)) > (60 * 60 * 100):
# MQTT-broker have not reported anything within hour, reboot!
but now I am afraid that could broker be silent for more than hour in some occasions?
The broker in this case is Raspberry PI paho-mqtt broker.
I implemented simple check, where:
...
client.subscribe("$SYS/broker/bytes/#")
and in case:
if (utime.ticks_diff(utime.ticks_ms(), mqtt_last_seen)) > (60 * 60 * 100):
# MQTT-broker have not reported anything within hour, reboot!
but now I am afraid that could broker be silent for more than hour in some occasions?
The broker in this case is Raspberry PI paho-mqtt broker.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Using mqtt broker status messages as connection alive checker
The broker will be silent unless you arrange for some code to publish periodically. This could be a Python script running on the Pi publishing every (say) 15 minutes. Incidentally an hour in ms is 60 * 60 * 1000...
You might also consider this library which (amongst other things) detects broker outages by means of MQTT ping packets.
You might also consider this library which (amongst other things) detects broker outages by means of MQTT ping packets.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
-
- Posts: 67
- Joined: Fri Sep 04, 2020 9:27 am
- Location: Hanko, Finland
- Contact:
Re: Using mqtt broker status messages as connection alive checker
Thank you about ms * 1000 correction.pythoncoder wrote: ↑Sun Oct 11, 2020 3:50 pmThe broker will be silent unless you arrange for some code to publish periodically. This could be a Python script running on the Pi publishing every (say) 15 minutes. Incidentally an hour in ms is 60 * 60 * 1000...
You might also consider this library which (amongst other things) detects broker outages by means of MQTT ping packets.
Mosquitto man page https://mosquitto.org/man/mosquitto-8.html says: "Clients can find information about the broker by subscribing to topics in the $SYS hierarchy as follows. Topics marked as static are only sent once per client on subscription. All other topics are updated every sys_interval seconds. If sys_interval is 0, then updates are not sent."
I have default mosquitto setup in my raspberry and it seems to send updates every 10th second anyways. This is ok for me, but perhaps for some low speed connection this type of broadcasting is too much.
I am about to test mqtt_async library with my next low power ESP32U projects. Then I have to wake up ESP32U, measure temp and rh with DHT22 (or perhaps with some I2C device if DHT22 consumes too much energy) and re-establish connection to the mqtt-broker (InfluxDB -> Grafana). In such scenario mqtt connection must be robust, including WiFi connection.
I learned my lesson about WiFi-connections recently (two WiFi subnets, another behind NAT): mqtt publish works fine through NATd network to broker, but other way not. I believed that if the client polls the broker, broker can transfer queued messages at the same time, but this is not true. If client subsribes something, the broker establish connection to the client, and if NAT do not have proper ports open and reverse NAT to client device, connection is dropped. Believing is not knowledge
-
- Posts: 969
- Joined: Sat Feb 03, 2018 7:02 pm
Re: Using mqtt broker status messages as connection alive checker
the broker never establishes a connection with the client. It is always the mqtt client that establishes a connection with the broker and the connection is being used bi-directionally. So if the client successfully connects to the broker, the broker can send messages to the client.
Kevin Köck
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode
Micropython Smarthome Firmware (with Home-Assistant integration): https://github.com/kevinkk525/pysmartnode
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Using mqtt broker status messages as connection alive checker
I don't think the asynchronous library is ideal in that scenario. Its purpose is to maintain a continuous connection for the case where the client subscribes and must be responsive to incoming publications for as much of the time as is possible. Applications which sleep, connect to WiFi and publish before going back to sleep comprise a quite different category.Divergentti wrote: ↑Mon Oct 12, 2020 9:02 am...
I am about to test mqtt_async library with my next low power ESP32U projects. Then I have to wake up ESP32U, measure temp and rh with DHT22 (or perhaps with some I2C device if DHT22 consumes too much energy) and re-establish connection to the mqtt-broker (InfluxDB -> Grafana). In such scenario mqtt connection must be robust, including WiFi connection...
I have considered writing a module for that common use case, but haven't got round to it. On the face of it doing a WiFi status check, followed by a qos==1 publication (which repeats until it gets a PUBACK) should suffice, but I haven't thought this through in detail.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
-
- Posts: 67
- Joined: Fri Sep 04, 2020 9:27 am
- Location: Hanko, Finland
- Contact:
Re: Using mqtt broker status messages as connection alive checker
Ok. I understand. Relay control and such are target for the mqtt_async.pythoncoder wrote: ↑Mon Oct 12, 2020 11:05 amI don't think the asynchronous library is ideal in that scenario. Its purpose is to maintain a continuous connection for the case where the client subscribes and must be responsive to incoming publications for as much of the time as is possible. Applications which sleep, connect to WiFi and publish before going back to sleep comprise a quite different category.Divergentti wrote: ↑Mon Oct 12, 2020 9:02 am...
I am about to test mqtt_async library with my next low power ESP32U projects. Then I have to wake up ESP32U, measure temp and rh with DHT22 (or perhaps with some I2C device if DHT22 consumes too much energy) and re-establish connection to the mqtt-broker (InfluxDB -> Grafana). In such scenario mqtt connection must be robust, including WiFi connection...
I have considered writing a module for that common use case, but haven't got round to it. On the face of it doing a WiFi status check, followed by a qos==1 publication (which repeats until it gets a PUBACK) should suffice, but I haven't thought this through in detail.
I try it with this code https://github.com/divergentti/kotiauto ... x2/main.py (Finnish comments, sorry). I seldomly have socket errors at the mosquitto server side (Raspberry), which are propably due to connectivity issues.
-
- Posts: 67
- Joined: Fri Sep 04, 2020 9:27 am
- Location: Hanko, Finland
- Contact:
Re: Using mqtt broker status messages as connection alive checker
Thank you for your correction.kevinkk525 wrote: ↑Mon Oct 12, 2020 10:15 amthe broker never establishes a connection with the client. It is always the mqtt client that establishes a connection with the broker and the connection is being used bi-directionally. So if the client successfully connects to the broker, the broker can send messages to the client.
So, the NAT is not problem if mqtt client can establish connection to the mqtt broker.
I have strugled with a bad WiFi-connectivity and without Wiresharking believed that the broker needs to establish connection to the registered client, when NAT will be problem.
-
- Posts: 67
- Joined: Fri Sep 04, 2020 9:27 am
- Location: Hanko, Finland
- Contact:
Re: Using mqtt broker status messages as connection alive checker
I have four topics, each in AIHE_RELE_x_y, which I subscribe:pythoncoder wrote: ↑Sun Oct 11, 2020 3:50 pmThe broker will be silent unless you arrange for some code to publish periodically. This could be a Python script running on the Pi publishing every (say) 15 minutes. Incidentally an hour in ms is 60 * 60 * 1000...
You might also consider this library which (amongst other things) detects broker outages by means of MQTT ping packets.
Code: Select all
async def conn_han(client):
await client.subscribe(AIHE_RELE1_1, 1)
await client.subscribe(AIHE_RELE1_2, 1)
await client.subscribe(AIHE_RELE2_1, 1)
await client.subscribe(AIHE_RELE2_2, 1)
Checking WiFi integrity.
Got reliable connection
Connecting to broker.
Connected to broker.
Task exception wasn't retrieved
future: <Task> coro= <generator object '_handle_msg' at 3ffe95b0>
Traceback (most recent call last):
File "uasyncio/core.py", line 1, in run_until_complete
File "mqtt_as.py", line 546, in _handle_msg
File "mqtt_as.py", line 439, in wait_msg
TypeError: function takes 2 positional arguments but 3 were given
Traceback (most recent call last):
File "main.py", line 167, in <module>
File "uasyncio/core.py", line 1, in run_until_complete
File "uasyncio/core.py", line 1, in run_until_complete
File "uasyncio/core.py", line 1, in wait_io_event
KeyboardInterrupt:
MicroPython v1.13 on 2020-09-02; ESP32 module with ESP32
row 167: loop.run_until_complete(main(client))
Does that mean that I shall build a list for single client.subscribe command, or should I make unique mqtt client object for each topic?
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Using mqtt broker status messages as connection alive checker
I'm not sure what's going wrong here based on these code fragments. I can't see anything obviously wrong, assuming the variables like AIHE_RELE1_1 are strings. What you are doing looks correct but the exception is occurring when the code attempts to call your subscription callback. I suspect your callback doesn't support the required three arguments (topic, msg, retained). See the examples, e.g. range.py.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
-
- Posts: 67
- Joined: Fri Sep 04, 2020 9:27 am
- Location: Hanko, Finland
- Contact:
Re: Using mqtt broker status messages as connection alive checker
You are absolutelly right.pythoncoder wrote: ↑Thu Oct 15, 2020 8:56 amI'm not sure what's going wrong here based on these code fragments. I can't see anything obviously wrong, assuming the variables like AIHE_RELE1_1 are strings. What you are doing looks correct but the exception is occurring when the code attempts to call your subscription callback. I suspect your callback doesn't support the required three arguments (topic, msg, retained). See the examples, e.g. range.py.
I did not have "retained" in my callback, because I ported script from simple mqtt-version and somehow was blind for such parameter.
So, in the callback definition I have now topic, msg, retained and async-version works like a charm! Thank you very much!
Checking WiFi integrity.
Got reliable connection
Connecting to broker.
Connected to broker.
15.10.2020 klo 14:14:15: b'koti/ulko/etela/valaistusautokatos' off
15.10.2020 klo 14:14:15: b'koti/ulko/etela/testi12' off
15.10.2020 klo 14:14:15: b'koti/ulko/etela/testi21' off
15.10.2020 klo 14:14:15: b'koti/ulko/etela/testi22' off
publish 0
publish 1
publish 2
RAM free 88528 alloc 22640
publish 3
publish 4
publish 5