A "resilient" asynchronous MQTT client

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
User avatar
pythoncoder
Posts: 3651
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: A "resilient" asynchronous MQTT client

Post by pythoncoder » Mon Dec 04, 2017 6:56 am

The performance of MQTT is quite hard to quantify, but fast it is not. One factor is the location of the broker. If it is on the internet then you can easily have delays of many seconds in receiving a response. And that is without outages...

If the broker is on the LAN there are still a considerable number of uncertainties, especially if you use qos==1. WiFi does not have perfect connectivity. Under conditions of RF interference or a weak signal my resilient driver will wait as long as necessary to ensure that the qos==1 contract is honoured. If you use the official driver under these conditions the contract will not be honoured. Unless the driver has recently been fixed, it can actually hang waiting for an acknowledge which is never received.

If performance is key then you could use the official driver with qos==0 and accept the risk of message loss. However you would need to consider various aspects of the MQTT protocol. For example if your client can spend long periods without publishing but needs to respond to subscriptions, then you need to ensure that the broker keeps the link open by sending ping packets. My driver aims to handle these details.

My personal view is that MQTT (especially running over WiFi) is really only suitable for applications where a response time >= 1s is acceptable. MQTT was developed by IBM for remote data acquisition over unreliable links. It was never intended as a realtime command and control protocol.
Peter Hinch

cefn
Posts: 227
Joined: Tue Aug 09, 2016 10:58 am

Re: A "resilient" asynchronous MQTT client

Post by cefn » Sat Dec 16, 2017 9:41 am

Thanks for sharing your thoughts, Peter. For fear of dragging this thread off topic, I've responded in a new thread
"MQTT for real-time applications - discussion of overheads"
...at... viewtopic.php?f=15&t=4175

cefn
Posts: 227
Joined: Tue Aug 09, 2016 10:58 am

Re: A "resilient" asynchronous MQTT client

Post by cefn » Sat Dec 16, 2017 6:51 pm

Had a temporary issue where the uptime of my mqtt_as client was very low, (kept losing connection to broker every 3 minutes). However, I then added code to make sure it was publishing as well as subscribing. Then it ran for many thousand seconds without losing connection. I don't know for sure these things are connected, but I don't recall changing anything else.

I am now wondering about the parameters _DEFAULT_MS and _SOCKET_POLL_DELAY in https://github.com/peterhinch/micropyth ... mqtt_as.py which sets a number of milliseconds to sleep
and indirectly the delays in 'SynCom'. The socket poll is inline in the _as_read and _as_write calls although I haven't worked out quite what role SynCom has, yet.

I reduced these delays and it seemed to make client transactions more responsive. Wasn't easy to check this interactively, as it's encoded as a const() value so I had to rebuild the module. Previously it seemed (perception) to take more than a second for every segment to arrive in my application, which is only a total of 48 bytes (3bytes for 16 segments). Now it seems much closer to the realtime data.

Currently I set the values to...

Code: Select all

_DEFAULT_MS = const(1)
_SOCKET_POLL_DELAY = const(0) 
Is there a cost to reducing these delays which I haven't anticipated? I figure these polls are interleaved with other things, so if there are other jobs to do, the repeated polls of sockets will be delayed anyway. Does this sound right? Can I legitimately run an application with these delay values?

User avatar
pythoncoder
Posts: 3651
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: A "resilient" asynchronous MQTT client

Post by pythoncoder » Sun Dec 17, 2017 7:53 am

I take it you're running mqtt_as.py. This doesn't use SynCom - the comment about it was for my own reference. The SynCom class provides a means of communication between the ESP8266 and a Pyboard without using the UART. It's bitbanged and asynchronous so to run at a reasonable speed it needs other coroutines to be well-behaved and not hog the CPU.

That is the issue behind the two constants

Code: Select all

_DEFAULT_MS = const(20)
_SOCKET_POLL_DELAY = const(5)  # 100ms added greatly to publish latency
They determine how frequently various tasks are scheduled and so how well mqtt_as.py cooperates with time-critical user tasks. In a typical web app with no task hoping for millisecond level scheduling these can be reduced and it should keep working with values of zero (uasyncio uses round-robin scheduling if you issue sleep_ms(0)).

As for subscribe-only applications mqtt_as.py aims to keep the connection to the broker open - see the _keep_alive task. If it hasn't had a communication from the broker in the time self._ping_interval it issues an MQTT ping packet. The ping interval is set to 1/4 of the keepalive period but you can reduce this with a config option. The idea here is that a subscribe only application will ping the broker four times per keepalive period which should keep the link open. However the operative word is should because if the connection is sufficiently bad these pings may not be received by the broker and it will time out. This is, of course, correct behaviour on the part of the broker which will then issue any last will message.
Peter Hinch

RobinMosedale
Posts: 11
Joined: Fri Jul 26, 2019 9:40 pm

Re: A "resilient" asynchronous MQTT client

Post by RobinMosedale » Fri Jul 26, 2019 9:55 pm

This query has been removed, and a new thread created:-
viewtopic.php?f=18&t=6738
Last edited by RobinMosedale on Sun Jul 28, 2019 3:45 pm, edited 1 time in total.

kevinkk525
Posts: 502
Joined: Sat Feb 03, 2018 7:02 pm

Re: A "resilient" asynchronous MQTT client

Post by kevinkk525 » Sat Jul 27, 2019 6:36 am

It's not a problem with the library, it's either how you use it or you running into microcontroller limitations.

I'm using that library since a long time now and have no problems.
However you need to be aware of the limitations.
In the range example you posted on raspberrypiforum there is a coroutine created every time a new message arrives, which publishes a mqtt message, waits one second and then toggles the led again. Now if you get too many messages within a short time, you will get a queue overflow. The uasyncio typically only runs with 16 slots in the queue. That example is not built for this type of test.
Could that be the reason you experience these overflows?

Also qos 1 works perfectly fine, I use it all the time (ok here I have to admit that Peter's implementation has a problem if multiple coroutines publish with qos 1 at the same time, I solved that in my fork, but it doesn't lead to queue overflows).

And before making bad comments about other people's work be sure to understand the library you use, the environment you're operating in and the example code you use. This library has been tested extensively and all limitations of both microcontroller and the library itself are documented in the github repository.
But we are always happy to help and get you up and running. Feel free to post any issues you encounter and we'll tell you why that is and how to solve it. Using microcontrollers can be tricky sometimes due to the constraint resources available.
Kevin Köck

RobinMosedale
Posts: 11
Joined: Fri Jul 26, 2019 9:40 pm

Re: A "resilient" asynchronous MQTT client

Post by RobinMosedale » Sat Jul 27, 2019 2:46 pm

This query has been removed, and a new thread created:-
viewtopic.php?f=18&t=6738
Last edited by RobinMosedale on Sun Jul 28, 2019 3:45 pm, edited 1 time in total.

kevinkk525
Posts: 502
Joined: Sat Feb 03, 2018 7:02 pm

Re: A "resilient" asynchronous MQTT client

Post by kevinkk525 » Sat Jul 27, 2019 5:34 pm

Could you please create a new thread with your problem? Makes it easier.

Commenting out the publish routine doesn't really solve the problem of queue overflow as that should only take 50ms tops. You have a wait 1 second command in there, so if you get many messages, that will be the bottleneck.
How are you sending messages to the device and how many per second?

You could add a lock so that a new pulse() coroutine is only started after the last one has ended.
But that is just a simple example. Generally the rule is indeed to make the callback for new messages as short as possible (but also not spam new coroutines if you can't ensure that the device won't get too many messages) and implement your own way of dealing with too many messages (either by having another queue, a lock or a limit).
Kevin Köck

RobinMosedale
Posts: 11
Joined: Fri Jul 26, 2019 9:40 pm

Re: A "resilient" asynchronous MQTT client

Post by RobinMosedale » Sat Jul 27, 2019 7:06 pm

I will indeed Kevin, and then perhaps remove this from an otherwise positive thread.

I am more than grateful for you help, Kevin, please take my at first sight sharp comments as from huge frustration. If you've seen from posts elsewhere, this has been a hugely frustrating element, a small tiny little corner of a very large application with a lot of personal investment and the effort and confusion has been inordinate. MQTT was a final resort and I'm afraid I vented my spleen.

Probably be 24 hours, and thanks again.
I was aware that the positive feedback from the publishing co routine had probably resulted in code resonance, just on the limit.
Soak test still running.

Post Reply