MicroPython Forum (Archive)

I'd like to evaluate how much interest there is for having a MQTT v5.0 client for micropython.

I looked at the commit in paho.mqtt for CPython that added the support for 5.0 and it should be fairly possible to implement at least a subset of the features in micropython without making the library a lot bigger. (I'm talking about the async implementation mqtt_as by Peter Hinch as a basis, not umqtt.simple)

Features of MQTT 5.0 that might be interesting:
- Session Expiry interval (receive messages missed during a short reconnect but not hours old messages, replacement for clean_session)
- Message expiry: Allow an expiry interval to be set when a message is published.
- Shared Subscriptions: Add shared subscription support allowing for load balanced consumers of a subscription
- Topic Alias: Decrease the size of the MQTT packet overhead by allowing the topic name to be abbreviated to a small integer.
- Flow control: Allow the Client and Server to independently specify the number of outstanding reliable messages (QoS>0) they allow.
- User properties: Add User Properties to most packets. User properties on PUBLISH are included with the message and are defined by the Client applications. The user properties on PUBLISH and Will Properties are forwarded by the Server to the receiver of the message. (that would however increase code size and RAM demands significantly but can probably be implemented as an addon)
- Maximum Packet Size: Allow the Client and Server to independently specify the maximum packet size they support.
- Will delay: Add the ability to specify a delay between the end of the connection and sending the will message. This is designed so that if a connection to the session is re-established then the will message is not sent. This allows for brief interruptions of the connection without notification to others.

Features copied from https://github.com/mqtt/mqtt.github.io/ ... .1-and-5.0

I see mosquitto already supports v5.0.

Some of those features seem over complex for typical MicroPython setups. Owing to RAM limitations I favour application level solutions over increasing the size of the library. Thus the overhead is experienced only by users needing the functionality. First my demolition job on features we might be able to do without, then the ones I like:

Topic alias? Just use short topic names I think this is aimed at ultra low bandwidth links, not WiFi.

Expiry options could be handled at application level by sending a timestamp as part of the message. Again, we're short of RAM not bandwidth.

Maximum packet size isn't really relevant in small systems designed as a whole. The feature strikes me as being for systems having components whose behaviour is not fully known.

User properties sound like bloat. Shared subscriptions aren't clearly explained. If you think these features are worth implementing perhaps you could make a case for them?

Now for
Flow control is well worth implementing as it could avoid concurrency problems on RAM constrained targets. Will delay sounds useful and is probably small in implementation; what's more the feature is more like a bugfix. Current broker behaviour is opaque.

Topic alias? Just use short topic names I think this is aimed at ultra low bandwidth links, not WiFi.

Yeah probably. A topic like "home/34jlsf930/device/set" might seem long but neither RAM demands nor the bandwith requrements are a real problem for micropython devices.

Expiry options could be handled at application level by sending a timestamp as part of the message. Again, we're short of RAM not bandwidth.

Question is wether doing everything on application level instead of library level is really more RAM efficient.

User properties sound like bloat. Shared subscriptions aren't clearly explained. If you think these features are worth implementing perhaps you could make a case for them?

The implementation of user properties are actually not big and would replace sending a json dict as payload.
The implementation of other properties would be very heavy but could be optional.

Flow control is well worth implementing as it could avoid concurrency problems on RAM constrained targets. Will delay sounds useful and is probably small in implementation; what's more the feature is more like a bugfix. Current broker behaviour is opaque.

These sounded most useful to me from all changes.

Additionally to Session Expiry interval (receive messages missed during a short reconnect but not hours old messages, replacement for clean_session)

kevinkk525 wrote: ↑

Sat Oct 26, 2019 10:27 am

...
Question is whether doing everything on application level instead of library level is really more RAM efficient...

My point is that if you include something in the library, then RAM is used whether or not the application requires the feature. Doing something at application level may (sometimes) be more costly, but the cost is zero if your application doesn't actually do it

Clearly it's a matter of degree. If a useful feature can be done in one line of code in the library but would require 20 in the application, then the answer is evident. If it required 20 in either case, then I'd leave it to the app. I'm wary of feature bloat in a library which tries to be reasonably "micro".

Session expiry interval is perhaps a case in point. Easily done at application level with a timestamp, but let's see what's involved in implementing it.

I completely understand your point and the esp8266 might not be a target for major new features anyway but the esp32 or the pyboard D are powerful enough and have enough RAM to even support the full list of MQTT 5.0 features. If those are really needed is a different matter..