Missing something with mqtt simple/robust?

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Missing something with mqtt simple/robust?

Post by Jibun no kage » Tue Aug 16, 2022 7:28 pm

Been playing with native MP variants of MQTT simple and robust, and after (whatever I set the keep alive value to) I see both wait_msg and check_msg drop an OSError -1 when debug is enabled. If was my understanding that robust, check_msg would not let the broker drop connection? Did I misunderstand this? So in my main lookup, I added a reconnect call and a re-subscribe call but now I get OSError 12? Seems odd.

User avatar
karfas
Posts: 193
Joined: Sat Jan 16, 2021 12:53 pm
Location: Vienna, Austria

Re: Missing something with mqtt simple/robust?

Post by karfas » Tue Aug 16, 2022 11:20 pm

Jibun no kage wrote:
Tue Aug 16, 2022 7:28 pm
If was my understanding that robust, check_msg would not let the broker drop connection?
The broker will disconnect if the client does not respond, either on TCP or (depending upon QOS) on MQTT level. Nothing on the client side can change this behaviour.
I assume you saturate the 8266 TCP stack with messages and do no proper cleanup during the reconnect - oserror 12 means out of memory.
What are your requirements in messages/sec ?
Maybe you need to rethink your hardware and software choices.
A few hours of debugging might save you from minutes of reading the documentation! :D
My repositories: https://github.com/karfas

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: Missing something with mqtt simple/robust?

Post by Jibun no kage » Wed Aug 17, 2022 12:35 am

Actually it was two different things I discovered, the broker was dropping the connection, looking at the robust code, there is a ping method, but apparently it is not being used, or correctly used internally. Once I added the ping method to a timer that was less than the keep alive, the disconnects stopped. I am using a clean session, so I know that the client must do a re-connect and a re-subscribe, again. However, the robust implementation does not address this directly looking at the code. So when the documentation states that the robust module handles this without qualification, I don't see that actually in the current code. A reconnect with a clean session is different than a reconnect with a non-clean session.

The mqtt_as documentation actually has a reference to the issue that the robust module fails to maintain the active session with the broker, if I recall right from memory. This is not persistence issue on the broker side, that is not relevant, because broker expects the client to do all the work under a clean session mode reconnect scenario.

As for error 12, I have not seen that again as yet, as long as the connection says live, even when I flood the MQTT client on purpose, I do get a stack/recursive error, which seems misleading at first glance but I would have to look at the C code to see how that specific error results. Since I am not using asyncio, I may be crashing the stack frame limit, but on a single core/thread running code that is stable at a slower rate, that seems a bit odd. Something is stepping on something. Even when it does report a recursive error, wrapping the code in a try/catch, lets the main loop continue, again as long as the connection stays up, no reset occurs. I guess I was expecting a (true) stack frame exhaustion error to crash the micro-controller and force a reset.

I am using a Pico, initially but once I get a custom firmware image that is specific to my use case will test an ESP with the same basic code. The good news is, as simple (no pun intended) as the native robust module is, it works pretty well, not withstanding the above discovery. For more demanding, message throughput, mqtt_as based on asyncio would be likely needed, but since I don't need high volume message handling, at most 1 message per second, looks like robust module is likely sufficient for my use case.

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: Missing something with mqtt simple/robust?

Post by Jibun no kage » Wed Aug 17, 2022 12:41 am

@karfas, Oh, your signature line... "A few hours of debugging might save you from minutes of reading the documentation!" Every time I see that, It cracks me up. I remember a professor of mine back in my university days that said something comparable... "The more time you spend reading the documentation, the less total time you waste debugging."

"Große Köpfe denken ähnlich?" My forum name maybe Japanese, but I spent a lot of time with the grandparents, most summers, in Munich. "Viel spaß"

Jibun no kage
Posts: 144
Joined: Mon Jul 25, 2022 9:45 pm

Re: Missing something with mqtt simple/robust?

Post by Jibun no kage » Wed Aug 17, 2022 2:14 am

FYI... Just did a quick test on a ESP module, my MP scripts are transportable, I took the time to abstract the bit of code that is different between ESP MP and Pico MP.

Test on ESP module, not only did the code work, but the message rate is a bit better on the ESP than the PicoW. The flooding of the device is a bit harder to do on the ESP module versus the Pico. If memory serves, the Pico is 2MB flash and the ESP I am using right now is 4MB flash. No stack crash errors on the ESP module as yet, and I have about 1/3 more messages hitting the ESP module than the PicoW I tested on this morning. Tomorrow I should be able to test the same code on a ESP01 (1M flash) with a custom firmware image to maximize resource availability. If it does as well as the others, I am in good shape.

Is there a way to change the default stack frame size? That would be interesting test, maybe to figure out how the Pico works with a slight bigger stack frame size.

Post Reply