Last will on umqtt.simple

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
guyd
Posts: 81
Joined: Fri Jul 20, 2018 6:08 am

Re: Last will on umqtt.simple

Post by guyd » Thu Sep 13, 2018 5:56 pm

another output:
keepalive=60
watch the repeatition of disconnects ( exactly 90 sec apart, resulting a last_will msg accordingly:

Code: Select all

Connected to MQTT server
[2018-09-13 20:47:15.03] Fail Status #1: Wifi is: True
Connected to MQTT server
disconnect #1:  duration: 0.7
[2018-09-13 20:48:45.03] Fail Status #1: Wifi is: True
Connected to MQTT server
disconnect #2:  duration: 0.8
[2018-09-13 20:50:15.03] Fail Status #1: Wifi is: True
Connected to MQTT server
disconnect #3:  duration: 0.8
[2018-09-13 20:51:45.03] Fail Status #1: Wifi is: True
Connected to MQTT server
disconnect #4:  duration: 0.7
[2018-09-13 20:53:15.03] Fail Status #1: Wifi is: True
Connected to MQTT server
disconnect #5:  duration: 0.8
[2018-09-13 20:54:45.03] Fail Status #1: Wifi is: True
Connected to MQTT server
disconnect #6:  duration: 0.8

guyd
Posts: 81
Joined: Fri Jul 20, 2018 6:08 am

Re: Last will on umqtt.simple

Post by guyd » Thu Sep 13, 2018 6:49 pm

and keepalive=15 gave same results,, 15 seconds apart

SpotlightKid
Posts: 377
Joined: Wed Apr 08, 2015 5:19 am

Re: Last will on umqtt.simple

Post by SpotlightKid » Thu Sep 13, 2018 7:15 pm

Without knowing your full code, posting your output here is no use to us at all. You should instead tell us, what you are doing to diagnose the problem. Then we can maybe help you with that.

Concerning the behaviour of umqtt.simple on an ESP8266 in the case of power-loss, I can confirm that the mosquitto server does not publish the last will until it detects that the connection to the client is gone. This may happen to be only very much later, when the client reconnects. If the client closes the socket (with client.sock.close()), but without doing client.disconnect() first, mosquitto will detect that the connection is closed immediately and publish the last will. The solution to this is, as @pythoncoder pointed out, to set a keep-alive time when connecting and then using client.ping() regularly with the keep-alive time.

Here's a simple script to test this with:

Code: Select all

import time
from umqtt.simple import MQTTClient

MQTT_CLIENT = 'upy'
MQTT_SERVER = '192.168.XXX.XXX'
MQTT_ROOT = '/'
MQTT_STATUS_TOPIC = '/status'


def sub_cb(topic, msg):
    print((topic, msg))


def main():
    client = MQTTClient(client_id=MQTT_CLIENT, server=MQTT_SERVER, keepalive=10)
    client.set_last_will(MQTT_STATUS_TOPIC, 'disconnected', retain=True)
    client.set_callback(sub_cb)
    client.connect()
    client.subscribe(MQTT_ROOT + '#')
    client.publish(MQTT_STATUS_TOPIC, 'connected', retain=True)

    try:
        while True:
            while client.check_msg():
                pass
            time.sleep(1.0)
            client.ping()
    except KeyboardInterrupt:
        client.sock.close()
Last edited by SpotlightKid on Thu Sep 13, 2018 8:03 pm, edited 1 time in total.

guyd
Posts: 81
Joined: Fri Jul 20, 2018 6:08 am

Re: Last will on umqtt.simple

Post by guyd » Thu Sep 13, 2018 8:00 pm

thank you for your kind help,
here 's the relevant portion of the code. In a word: if loop fail 3 conseq times - it goes for "emergency state" for few minutes, without seeking broker, to enable user operate manual switch without disturbance.

Code: Select all

def startMQTTclient(self):
        self.mqtt_client = MQTTClient(self.mqtt_client_id, self.server, self.qos, user=self.user,
                                      password=self.password, keepalive=0) -----------> tried some values from 0 to 60
        self.mqtt_client.set_callback(self.on_message)
        self.mqtt_client.set_last_will(topic=self.topic2, msg="last_will", retain=False)

        try:
            self.mqtt_client.connect()
            for topic in self.topic1:
                self.mqtt_client.subscribe(topic)
            print("Connected to MQTT server")
            return 1
        except OSError:
            self.notify_error("Error connecting MQTT broker")
            return 0

Code: Select all

def mqtt_wait_loop(self):
        fails_counter = 0
        off_timer = 0
        tot_disconnections = 0
        self.last_buttons_state = [self.but_up_state(), self.but_down_state()]  -------- > verify physical switchs' state

        while True:
            # detect button press
            self.check_switch_change()     --------> switch's state 
            self.clock_update()                 -------> happens every 12 hrs

            try:
                # verify existance of MQTT server
                self.mqtt_client.check_msg()
                
                # this part happens after reconnection 
                if off_timer != 0:
                    tot_disconnections += 1
                    print("disconnect #%d:  duration: %.1f" % (tot_disconnections, (utime.ticks_ms() - off_timer)/1000))
                    off_timer = 0

            except OSError:
                fails_counter += 1  -------> after 3 fails it gets to emergency state
                if fails_counter == 1:
                    off_timer = utime.ticks_ms()

                self.notify_error("Fail Status #%d: Wifi is: %s [sec]" % (fails_counter, self.is_connected()))

                # Check Wifi connectivity
                if self.is_connected() == 0:
                    self.notify_error("Try reconnect wifi #%d" % fails_counter)
                else:
                    # Try reconnect MQTT client
                    if fails_counter <= self.num_of_fails:
                        if self.startMQTTclient() == 0:
                            utime.sleep(2)
                        else:
                            fails_counter = 0
                    # after  - only button switch without MQTT for some time
                    else:

                        #     # Emeregncy mode- stop looking for MQTT for some time, and comply only to physical switches
                        self.notify_error(
                            "fail reaching MQTT server- %d times, entering emergency mode for %d minutes" % (
                                self.num_of_fails, self.minutes_in_emergency_mode))

                        start_timeout = utime.ticks_ms()
                        time_in_loop = 0
                        while time_in_loop < self.minutes_in_emergency_mode:
                            print("in emergency")
                            self.check_switch_change()
                            time_in_loop = (utime.ticks_ms()-start_timeout)/1000/60
                            utime.sleep(self.t_SW)

                        fails_counter = 0

            utime.sleep(self.t_SW)

        # Try to disconnect

        # self.mqtt_client.disconnect()
        # except OSError:
        #     self.notify_error("fail to disconnect broker")


SpotlightKid
Posts: 377
Joined: Wed Apr 08, 2015 5:19 am

Re: Last will on umqtt.simple

Post by SpotlightKid » Thu Sep 13, 2018 8:11 pm

So this is a totally different problem now than what you started the thread with? I.e. your client keeps losing the connection?

Please open a new topic for that. and before you do, try to come up with the smallest possible amount of complete code that demonstrates the problem. Nobody wants to debug your code for you.

guyd
Posts: 81
Joined: Fri Jul 20, 2018 6:08 am

Re: Last will on umqtt.simple

Post by guyd » Thu Sep 13, 2018 8:29 pm

SpotlightKid wrote:
Thu Sep 13, 2018 8:11 pm
So this is a totally different problem now than what you started the thread with? I.e. your client keeps losing the connection?

Please open a new topic for that. and before you do, try to come up with the smallest possible amount of complete code that demonstrates the problem. Nobody wants to debug your code for you.
No- this is not the problem.
it happens ONLY when Im trying to use LAST WILL, and as you can see in posted results- it hangs up connection exactly 1.5Xtime of keepalive. The last will is causing losing connection.

Guy

SpotlightKid
Posts: 377
Joined: Wed Apr 08, 2015 5:19 am

Re: Last will on umqtt.simple

Post by SpotlightKid » Thu Sep 13, 2018 8:50 pm

Yes, because if you set keepalive, you have to send a PING packet (or any other packet) to the server regularly, otherwise it will disconnect the client if it hasn't seen any message form the client for keepalive time*1.5. I suggest that you read the MQTT spec thoroughly.

http://docs.oasis-open.org/mqtt/mqtt/v3 ... .1-os.html

Or just change your code to send the PING packets in time as we suggested already.

guyd
Posts: 81
Joined: Fri Jul 20, 2018 6:08 am

Re: Last will on umqtt.simple

Post by guyd » Fri Sep 14, 2018 4:37 am

now I see:

Code: Select all

    try:
        while True:
            while client.check_msg():
                pass
            time.sleep(1.0)
            client.ping()           -------------------> Now I see that this is A CRUCIAL feature I missed
    except KeyboardInterrupt:
        client.sock.close()
[/quote]

User avatar
pythoncoder
Posts: 3064
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Last will on umqtt.simple

Post by pythoncoder » Fri Sep 14, 2018 6:52 am

As I said before
the device needs to keep accessing the broker to prevent the last will from being sent.
The official version is very basic, requiring you to manage such details of the protocol in code. It has other issues limiting its ability to maintain a link over long periods. I found it would fail after an hour or two even with the AP in the next room on a wireless network which, in normal use at that range, is rock-solid. Which is why I wrote the resilient version.

The asynchronous resilient version handles details like pinging the broker for you, and improves the ability to withstand outages and disruptions of the WiFi link, and also to work near the limit of WiFi range. Using it does require familiarity with asynchronous programming and uasyncio.
Peter Hinch

Post Reply