umqtt_simple - bytes index out of range new error

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

umqtt_simple - bytes index out of range new error

Post by devnull » Tue Sep 03, 2019 12:00 pm

Just updated micropython on the SF2 and are now noticing mqtt errors (umqtt_simple line 84) during the connection process.

This tries to connect 3 times and if it fails will goto sleep and try again later.

But I don't recall seeing this "bytes index out of range" error ever before, and I have been using this umqtt_simple module for several years.

Could this be something that has been introduced due to updates in micropython v1.11-218-g3eff81288 or am I barking up the wrong tree and this is just a coincidence ??

Code: Select all

Client: 484a300173f0 mqtt.thingspeak.com 1883
mqtt-connecting...
mqtt-connect error
bytes index out of range
mqtt-connecting...
mqtt-connect error
bytes index out of range
mqtt-connecting...
Thanks

Pete

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by jimmo » Tue Sep 03, 2019 1:02 pm

Are you able to add some debugging to umqtt/simple.py

It seems pretty surprising that -- `premsg = sz` could be failing because i would have to be >= 6, which means that sz was a very large number? But this code looks like it mostly only uses data provided by the user, so hopefuly some print() statements will explain pretty quickly.

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by devnull » Tue Sep 03, 2019 1:13 pm

Hello Jimmo, thanks for helping.

Sorrie, specifying the line number was misleading as I maye have added some comments to the top of the file, this in fact is line 84:

Code: Select all

assert resp[0] == 0x20 and resp[1] == 0x02
Cheers

PeterC

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by devnull » Tue Sep 03, 2019 11:34 pm

When I revert back to an older build, the problem goes away and connection is successful every time with no errors:

Code: Select all

MicroPython v1.9.3-1869-ga9b1d3ca3
Using my latest, I get the bytes index out of range errors, and it very very rarely connects:

Code: Select all

micropython v1.11-218-g3eff81288

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by jimmo » Tue Sep 03, 2019 11:46 pm

Ok interesting! Do you have some simple steps I can use to repro on my SF2 board?

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by devnull » Wed Sep 04, 2019 7:19 am

Hi;

Thanks so much for offering help Jimmo, I have yet to debug myself, this is already mounted to a PCB and boxed.

I need to first do some basic debugging and potentially create some steps to replicate.

Will get beck to you once I have done that.

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by devnull » Wed Sep 04, 2019 10:23 am

OK, here's some debug info that has been added to mqtt_simple:

This is the code from mqtt_simple:

Code: Select all

        self.sock.write(premsg, i + 2)
        self.sock.write(msg)
        print('dbug-msg:',hex(len(msg)), hexlify(msg, ":"))
        self._send_str(self.client_id)
        if self.lw_topic:
            self._send_str(self.lw_topic)
            self._send_str(self.lw_msg)
        if self.user is not None:
            self._send_str(self.user)
            self._send_str(self.pswd)
        resp = self.sock.read(4)
        print('dbug-resp:',resp)
        assert resp[0] == 0x20 and resp[1] == 0x02
        if resp[3] != 0:
            raise MQTTException(resp[3])
        return resp[2] & 1
This is the output using MicroPython v1.11-218-g3eff81288 which fails after 3 attempts;

Code: Select all

484a300173f0 mqtt.thingspeak.com 1883
mqtt-connecting...
dbug-msg: 0x9 b'04:4d:51:54:54:04:02:00:00'
dbug-resp: b''
bytes index out of range
dbug-msg: 0x9 b'04:4d:51:54:54:04:02:00:00'
dbug-resp: b''
bytes index out of range
dbug-msg: 0x9 b'04:4d:51:54:54:04:02:00:00'
dbug-resp: b''
bytes index out of range
And this is using MicroPython 1.9.3-1869-ga9b1d3ca3 which succeeds on first attempt:

Code: Select all

484a300173f0 mqtt.thingspeak.com 1883
mqtt-connecting...
dbug-msg: 0x9 b'04:4d:51:54:54:04:02:00:00'
dbug-resp: b' \x02\x00\x00'

User avatar
devnull
Posts: 473
Joined: Sat Jan 07, 2017 1:52 am
Location: Singapore / Cornwall
Contact:

Re: umqtt_simple - bytes index out of range new error

Post by devnull » Wed Sep 04, 2019 11:40 pm

Can anybody else confirm with a live mqtt setup (simple_mqtt or robust_mqtt) on the pyboard D series that mqtt is working for them ??

My gut feeling is that this is somehow related to the socket, as this code is pretty much only reading and writing to the socket and is not getting a reply.

Post Reply