esp32 OTA can be so aggravating

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
ChrisO
Posts: 48
Joined: Mon Apr 06, 2020 6:16 pm

esp32 OTA can be so aggravating

Post by ChrisO » Fri May 13, 2022 10:29 am

Hi,

I just want to vent (and secretly hope for magical solutions but do not expect any).
Micropython + OTA + ESP32 is just so shit.
When you flash the build results of a project:
parition-table.bin, ota_data_initial.bin, bootload.bin and micropython.bin
have that firmware download the exact same micropython.bin and write it to the other OTA partition, shit just does not work reliably.

sometimes, the devices reboots and everything works fine and dandy... but most of the time I run into shit like:
OSError -17040 and its a huuuuuge pain to learn about the meaning of the error, or how to fix it.

I desperately tried stuff like decreasing micropython heap size, disabling BLE right after OTA and simple doing nothing new.... and sometimes all of these changes seem to help, only to realise a few commits on that those things did nothing...

aaarrghhhghghgh

thanks for listening.
Chris

bulletmark
Posts: 59
Joined: Mon Mar 29, 2021 1:36 am
Location: Brisbane Australia

Re: esp32 OTA can be so aggravating

Post by bulletmark » Fri May 13, 2022 11:23 pm

All I can say is that I added OTA firmware update to my web application on an ESP32 and it works 100% reliably so I guess it is something wrong in your code/approach?

I really just followed the documentation. Do a `partition.writeblocks()` each time you have enough chunks to bundle up a block. Block size is 4096 bytes for me but I HTTP post in lesser chunks, e.g. 2048 (or even 512 when I am running with low memory). Then do a `partition.set_boot()` at the end. I also do a final `partition.ioctl(3, None)` to force sync but not sure if that is required. Then of course a `Partition.mark_app_valid_rollback()` after next boot. I'm just using the standard `partitions-ota.csv`.

ChrisO
Posts: 48
Joined: Mon Apr 06, 2020 6:16 pm

Re: esp32 OTA can be so aggravating

Post by ChrisO » Sat May 14, 2022 6:41 pm

Thanks for your response.
Do you also append 0xFF to your last write in case the firmware is not exactly divisible by 4k?

I've briefly looked at the documentation for that ioctl command and did not quite understand it yet, I'll have another look on Monday and maybe give it a try. Thanks.
Chris

bulletmark
Posts: 59
Joined: Mon Mar 29, 2021 1:36 am
Location: Brisbane Australia

Re: esp32 OTA can be so aggravating

Post by bulletmark » Sat May 14, 2022 10:00 pm

ChrisO wrote:
Sat May 14, 2022 6:41 pm
Do you also append 0xFF to your last write in case the firmware is not exactly divisible by 4k?
Yes, you have to always write the full 4096 bytes [actually BLOCKSIZE = `partition.ioctl(5, None)` so don't hard code it]. I fill append the last block with 0xff to ensure this.

ChrisO
Posts: 48
Joined: Mon Apr 06, 2020 6:16 pm

Re: esp32 OTA can be so aggravating

Post by ChrisO » Tue May 17, 2022 8:28 am

bulletmark wrote:
Sat May 14, 2022 10:00 pm
ChrisO wrote:
Sat May 14, 2022 6:41 pm
Do you also append 0xFF to your last write in case the firmware is not exactly divisible by 4k?
Yes, you have to always write the full 4096 bytes [actually BLOCKSIZE = `partition.ioctl(5, None)` so don't hard code it]. I fill append the last block with 0xff to ensure this.
would you mind sharing the details of your dev environment?
What commit of micropython are you on and what version of esp-idf are you using?
I'm currently using `ci_esp32_setup_helper v4.2.3` to install esp-idf and on `665f0e2a68dfa4944f60e85a00d0543bb3d7fee8` for my mp builds.

regards,
Chris
Chris

bulletmark
Posts: 59
Joined: Mon Mar 29, 2021 1:36 am
Location: Brisbane Australia

Re: esp32 OTA can be so aggravating

Post by bulletmark » Tue May 17, 2022 10:38 am

ChrisO wrote:
Tue May 17, 2022 8:28 am
would you mind sharing the details of your dev environment?
What commit of micropython are you on and what version of esp-idf are you using?
I've used various versions of both Micropython and IDF over the last few months so I don't think it matters. My current build is using Micropython 5e685a9c6faeb764c0b689bc736ed00736bfd6b6 (v1.18-262-g5e685a9c6) and IDF 4.4.

ChrisO
Posts: 48
Joined: Mon Apr 06, 2020 6:16 pm

Re: esp32 OTA can be so aggravating

Post by ChrisO » Thu Jun 09, 2022 12:11 pm

bulletmark wrote:
Tue May 17, 2022 10:38 am
ChrisO wrote:
Tue May 17, 2022 8:28 am
would you mind sharing the details of your dev environment?
What commit of micropython are you on and what version of esp-idf are you using?
I've used various versions of both Micropython and IDF over the last few months so I don't think it matters. My current build is using Micropython 5e685a9c6faeb764c0b689bc736ed00736bfd6b6 (v1.18-262-g5e685a9c6) and IDF 4.4.
Hi Bulletmark,

Another follow-up question.
Did you have to change any MP code or config when switching between IDF versions?

I now have a codebase + micropython setup which works with ESP-IDF v4.2.3, when I install v4.4.1 and rebuild (mpy-cross and my custom board) everything compiles properly, but during runtime I see a difference.

With v.4.2.3 `sock = ssl.wrap_socket(sock)` works and with v4.4.1 I get `OSError -0x7080`
I've double checked to confirm that the build includes ssl support

Code: Select all

// from micropython/ports/esp32/mpconfigport.h
#define MICROPY_PY_USSL                     (1)
#define MICROPY_SSL_MBEDTLS                 (1)
Chris

ChrisO
Posts: 48
Joined: Mon Apr 06, 2020 6:16 pm

Re: esp32 OTA can be so aggravating

Post by ChrisO » Thu Jun 09, 2022 12:34 pm

nvm, I shot myself in the foot, big time, by making micropython a submodule of my project.
Turns out I butchered merging code and was missing stuff all kinds of mbed_tls related C code.

Thanks for all your time and info.
Chris

Post Reply