OTA Firmware update with pure uPy script possible?

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
jedie
Posts: 247
Joined: Fri Jan 29, 2016 12:32 pm
Contact:

OTA Firmware update with pure uPy script possible?

Post by jedie » Fri Dec 27, 2019 2:45 pm

On ESP8266 and ESP32 we have the exp functions esp.flash_erase(sector_no) and esp.flash_write(byte_offset, bytes): http://docs.micropython.org/en/latest/l ... flash_read

At least reading access to the firmware section is possible, as my experiments here have shown: viewtopic.php?f=16&t=7467

Is write access also possible? If so, then it should be possible to overwrite the firmware in pure python code, shouldn't it?

How do I get the exact position of the firmware?

jedie
Posts: 247
Joined: Fri Jan 29, 2016 12:32 pm
Contact:

Re: OTA Firmware update with pure uPy script possible?

Post by jedie » Fri Dec 27, 2019 2:55 pm

It must be possible! I can change the firmware via esp.flash_write() e.g.:

Code: Select all

import esp

SECTOR_SIZE = const(4096)
BUFFER = memoryview(bytearray(SECTOR_SIZE))

if __name__ == '__main__':
    sector_no = 151  # sector with: 'MicroPython v1.12 on 2019-12-20;...'
    byte_offset = sector_no * SECTOR_SIZE  # byte offset: 618496 0x97000
    print('sector no: %i -> byte offset: %i 0x%x' % (sector_no, byte_offset, byte_offset))

    # check fw with not changed content:
    # size: 619812
    # md5: 439de24004683e779dfff08f96df900d
    print(esp.check_fw())  # will return True

    # read the 4KB sector:

    esp.flash_read(byte_offset, BUFFER)
    # print(bytearray(BUFFER))
    print(bytearray(BUFFER[0x38a:0x3aa]))  # bytearray(b'MicroPython v1.12 on 2019-12-20;')

    # Change the content:
    BUFFER[0x38a:0x3aa] = b'MICROPYTHON v9.99 on 2200-12-24;'
    esp.flash_erase(151)
    esp.flash_write(byte_offset, BUFFER)

    # Check the content:
    esp.flash_read(byte_offset, BUFFER)
    print(bytearray(BUFFER[0x38a:0x3aa]))

    # check fw with changed content:
    # size: 619812
    # md5: 8696b82c53ea091bd2d6cd9a24796f3e
    print(esp.check_fw())  # False

    # Change back to origin:

    BUFFER[0x38a:0x3aa] = b'MicroPython v1.12 on 2019-12-20;'
    esp.flash_erase(151)
    esp.flash_write(byte_offset, BUFFER)

    # Check flash:
    esp.flash_read(byte_offset, BUFFER)
    print(bytearray(BUFFER[0x38a:0x3aa]))

    # check fw with not changed content:
    # size: 619812
    # md5: 439de24004683e779dfff08f96df900d
    print(esp.check_fw())

Important is to change the complete sector and do esp.flash_erase() before flash.

jedie
Posts: 247
Joined: Fri Jan 29, 2016 12:32 pm
Contact:

Re: OTA Firmware update with pure uPy script possible?

Post by jedie » Fri Dec 27, 2019 5:26 pm

jedie wrote:
Fri Dec 27, 2019 2:45 pm
How do I get the exact position of the firmware?
It is: 0x40200000 can be found in: https://github.com/micropython/micropyt ... 6/modesp.c and other places: https://github.com/micropython/micropyt ... 0x40200000

e.g.:

Code: Select all

>>> print(repr(uctypes.bytearray_at(FIRMWARE_START, 16)))
bytearray(b'\xe9\x03\x03 \xa0\x04\x10@\x00\x00\x10@py\x00\x00')
and the esp8266-20191220-v1.12.bin file starts with the same values.

jedie
Posts: 247
Joined: Fri Jan 29, 2016 12:32 pm
Contact:

Re: OTA Firmware update with pure uPy script possible?

Post by jedie » Mon Dec 30, 2019 5:12 pm

This can't work: Because the running micropython interpreter must erase/write this own code in 4K blocks. So the runnig interpreter will crash, isn't it?

jedie
Posts: 247
Joined: Fri Jan 29, 2016 12:32 pm
Contact:

Re: OTA Firmware update with pure uPy script possible?

Post by jedie » Tue Dec 31, 2019 11:34 am

Whatever comes to mind: You probably can't overwrite the currently running python interpreter in flash.

But it should be possible to overwrite all other parts... So if yaota is used: It should be possible to update the yaota firmware part... and what is much more interesting: It must be possible to overwrite the flash part with the frozen modules, isn't it?

However, some conditions would have to be met for this to happen:

The frozen modules must not overlap with the interpreter. But maybe this is the case with 4KB block size?

One must know the exact position.

You only need to be able to compile the part of the frozen modules. But what does it look like exactly?

User avatar
tve
Posts: 202
Joined: Wed Jan 01, 2020 10:12 pm
Location: Santa Barbara, CA
Contact:

Re: OTA Firmware update with pure uPy script possible?

Post by tve » Sat Jan 04, 2020 12:39 am

I don't see much of an issue in doing OTA in mostly python. There is support in ESP-IDF for the esp32 (I can't comment on the esp8266 as I have not kept track of developments): https://docs.espressif.com/projects/esp ... m/ota.html
The arduino (esp32) Updater class is a little easier to use wrapper on top of that that may be useful, of course that's C++, I would probably insert that into the port almost as-is, but you could also use it as inspiration and rewrite it in python: https://github.com/espressif/arduino-es ... Update/src
I like asyncio, so I would use that to implement an update protocol. I recently implemented a simple one using MQTTS and HTTP in C++: https://github.com/tve/esp32-secure-bas ... rc/ota.cpp, basically it sends the esp32 a URL for the download and an MD5 checksum.
Of course many variations on the theme are possible... You could also do it all in micropython as long as there's a function to read/write flash, but to be honest, I don't think it's a good idea not to use the ESP-IDF primitives.


User avatar
tiny
Posts: 6
Joined: Tue Dec 31, 2019 1:26 am

Re: OTA Firmware update with pure uPy script possible?

Post by tiny » Sun Jan 05, 2020 3:57 am

Im dreaming way above my skill level but - what about making a little bios/hypervisor kind of setup. A smaller partition whose sole job it is to boot the partition you plan to write and rewrite with OTA. This little section would also store your reflashing script. Is that feasible?


jomas
Posts: 49
Joined: Mon Dec 25, 2017 1:48 pm
Location: Netherlands

Re: OTA Firmware update with pure uPy script possible?

Post by jomas » Sun Jan 05, 2020 1:12 pm

jedie wrote:
Sat Jan 04, 2020 7:14 am
Think that is a ESP32 only feature...

Arduino has OTA updates on ESP8266: https://arduino-esp8266.readthedocs.io/ ... eadme.html

But I don't know if that will replace the complete firmware or only the user code?
The ESP8266 has FOTA (Firmware Over The Air) build in since day one.
Its API is just 5 system calls.
When I was programming in C, I used it to flash my esp. It is so easy and very fast.
I don't understand why it was never implemented in MicroPython and people try to reinvent the wheel.
(Same for smartconfig and espnow).

Post Reply