Use micropython to do dead simple "over-the-air" code updates

[HermannSW]

There is a standard method to do what you want on MicroPython: Enable WebREPL and you can do OTA updates.
Basically WebREPL is the server side of what you look for.

From laptop to MicroPython

Using webrepl_cli.py

With webrepl_cli.py https://github.com/micropython/webrepl you can copy files to and from MicroPython, similar to "scp".
After you copied a module over to MicroPython, you can use dhyland's "reload(mod)" to reload the copied over module and make the changes effective without MicroPython reload:
viewtopic.php?f=2&t=413&start=30#p7609

Using webrepl_client.py

You can use WebREPL shell https://github.com/Hermann-SW/webrepl#webrepl-shell which is OTA shell in case of wireless connection. You can bring in arbitrary new code or renew code, see the last example in the README.md section I pointed to. Instead of "eval()" as you want to do, you will enter paste mode (CTRL-E), send whatever code you want (I used "cat sc.py" in the example) and make it active with CTRL-D.


From MicroPython to MicroPython

Both methods described do not work from MicroPython to MicroPython.

Yesterday I learned how to use websocket client from danni:
https://twitter.com/HermannSW/status/10 ... 2988877824

It is so simple to execute WebREPL shell commands, although the current logic is for normal mode only. For paste mode you would just need to wait for "=== " prompt instead of ">>> " after each command (in "wait_for_prompt()").

I import drop.py onto ESP01s MicroPython 192.168.178.121, do CTRL-D and "import drop".
That module connects to WebREPL 192.168.178.122:8266 and remotely executes the commands on 2nd ESP01s OTA.
The endless loop is for testing so that I can see toggle of 2nd module builtin LED every second.
I did let it run for >30 minutes just to verify that the method is reliable.

Code: Select all

$ cat drop.py 
import uwebsockets.client
import os
import time

websocket = uwebsockets.client.connect('ws://192.168.178.122:8266/')

def wait_for_prompt():
    resp = ""
    while not(">>> " in resp):
        resp = websocket.recv()
        print("< {}".format(resp))

def do(cmd):
    websocket.send(cmd+"\r\n")
    wait_for_prompt()

resp = websocket.recv()
assert resp == "Password: ", resp

do("abcd")

do("import payload")

while True:
    do("payload.flip()")
    time.sleep(1)
$ 

P.S:
For installation of uwebsockets
https://github.com/danni/uwebsockets

follow the instructions. You can use "ampy" as described there, or "webrepl_cli.py" as described above to copy the files. I did create the directory on MicroPython with "uos.mkdir('uwebsockets')" before copying. For websocket client only two files "uwebsockets/uwebsockets/*.py" need to be copied into "uwebsockets" directory.

As mentioned at end of danni's README.md "logging" module is required.
I did copy this python script to top level directory on MicroPython, as well as my drop.py:
https://raw.githubusercontent.com/micro ... logging.py


P.P.S:
ESP01s modules (cheap and lightweight, 1.5g only) have wireless for free, and 1MB flash allows for full MicroPython to be installed.


P.P.P.S:
Here is the remote payload.py module for completeness:

Code: Select all

$ cat payload.py 
import machine

pwm0 =  machine.PWM(machine.Pin(0), freq=50)
led = machine.Pin(1, machine.Pin.OUT)

def lock():
#    pwm0.duty(50)
    led.value(1)

def drop():
#    pwm0.duty(100)
    led.value(0)

def flip():
    if led.value():
        drop()
    else:
        lock()

drop()
$ 

payload.py will run on ESP01s attached to drone, while drop.py will run on ESP01s attached to drone transmitter for remote command execution (drop payload). The drone ESP01s will run as AP (192.168.4.1), and the transmitter ESP01s will connect to it in station mode.

[HermannSW]

duplicate deleted

[WhiteHare]

@HermannSW Thanks for your post. What are your hardware assumptions? i.e. Will that work on even a plain vanilla BBC micro:bit? Or, does it assume Wi-Fi for creating the websockets, etc. ?

I'd like to find something that will run on the nRF5x series, which doesn't have built-in Wi-Fi, but does have Nordic's radio built in. My reason? Power consumption. Unless (?) someone has cracked how to do point-to-point radio on the ESP8266 or the ESP32 without going through a Wi-Fi router (where even just setting up the link consumes a lot of time and energy), then the power consumption on nRF5x sensor nodes is simply a lot lower.

[HermannSW]

Code: Select all

$ cat ../drop.py 
import uwebsockets.client
import os
import time

websocket = uwebsockets.client.connect('ws://192.168.178.122:8266/')

def wait_for_prompt():
    resp = ""
    while not(">>> " in resp):
        resp = websocket.recv()
        print("< {}".format(resp))

def do(cmd):
    websocket.send(cmd+"\r\n")
    wait_for_prompt()

resp = websocket.recv()
assert resp == "Password: ", resp

do("abcd")

do("import payload")

while True:
    do("payload.flip()")
    time.sleep(1)
$ 

I just wanted to see how the simple MicroPython drop.py script can be done on laptop, with Python or NodeJS.
Surprisingly those scripts are more complicated than drop.py script, although doing the same.

Python 2/3 script:

Code: Select all

$ cat drop_py23.py 
import websocket
try:
    import thread
except ImportError:
    import _thread as thread
import time

prompt_seen = False
prompt = "Password: "

def on_message(ws, resp):
    global prompt_seen
    prompt_seen = prompt in resp
    print("< {}".format(resp))

def on_error(ws, error):
    print(error)

def on_close(ws):
    print("### closed ###")

def wait_for_prompt(newPrompt):
    global prompt_seen
    global prompt
    prompt = newPrompt
    while not(prompt_seen):
        time.sleep(0.1)

    prompt_seen = False

def do(ws, cmd):
    ws.send(cmd+"\r\n")
    wait_for_prompt(">>> ")

def on_open(ws):
    def run(*args):
        wait_for_prompt('Password: ')

        do(ws, "abcd")
        
        do(ws, "import payload")
        
        while True:
            do(ws, "payload.flip()")
            time.sleep(1)

        ws.close()
        print("thread terminating...")
    thread.start_new_thread(run, ())


if __name__ == "__main__":
    websocket.enableTrace(False)
    ws = websocket.WebSocketApp("ws://192.168.178.122:8266/",
                              on_message = on_message,
                              on_error = on_error,
                              on_close = on_close)
    ws.on_open = on_open
    ws.run_forever()
$ 

NodeJS script:

Code: Select all

$ cat drop.js
var ws = new (require('ws'))('ws://192.168.178.122:8266/');

prompt_seen = false;
prompt = "Password: ";

function wait_for_prompt(newPrompt) {
    prompt = newPrompt
    wait_for_prompt_();
}

function wait_for_prompt_() {
    if (!prompt_seen) {
        setTimeout(wait_for_prompt_, 100);
    } else {
        prompt_seen = false
    }
}

function Do(ws, cmd) {
    ws.send(cmd+"\r\n")
    wait_for_prompt(">>> ")
}

function flip(ws) {
    Do(ws, "payload.flip()");
    setTimeout(flip, 1000, ws);
}

ws.onopen = function () {
    wait_for_prompt('Password: ');

    Do(ws, "abcd");
        
    Do(ws, "import payload");

    flip(ws);
}

ws.onmessage = function (ev) {
    prompt_seen = ev.data.includes(prompt)
    console.log("< "+ev.data);
}
$ 

[HermannSW]

@HermannSW Thanks for your post. What are your hardware assumptions? i.e. Will that work on even a plain vanilla BBC micro:bit? Or, does it assume Wi-Fi for creating the websockets, etc. ?

I don't have BBC micro:bit, but since MicroPython runs on micro:bit it can work as long as you have any network connectivity to the the other micro:bit. I did read in danny's uwebsockets github repo that ram size may be an issue for some ESP8266 systems.
https://github.com/danni/uwebsockets/issues/1
Since micro:bit only has 16KB of Ram you need to test (the ESP8266 controllers have 80KB).

I'd like to find something that will run on the nRF5x series, which doesn't have built-in Wi-Fi, but does have Nordic's radio built in. My reason? Power consumption. Unless (?) someone has cracked how to do point-to-point radio on the ESP8266 or the ESP32 without going through a Wi-Fi router (where even just setting up the link consumes a lot of time and energy), then the power consumption on nRF5x sensor nodes is simply a lot lower.

I don't know nRF5x, but googling for "mRF5x ethernet" gives quite some hits. As soon as your bbc:micro gets an IP address, the WebREPL solution will work. I once got ethernet on an Arduino Uno with an ENC28J60 ethernet module connected via SPI. Without ethernet your initially proposed solution seems to be the way to go.

[WhiteHare]

So, just to be clear, if I have an ESP8266 connected to, say, an nRF52832 on one end of the link (perhaps either the same or similar to: https://www.openhardware.io/view/443/nR ... os-D1-Mini), then, at the other end of the link, a target nRF5x device running micropython (but without any directly connected ESP or ethernet support) can be OTA updated using just its nRF5x radio? If so, then I'd be happy to give it a try. I just want to make sure that I'm understanding what's needed to make it work.

[HermannSW]

If you enable WebREPL on target nRF5x device running micropython with "import webrepl_setup", then WebREPL will run on that device listening on port 8266. Since you don't have ethernet support, you would not enable WebREPL on the ESP8266, and need a transparent proxy that would forward requests to 127.0.0.1:8266 on the ESP to port 8266 on the nRF5x over radio. I don't know whether such proxy exists somewhere already.

[WhiteHare]

Sounds like the critical piece is missing then. Thanks for your input though.

What does sound promising is Bluetooth LE REPL, which is referred to on the github: https://github.com/micropython/micropyt ... /ports/nrf

It also refers to WebBluetooth REPL (experimental)

Of course, neither would work for the BBC micro:bit, but since I now have micropython running on both an nRF52832 and an nRF52840, I'll can give them a try.

Alternatively..... Will the code for the micropython "radio" module (which is featured on the BBC micro:bit) also run on the nRF52832 and nRF52840? I don't know, but I can say that "radio" doesn't seem to be baked into the build:

Code: Select all

MicroPython v1.9.4-623-g34af10d2e on 2018-10-05; PCA10040 with NRF52832
Type "help()" for more information.
>>> import radio
Traceback (most recent call last):
  File "<stdin>", in <module>
ImportError: no module named 'radio'
>>> radio.on()
Traceback (most recent call last):
  File "<stdin>", in <module>
NameError: name 'radio' isn't defined
>>>

Also, looking at the micropython documentation (http://docs.micropython.org/en/latest/l ... twork.html), I don't see any reference to a radio module.

So, at this moment, I think getting Bluetooth REPL working is probably the shortest path to a solution.

[WhiteHare]

UPDATE:
I posted an inquiry about how to do Bluetooth REPL, but no answers as yet: viewtopic.php?f=12&t=5364

Maybe (?) it would be easier to port the Radio library from the micro:bit so that it runs on the nRF52x? I posted an inquiry about that here: viewtopic.php?f=15&t=5370

If that doesn't yield any replies either, then rather than loading a library I may have to take a stab at rolling my own radio code from within micropython.

At this moment in time I'm not sure which of the three paths will be the easiest.

[WhiteHare]

Yet another option may be to install Apache's MyNewt OS. According to their documentaton, it's an open source Bluetooth 5 software stack that completely replaces softdevice on the Nordic nRF52 series. I assume it must also be easier to use; otherwise, who would bother with it?

Anyone here ever use MyNewt OS? If so, how do you like it? How does micropython get loaded into it?