possible to send command directly to WebREPL

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
skimj
Posts: 3
Joined: Thu Mar 09, 2017 5:25 am

possible to send command directly to WebREPL

Post by skimj » Thu Mar 09, 2017 5:13 pm

Is there any trivial way to programmatically send a single command directly to WebREPL?

For a simple, brute example, imagine that you need to reset the device. Your options are: 1) physically press reset button or 2) start a browser with the WebREPL client, press connect, login, and finally type:`machine.reset()` at the terminal prompt. It would be really cool if it were possible to do something like: `webrepl_cli.cmd('machine.reset()')`

Following the webrepl_cli.py code, I'm able to create a connection, login and get the remote WebREPL version. webrepl_cli.send_req() uses a coded 'op' of 1, 2, or 3 to simplify the communication. Possibly, I could add an option 4 and hard-code the desired command in the micropython device. (I can't find the receiving end of this WebREPL code)

Obviously, the WebREPL http client ternimal is capable of sending individual characters, same as if they were typed at a REPL prompt, which tells me there would at least be a way to send raw socket data to accomplish the same goal? I don't have a clue how to even approach this method.

Any other ideas?

User avatar
aivarannamaa
Posts: 171
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: How to send commands directly to WebREPL? Raw-mode?

Post by aivarannamaa » Sun Jun 10, 2018 12:12 pm

I'm also interested in accessing the REPL programmatically over network, instead of serial.

I noticed that WebREPL doesn't respond to Ctrl+A (but does respond to Ctrl+B). Is it a browser issue? If I implement my own client for the WebREPL interface, can I put it into raw mode by sending a `\x01` ?
Aivar Annamaa
https://thonny.org

User avatar
aivarannamaa
Posts: 171
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: possible to send command directly to WebREPL

Post by aivarannamaa » Sun Jun 10, 2018 3:26 pm

Looks like WebREPL can be used quite like serial REPL, including raw-mode, interrupts etc.

Here's my proof-of-concept client script (with https://github.com/websocket-client/websocket-client). Note that here you need to manually suffix each command in normal mode with \n and in raw mode with \x04:

Code: Select all

import websocket
import threading
from time import sleep

def on_message(ws, message):
    print(message, end="")

def on_close(ws):
    print("### closed ###")

websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://192.168.1.226:8266", on_message = on_message, on_close = on_close)
wst = threading.Thread(target=ws.run_forever)
wst.daemon = True
wst.start()

conn_timeout = 5
while not ws.sock.connected and conn_timeout:
    sleep(1)
    conn_timeout -= 1
    
while ws.sock.connected:
    inp = (input()
           .replace("\\n", "\r\n")      # end of command in normal mode
           .replace("\\x01", "\x01")    # switch to raw mode
           .replace("\\x02", "\x02")    # switch to normal mode
           .replace("\\x03", "\x03")    # interrupt
           .replace("\\x04", "\x04"))   # end of command in raw mode

    if inp == "exit":
        ws.close()
    else:
        ws.send(inp)
Aivar Annamaa
https://thonny.org

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Wed Sep 05, 2018 8:15 pm

Hi aivarannamaa,

your script looks exactly like what I was searching for!

I tried it with python on Raspberry, but get this error:

Code: Select all

$ python webrepl_client.py 
  File "webrepl_client.py", line 8
    print(message, end="")
                      ^
SyntaxError: invalid syntax
$ 
Then I ran the script with python3, and got this error:

Code: Select all

$ python3 webrepl_client.py 
Traceback (most recent call last):
  File "webrepl_client.py", line 13, in <module>
    websocket.enableTrace(True)
AttributeError: module 'websocket' has no attribute 'enableTrace'
$ 
What am I doing wrong?
I hat to pip/pip3 install websocket first which worked fine.

P.S:
I modified that script print statement for python, now python runs into same enableTrace error as python3 ...
Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Wed Sep 05, 2018 8:52 pm

My fault.

Because of print statement your script is for python3.
After I did "pi3 install websocket_client" it works!
I only had to change IP address to 192.168.4.1 to connect to MicroPython webrepl on a 1MB ESP-01s module.
For some reason I have to press just enter key after a command completed in order to get repl prompt in your client, on the ESP-01s screen session it appears without.
But it works, as long as I do not forget to add "\n" suffix.
I cannot test webrepl_client now on my target platform (ESP32 Micropython), will do when back home.
I used Micropython as debug shell for ESP32 sofar (and its OLED):
https://esp32.com/viewtopic.php?t=3476
Now I want to use that ESP32 and its OLED with your webrepl client to connect to webrepl of ESP-01s flying attached to E52 drone [to trigger servo bomb drop mechanism ;-)].

Code: Select all

$ python3 webrepl_client.py 
--- request header ---
GET / HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Host: 192.168.4.1:8266
Origin: http://192.168.4.1:8266
Sec-WebSocket-Key: gdpLpeA8jYju6OimuMhOlw==
Sec-WebSocket-Version: 13


-----------------------
--- response header ---
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: 5ySptvGxJqqRWr4n3LtuPzIMhoU=
-----------------------
Password: 12345678\n
send: b'\x81\x8a\xa3\xe3\x82\x1a\x92\xd1\xb1.\x96\xd5\xb5"\xae\xe9'

WebREPL connected
>>> 2**3\n
send: b'\x81\x86\xe0c\xc9\xeb\xd2I\xe3\xd8\xedi'
2**3
8

send: b'\x81\x80\xa0\x1a\xfc\xee'
>>> uos.listdir()\n
send: b'\x81\x8f\x8b\x0bo8\xfed\x1c\x16\xe7b\x1cL\xefb\x1d\x10\xa2\x06e'
uos.listdir()
['boot.py', 'webrepl_cfg.py', 'utelnetserver.py']

Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Fri Sep 14, 2018 1:32 am

Hi aivarannamaa,

after getting used to the suffixes your webrepl client is really cool, after I fixed the mentioned REPL prompt flushing issue. This is the small diff to your posted code (192.168.4.1 is default ESP AP):
https://stamm-wilbrandt.de/en/forum/webrepl_client.py

Code: Select all

$ diff webrepl_client.orig.py webrepl_client.py 
0a1,5
> #!/usr/bin/env python3
> #
> # console webrepl client from aivarannamaa:
> # https://forum.micropython.org/viewtopic.php?f=2&t=3124&p=29865#p29865
> #
6c11
<     print(message, end="")
---
>     print(message, end="", flush=True)
12c17
< ws = websocket.WebSocketApp("ws://192.168.1.226:8266", on_message = on_message, on_close = on_close)
---
> ws = websocket.WebSocketApp("ws://192.168.4.1:8266", on_message = on_message, on_close = on_close)
$ 
I tested all these options, "\x03" breaks endless loops in remote MicroPython, pressing CTRL-C exits webrepl_client.py:

Code: Select all

           .replace("\\n", "\r\n")      # end of command in normal mode
           .replace("\\x01", "\x01")    # switch to raw mode
           .replace("\\x02", "\x02")    # switch to normal mode
           .replace("\\x03", "\x03")    # interrupt
           .replace("\\x04", "\x04"))   # end of command in raw mode
Today I finished soldering of my ESP-01s MicroPython servo bomb drop mechanism successsfully:
https://twitter.com/HermannSW/status/10 ... 4566469632

I used webrepl_client.py already as remote MicroPython shell to control bomb drop mechanism ...
Image


... from my mobile Pi ZeroW:
Image


P.S:
Here is screenshot of mobile Pi ZeroW remote servo control MicoPython session:
Image
Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Fri Sep 14, 2018 8:25 am

I did one more small change, adding "-s" silent flag, allowing to hide the websocket trace messages:
https://stamm-wilbrandt.de/en/forum/webrepl_client.py

Code: Select all

$ diff webrepl_client.py.old webrepl_client.py
5a6
> import sys
16c17,23
< websocket.enableTrace(True)
---
> silent = False
> for i in range(len(sys.argv)):
>     if sys.argv[i] == '-s':
>         silent =  True
>         break
> 
> websocket.enableTrace(not silent)
$ 

Much less output this way for webrepl session:
Image
Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Fri Sep 14, 2018 8:46 am

aivarannamaa, I just commented on a webrepl repo feature request issue asking for what you provided:
... I don't like using a web page as a terminal. It's pretty slick, but I'm very good at using a terminal. I would like to be able to do the webREPL from the command line. ...
https://github.com/micropython/webrepl/ ... -421275959
Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Sat Sep 15, 2018 6:54 pm

I made further changes:
https://stamm-wilbrandt.de/en/forum/webrepl_client.py

First I avoided output of command line a second time as can be seen in previous screenshot.
This is the new on_message function:

Code: Select all

inp = ""
def on_message(ws, message):
    global inp
    if silent:
        while (inp != "") and (message != "") and (inp[0] == message[0]):
            inp = inp[1:]
            message = message[1:]
        if (message != ""):
            inp = ""
    print(message, end="", flush=True)

Next I added some checks for ws.sock for graceful endings in some situations.
And finally I added try/except for graceful ending on pressing CTRL-C.

Here you can see a complete webrepl_client.py session:
  • shows commands without duplicate output
  • endless loop on remote MicroPython
  • terminating remote endless loop by CTRL-C remotely, which is done by \x03
  • normal command, followed by pressing CTRL-C ending webrepl_client.py
    (you can enter "exit" as well)
Image
Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

HermannSW
Posts: 197
Joined: Wed Nov 01, 2017 7:46 am
Contact:

Re: possible to send command directly to WebREPL

Post by HermannSW » Sat Sep 15, 2018 7:34 pm

One convenient further change, only for silent mode -- make the "\n" suffix superfluous!
https://stamm-wilbrandt.de/en/forum/webrepl_client.py

While "\x0[1-4]" suffixes are still needed, getting rid of "\n" suffix is cool:

Code: Select all

$ ./webrepl_client.py -s
Password: 12345678

WebREPL connected
>>> 4**3**2
262144
>>> \x01

raw REPL; CTRL-B to exit
>\x02

MicroPython v1.9.4-481-g3cd2c281d on 2018-09-04; ESP module with ESP8266
Type "help()" for more information.
>>> 
>>> exit
### closed ###
$ 
P.S:
Although password is not shown anymore on input, WebREPL keeps being unsafe because the password can be looked up in MicroPython qstr pool, see this thread for what the security issue is:
viewtopic.php?f=2&t=5284
Last edited by HermannSW on Mon Sep 24, 2018 3:13 pm, edited 2 times in total.
Pico-W Access Point static file webserver:
https://github.com/Hermann-SW/pico-w

Tiny MicroPython robots (the PCB IS the robot platform)
viewtopic.php?f=5&t=11454

webrepl_client.py
https://github.com/Hermann-SW/webrepl#webrepl-shell

Post Reply