Can't connect to my board with rshell, File Size Issues with pyboard.py

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
walecka
Posts: 17
Joined: Wed Apr 08, 2020 9:50 pm

Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by walecka » Thu May 21, 2020 8:38 pm

Big Picture:
I am working on an STM32 B-L475E-IOT01A discovery board. I am trying to upload a few libraries I have written onto my board, and have run into issues with both rshell and pyboard.py

Rshell:
I have been unable to connect to my board via Rshell. whenever I enter the command

Code: Select all

rshell connect serial /dev/tty.usbmodem14113
I get the following results before the command line hangs until I press ^C

Code: Select all

Trying to connect to REPL  connected
Testing if sys.stdin.buffer exists ... 
I get the same result if I specify a baudrate of 115200, which is what I normally use to connect to the REPL.

When I enter the command:

Code: Select all

rshell -p /dev/tty.usbmodem14113
I get the following results before the the command line hangs:

Code: Select all

Using buffer-size of 32
Connecting to /dev/tty.usbmodem14113 (buffer-size 32)...
Trying to connect to REPL  connected
Testing if ubinascii.unhexlify exists ... 
Again, nothing changes if I specify a baudrate, or if I use the -a flag, as I have seen suggested in a few forum posts. I also have to perform a hard reset (unplug/replug-in my board) each either of these things happens. Hopefully someone on here has had experience with the same issue and can help me resolve this!

Pyboard.py:
My issue with the pyboard tool is that, while I am able to upload short snippets of code and run short scripts with it, I have issues when the python file size gets too big. When the file size exceeds 224 bytes, the command window hangs and again I must do a hard reset of the chip. I have written code in the REPL that works perfectly, but when I try to upload or run the exact same file using pyboard.py, I run into this issue, so I am pretty positive this is not related to errors in my code.
To further prove this, once I caught on to what was happening, I tested this empirically by creating a file where I simply assigned variables (a=1 b=2 etc...). This script uploads successfully until the file size reaches the 224 byte mark, where the issue arises again. Hopefully someone has an easy fix for this, as I seem to have discovered what is causing the problem.

Thanks in advance for the help! And for providing the entire uPython suite in the first place!

User avatar
dhylands
Posts: 3454
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by dhylands » Thu May 21, 2020 10:14 pm

rshell actually uses pyboard.py for its low level communications.

[EDIT: I discovered that the user manual says that an 8MHz crystal and a couple of capacitors need to be installed in order to use CN9 so most of my comments below won't apply]

Which connector are you using on the board? I would expect CN9 (USB_FS_OTG) to work better than CN7 (the STLink connection).

When using CN9, you shouldn't have to specify the serial port, rshell should just find it.

I don't have the STM32 B-L475E-IOT01A board, but I do have an STM32L4-DISCOVERY which has the same STM32L476 processor on it.

I was able to connect with rshell and copy a 10K file with no issues.

Also, try using the /dev/cu.usbmodem14113 (rather than /dev/tty.usbmodem14113) The /dev/tty.* ports are designed for incoming connections via a modem, where the /dev/cu.* ports are designed for outgoing connections and don't wait for DCD. Under some circumstances the /dev/tty.* ports will hang during opening.

This is what I get when connecting to my STM32L4-DISCOVERY:

Code: Select all

$ rshell
Connecting to /dev/cu.usbmodem206D357153362 (buffer-size 512)...
Trying to connect to REPL  connected
Testing if sys.stdin.buffer exists ... Y
Retrieving root directories ... /flash/
Setting time ... May 21, 2020 15:01:46
Evaluating board_name ... pyboard
Retrieving time epoch ... Jan 01, 2000
Welcome to rshell. Use Control-D (or the exit command) to exit rshell.
/Users/davehylands/micropython/micropython/ports/stm32> 
You can also use the rshell --list command to display your serial ports. If plugged into the CN9 connector on your board, you should see the board advertised as 'Micropython':

Code: Select all

$ rshell --list
Serial Device: /dev/cu.Bluetooth-Incoming-Port
Serial Device: /dev/cu.Dave-Headphones-SPPDev
USB Serial Device f055:9800 with vendor 'MicroPython' serial '206D35715336' found @/dev/cu.usbmodem206D357153362 *
How are you powering your board? From looking at the User manual, you can setup JP4 to use 5V_USB_FS and then it will power the board via the CN9 connection as well. It looks like 5V_ST_LINK is the default selection.

User avatar
dhylands
Posts: 3454
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by dhylands » Thu May 21, 2020 10:32 pm

How is the clocking setup for your board? I suspect that this might be the cause of the problems You could try using a lower baud rate (say 38400 or 19200) which would require a change in the micropython firmware. You'd need to change the MICROPY_HW_UART_REPL_BAUD in your boards mpconfigboard.h file. This line in the STM32L476-DISCOVERY board definition: https://github.com/micropython/micropyt ... oard.h#L50

walecka
Posts: 17
Joined: Wed Apr 08, 2020 9:50 pm

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by walecka » Thu May 21, 2020 11:22 pm

Hi David,

Thanks for getting back to me. I tried adjusting the baudrate and re-flashing my board but unfortunately that does not change the outcome. I can still use the REPL just fine, but rshell and pyboard both hang in the command line when trying to connect.

Last night I came across a forum post that described a different issue with rshell not working. It did not seem relevant at the time, but now I think it might be. The solution required adding a delay somewhere in either main.py or boot.py. Does this sound familiar?

User avatar
dhylands
Posts: 3454
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by dhylands » Thu May 21, 2020 11:32 pm

The delay thing was primarily for the ESP boards, which sometimes take a while booting (many reboot when you open the serial port).

I've since modified rshell to send a Control-C every 100 msec (for 20 times) waiting to get the >>> prompt.

On the STM32, it doesn't reset when opening the serial port, and if there is no main.py running then it's normally just sitting at the >>> prompt.

You could try adding a -d option to rshell, which causes a bunch more information to be output.

Of course, the first thing is to make sure you can connect using a regular serial terminal (perhaps screen) and get the >>> prompt. If you can't do that then rshell won't work.

walecka
Posts: 17
Joined: Wed Apr 08, 2020 9:50 pm

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by walecka » Thu May 21, 2020 11:43 pm

screen is the application that I normally use to connect to the REPL and get the >>> prompt, and that has always worked well.

when I input

Code: Select all

$ rshell -p /dev/tty.usbmodem14113 -d -a
into the command line, here is what I get:

Code: Select all

Debug = True
Port = /dev/tty.usbmodem14113
Baud = 115200
User = micro
Password = python
Wait = 0
List = 0
nocolor = 0
ascii = 1
Timing = 0
Quiet = 0
BUFFER_SIZE = 512
Cmd = []
Using buffer-size of 32
Connecting to /dev/tty.usbmodem14113 (buffer-size 32)...
Trying to connect to REPL  connected
Testing if ubinascii.unhexlify exists ... ----- About to send 304 bytes of code to the pyboard -----
def test_unhexlify():
    """Checks the micropython firmware to see if ubinascii.unhexlify exists."""
    import ubinascii
    try:
        _ = ubinascii.unhexlify
        return True
    except:
        return False
output = test_unhexlify()
if output is None:
    print("None")
else:
    print(output)

-----
I should also not that, as recommended in another forum post, I tried and was able to do the following in the REPL:

Code: Select all

>>>import ubinascii
>>> dir(ubinascii)
['__class__', '__name__', 'a2b_base64', 'b2a_base64', 'hexlify', 'unhexlify']
>>> 

User avatar
dhylands
Posts: 3454
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by dhylands » Thu May 21, 2020 11:55 pm

This seems to be related to the fact that you can't send more than 224 bytes (rshell was trying to send 304).

I have no idea why there would be a problem sending more than 224 bytes.

walecka
Posts: 17
Joined: Wed Apr 08, 2020 9:50 pm

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by walecka » Fri May 22, 2020 4:08 am

Maybe it has something to do with the buffer size? Truth be told I have no clue how the buffer size factors into a call to pyboard but the BUFFER_SIZE variable's value of 512 conflicts with the buffer-size of 32. Of course, neither of these is very close to 224...

Is there any possibility that messing with the buffer size when using rshell could make a difference?

User avatar
dhylands
Posts: 3454
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by dhylands » Fri May 22, 2020 5:00 am

Using a small buffer is needed when the device has a small buffer. The host can generally send data faster than the device can deal with it, so the buffer size should be smaller than the buffer on the receiving end. If you’re connected over a directly connected USB connection then 512 is ok. If there is a USB-to-serial converter on the middle (I.e using a uart) then you need to pick a buffer size smaller than the uart buffer.

Picking a buffer size that’s too small just makes rshell transfers take longer.

walecka
Posts: 17
Joined: Wed Apr 08, 2020 9:50 pm

Re: Can't connect to my board with rshell, File Size Issues with pyboard.py

Post by walecka » Fri May 22, 2020 7:13 pm

The B-L475E-IOT01AA Discovery board has a built-in ST-LINK/V2-1 that programs the actual STM32 using a uart (IE, USB->ST_LINK->UART->STM32), so I tried reducing the buffer size incrementally all the way down to 1. Same results, no joy - IE, rshell would not connect and pyboard.py could only send files with size of <=224 bytes. The same file size applies for different baud rates too, which definitely means there is something special about that number.

I am only now realizing that I have been overlooking the first part of the error message that I get when ^C-ing after

Code: Select all

Testing if ubinascii.unhexlify exists ...
The error is as follows

Code: Select all

Traceback (most recent call last):
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1307, in connect
    ip_address = socket.gethostbyname(port)
socket.gaierror: [Errno 8] nodename nor servname provided, or not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/SupremeOverlord/micropython/venv/bin/rshell", line 8, in <module>
    sys.exit(main())
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/command_line.py", line 4, in main
    rshell.main.main()
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 2966, in main
    real_main()
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 2928, in real_main
    connect(args.port, baud=args.baud, wait=args.wait, user=args.user, password=args.password)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1313, in connect
    connect_serial(port, baud=baud, wait=wait)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1337, in connect_serial
    dev = DeviceSerial(port, baud, wait)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1627, in __init__
    Device.__init__(self, pyb)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1391, in __init__
    unhexlify_exists = self.remote_eval(test_unhexlify)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1515, in remote_eval
    return eval(self.remote(func, *args, **kwargs))
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/main.py", line 1495, in remote
    output = self.pyb.exec_raw_no_follow(func_src)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/rshell/pyboard.py", line 248, in exec_raw_no_follow
    data = self.serial.read(2)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/serial/serialposix.py", line 483, in read
    ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())
KeyboardInterrupt
Perhaps the part at the top is important? I am not sure why the device ip_address is important, but the B-L475E-IOT01A is certainly not connected to Wifi, nor has it been given a statically allocated IP address.

That is the error message that I get when interrupting while trying to connect using rshell, and the following is what I get when interrupting while trying to send files directly via pyboard.py:

Code: Select all

Traceback (most recent call last):
  File "./pyboard.py", line 677, in <module>
    main()
  File "./pyboard.py", line 639, in main
    filesystem_command(pyb, args.files)
  File "./pyboard.py", line 505, in filesystem_command
    op(src, dest2)
  File "./pyboard.py", line 445, in fs_put
    self.exec_("w(" + repr(data) + ")")
  File "./pyboard.py", line 393, in exec_
    ret, ret_err = self.exec_raw(command, data_consumer=data_consumer)
  File "./pyboard.py", line 384, in exec_raw
    self.exec_raw_no_follow(command)
  File "./pyboard.py", line 379, in exec_raw_no_follow
    data = self.serial.read(2)
  File "/Users/SupremeOverlord/micropython/venv/lib/python3.7/site-packages/serial/serialposix.py", line 483, in read
    ready, _, _ = select.select([self.fd, self.pipe_abort_read_r], [], [], timeout.time_left())
KeyboardInterrupt
perhaps these will provide further insight

Post Reply