USB virtual com RTS/CTS?

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

USB virtual com RTS/CTS?

Post by kwiley » Fri Jun 08, 2018 9:35 pm

I am communicating with a PyBoard over USB from a desktop (Jupyter on a Mac if you're curious). The PyBoard is using pyb.USB_VCP to read and write byte streams and the desktop has opened serial communication to the PyBoard. This works fine so long as I close my terminal REPL so the data isn't redirected to the wrong location.

When I send data from the desktop the process seems to fall over at some point. I haven't determined the cause yet, but I have noticed that if I explicitly slow down the data stream from the desktop, the problem goes away. One theory is that the PyBoard's incoming data buffer is filling up from the desktop faster than the program can evacuate it for processing. Does the PyBoard not implicitly engage in some sort of RTS/CTS arrangement with the desktop so as to hold off data transmissions until the buffer has room? I'm not talking about baud issues. That is synced up by establishing the serial comm in the first place. I'm just asking how the PyBoard manages its incoming data buffer.

If it doesn't RTS/CTS its incoming buffer by default, then is there some way to enable such behavior? I don't see any mention of these issues in the USB_VCP documentation.

Obviously, I can implement an application-level synchronization such that the desktop software sends a little data, then reads from the serial port waiting for a go-ahead message from the PyBoard, then sends some more data, etc. but if such coordination can be accomplished at a lower level, I would prefer to do so.

Thanks.

User avatar
marfis
Posts: 215
Joined: Fri Oct 31, 2014 10:29 am
Location: Zurich / Switzerland

Re: USB virtual com RTS/CTS?

Post by marfis » Fri Jun 08, 2018 11:00 pm

if the receive circular buffer is full the pyboard will discard the rest of the data (and any following data frames as long as the data isnt processed by the pyboard). So no - there is no rts/cts type of flow control.

regardind jupyter integration - fyi: I‘ve an example here
https://github.com/hoihu/projects/tree/ ... Py/jupyter

it uses the mpy-repl-tools toolchain from chris liechti (downloadsble over pypi). docs: http://mpy-repl-tool.readthedocs.io/en/latest/

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: USB virtual com RTS/CTS?

Post by pythoncoder » Sat Jun 09, 2018 8:50 am

It's perhaps worth noting that two of the UARTs support flow control.
Peter Hinch
Index to my micropython libraries.

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: USB virtual com RTS/CTS?

Post by kwiley » Mon Jun 11, 2018 4:49 pm

Okay. Thanks. So are you saying that flow control is enabled by some of the UARTs, but there is no simple way to take advantage of them over a USB-to-desktop serial connection? So even though PyBoard can engage in flow control communication, it can't do with with a desktop for the purpose of shoveling data back and forth between the two?

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: USB virtual com RTS/CTS?

Post by kwiley » Mon Jun 11, 2018 4:58 pm

I see the following, which explains that UARTs 2 and 3 offer flow control:
http://docs.micropython.org/en/latest/p ... .UART.html

So, does USB_VCP rely on (utilize) one of the UARTs for its communications? I would generally assume it does, but perhaps that makes no sense, perhaps it doesn't use a UART to accomplish the USB communication. If it does do so, which UART does USB_VCP make use of? That isn't stated on:
http://docs.micropython.org/en/latest/p ... B_VCP.html

Thanks.

User avatar
marfis
Posts: 215
Joined: Fri Oct 31, 2014 10:29 am
Location: Zurich / Switzerland

Re: USB virtual com RTS/CTS?

Post by marfis » Mon Jun 11, 2018 8:40 pm

So, does USB_VCP rely on (utilize) one of the UARTs for its communications
No it doesn‘t. Most stm32 cpus have a built in USB controller. That makes them capable of directly handle the VCP protocol (in contrast to cpus with dont have this - they need a separate usb-serial driver chip which would then be connected to an uart module).

An interesting point: the baudrate of the vcp com port is pretty much ignored. Regardless of what you are set your terminal to it will always transmit as fast as possible over USB. I measured once up to 8Mbaud - close to the maximum of USB FS speed (but not guaranteed speed since the bus might be shared).

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: USB virtual com RTS/CTS?

Post by kwiley » Mon Jun 11, 2018 8:50 pm

Thank you. I had noticed the other discussions in the forums that explained that the baud rate was ignored, so I'm aware of that. I'm just trying to figure out what to do about the lack of flow control. It looks like I'll have to implement flow control at the application layer. I have to have the PyBoard detect corrupt messages (presumably due to buffer overflow) and have it send a message back to the desktop informing the sender to resend any messages after the corruption occurred.

Hmph. :-)

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: USB virtual com RTS/CTS?

Post by pythoncoder » Wed Jun 13, 2018 7:04 am

One approach would be to implement a half-duplex protocol with one end of the link being designated the master: only it can send unsolicited messages. The master sends a request for data and awaits a response. The slave responds when it is ready. If the master needs to send a block of data too large for the receive buffer, it sends a request. The slave acknowledges this and goes into listen mode.

All requests from master to slave are designed to be short enough to fit in the buffer. That way the slave does not need to constantly service the interface (neither does the master).

The other approach would be to use a UART via an FTDI USB adaptor. You could then use uasyncio's StreamReader and StreamWriter objects to enable background communication. There may be a way to do this with the onboard USB but I've never tried this; any fiddling with the onboard USB is liable to compromise the reporting of runtime exceptions. For this reason I'd tend to favour a separate communication channel for data.
Peter Hinch
Index to my micropython libraries.

User avatar
marfis
Posts: 215
Joined: Fri Oct 31, 2014 10:29 am
Location: Zurich / Switzerland

Re: USB virtual com RTS/CTS?

Post by marfis » Wed Jun 13, 2018 8:06 pm

One thing to try out is to NACK the USB packet if the rx buffer is full (or earlier, so that the received packet can still be fully stored in the ringbuffer).

NACKing the USB packet solves flow control issues on the USB PHY level, without needing to configure flow control on the appliacation layer.

IfI you are willing to give it a try, you may start here:
https://github.com/micropython/micropyt ... ace.c#L233

and return USBD_BUSY instead of just breakin the loop (and continue receiving data)

If I find some time, I can also look into this. Seems like a nice feature.

kwiley
Posts: 140
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: USB virtual com RTS/CTS?

Post by kwiley » Wed Jun 20, 2018 3:51 pm

Thanks for these later responses. There are some good ideas here. I'm balancing the various pros and cons of all the ways to get this running smoothly. I'd hate to have to reinvent, at a high level, protocols that are generally presumed on offer by lower layers after decades of standardization and convention, but I admit that products like MicroPython and PyBoard are still under development and therefore may have some rough edges that are best handled directly on a per-application basis.

Cheers!

Post Reply