Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Questions and discussion about running MicroPython on a micro:bit board.
Target audience: MicroPython users with a micro:bit.
Post Reply
User avatar
aivarannamaa
Posts: 171
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Post by aivarannamaa » Fri Sep 22, 2017 3:45 pm

Hi!

Let's say my PC program is connected to microbit USB serial port (using PySerial library) in order to communicate with the program running on microbit.

The desktop program thinks the microbit program requires input and starts writing something to the serial. Now it turns out that the microbit program was not waiting for input yet, but continued writing something to the PC.

Can this kind of simultaneous write from both end lead to data corruption?

If this is not a problem, then I assume that when one end sends lot of data, the other end can't wait too much with reading the data, otherwise somewhere some buffers become full. Can you provide more details on this issue?

I'm asking, because I helped developing https://bitbucket.org/KauriRaba/thonny-microbit and during testing we occasionally found that while communicating with raw-REPL, some data sent from the PC to Micro:bit became corrupted. Can you give me some recommendations, how to track down the reason for this?

I know that by checking output terminator bytes, I can talk with raw-REPL in half-duplex mode, ie. avoiding writing from both directions at the same time. But I'd like to be able to respond correctly also when the REPL is running an input(...) command, ie. when the switch from outputting to waiting the input is not visible to the other end.

best regards,
Aivar

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Post by deshipu » Fri Sep 22, 2017 11:24 pm

Micro-bit uses a full-duplex serial connection, so communication in one direction doesn't affect the communication in the opposite direction — they are done on separate wires, and have their own buffers. However, the problem you mention with data corruption when too much data is sent too fast is a real problem. Normally this would be solved with a hardware flow control mechanism, where additional wires would be used to signal between the two devices when buffers become full and the communication needs to be paused. This is not the case with the micro-bit, and the buffers overflow, you start losing data. The solution to this is to send data in small chunks and with enough delay for the devices to process it all. The exact speed and delays need to be determined experimentally, because they would depend on what your program is actually doing and how fast.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Post by pythoncoder » Sat Sep 23, 2017 4:47 am

Even better is to have a communications protocol where each end sends an acknowledgement (ACK) for every small chunk received. This ensures that overruns can never occur. When complete the transmitter sends EOT (end of transmission) to enable the current receiver to assume the role of transmitter if it becomes necessary.

This can be taken further in the case of an unreliable link. The receiver sends ACK or NACK (negative ACK) in the case of failure. In the latter case the transmitter re-sends the block. Checksums can further improve reliability.

This is just the outline of a protocol - it needs to be designed in the context of link reliability, buffer size and the application.

The penalties of this approach are that you're restricting the link to half-duplex unless you design the protocol with great care. You're also restricting the throughput: if the receiver is busy the transmitter stalls waiting for ACK. However if the alternative is data corruption there may be no real choice.
Peter Hinch
Index to my micropython libraries.

User avatar
aivarannamaa
Posts: 171
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Post by aivarannamaa » Sat Sep 23, 2017 5:58 am

Thanks a lot!

I was thinking about similar schemes, but worried that these might be more complex than necessary and I'm missing a shortcut. I'll continue experimenting with different sized chunks and different waiting times.

mwm
Posts: 36
Joined: Wed Aug 09, 2017 3:52 pm

Re: Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Post by mwm » Sun Sep 24, 2017 3:08 pm

I've done quite a bit of protocol design work, and there are some techniques that can help with things.

For instance, you can design the protocol so that it includes acks in the data packets. Having options to indicate "all packets to X received OK" as well as "nothing to ack at this time" can be useful for these.

You can keep sending data rather than simply blocking waiting for an ack. This requires you be able to recreate - or just keep around - packets before they get acked if you want to resend them. At some point, you'll block - but this puts it off for a while. It also requires that the receiver be able to deal with out of order packets.

Consider whether or not you want to resend corrupt packets at all. I'm doing an RC protocol currently, and many of the values go stale faster than I transmit packets. There's no point in resending bad packets; the old data has no value, and I'll have new values to send well before I could get a nack back. I've also had systems that did end-to-end checking on relatively reliable media. A packet lost would be seen at one end, triggering a resend of that packet from the other even if the intermediate layers discarded errors.

Don't, don't, don't run multiple independent layers of checking. Triggering resends of the same packet at two different layers in the stack is a good way to kill your throughput. So if you're thinking about maybe using a reliable media - say going to TCP over WiFi - then have a way to modify your checking to deal with complete breakdowns appropriate for that media. Or maybe use an unchecked version of the media, like UDP over Wifi.

User avatar
titimoby
Posts: 18
Joined: Thu Sep 29, 2016 6:54 am

Re: Are simultaneous writes from both ends of the serial interface safe in micro:bit?

Post by titimoby » Fri Sep 29, 2017 9:25 am

Maybe you can get some inspiration by looking at David work on Bitio :
https://github.com/whaleygeek/bitio

Post Reply