working with UART and Bytearrays

[dhylands]

Yeah - you're right.

I was trying to point out that if you do:

Code: Select all

uart.readinto(buf[3:6])

then that won't modify the underlying buffer (it will instead modify a temporary slice of the buf), but if you do:

Code: Select all

uart.readinto(mv[3:6])

then that will modify the underlying buffer.

[EasyRider]

Thanks,

In my experience, trying to do multi-byte reads will inevitably screw up somewhere down the line because the timing won't be exactly what you expect.

Timing/synchronizing to appears to be an issue that I am seeing. I am expecting reasonable synchronized writes and reads from the uart port in this case

Single bytearray write read work OK.

With multiple bytearray write/read, timing appears as the issue. Write bytearray then go read response , even with readchar() it doesn't start reading the bytearray at first byte it may start somewere in the middle of the array.

Code: Select all

uart.write(bytearray(message)) 
if uart.any(): #YES, complete bytearray/continuous packet is available after about 5ms.
     for i in range(0, 15):
          response3[i]=uart.readchar()  

With above code read start byte 0 (response3[0]) is actually the 9th byte of the bytearray at the RX port, then it reads the remainder of that bytearray and the rest of the response3 (15 bytes total) is filled from the next bytearray at the RX port that may not arrive until whenever (>1000ms).
I would expect that reponse3 would read the whole bytearry at the uart Rx port , considering that bytearray (as continuous packet) is available to be read about 5ms after uart.write. This is ONLY the case if I have multiple write read in a loop as I have explained in previous post. However, a single write read in a loop works perfectly OK.

My problem is that I can't use known reference start/stop bytes or known length bytearrays to be able to read X bytes in a read buffer then process it. I have to relay on reasonable synchronized read timing.

[dhylands]

Do you have 2 pyboards connected together? Or are you looping back with one?

Reading waiting for the gaps should also work, but maybe not with one pyboard looped back to itself.

When you specify an rx buffer length > 0 then the chars are buffered via IRQ into the buffer and all sense of timing is lost.

With pyboard looped back on itself, the write is synchronous, so by the time the write finishes, all of the bytes will already be in the rx buffer.

I think I'd have to see the complete code for the reader and writer and know whether you've got one or 2 pyboards connected together to explain what you're seeing.

[EasyRider]

Thanks Dave,

No Pyboard loopback.

At this stage Pyboard is the master with this issue and response is generated by PC app verified.
I have the digital scope with serial Uart decoding so signal integrity and timing checks OK.

Also correction to previous post. I do know the length of the expected bytearray/packets to be received and would also know the starting byte but the starting byte is not unique it can be same as other bytes in the bytearray.

Is there a way I can send you code as PM rather than posting here?

Regards
John

[dhylands]

You could send it via email to dhylands@gmail.com or pastebin it and post a link here

[EasyRider]

I will try to simplify this to work with 2 pyboards to make it easier to evaluate.
Will let you know.

Regards
John

[Sokrates]

I will try to simplify this to work with 2 pyboards to make it easier to evaluate.
Will let you know.

Regards
John

I had the same issue.
In my project i have to read 25byte frames but I set the buffer read_buf_len to 250. In my understanding it is the "physical" buffer and when you call readinto(myByff, 25) the first 25 bytes are copied from that buffer to your myBuff var. Setting read_buf_len to 250 gave me some timing-related robustness.
My frame has known start/stop bytes so i know exactly when i received a valid frame.

I initially call a method sync() that read all the byte until it finds a the start byte. Then it checks if after 25 bytes there is the end bytes. If the check is true we have the sync so the application start reading 25 bytes. if it receives a frame with the first byte different from start byte and the last byte different from the end byte it mark the frame as lost. if the number of lost frame goes above a threshold it start the sync process again.

For me this works since i can afford loosing some frames.

Please anybody feel free to correct me if I'm wrong

[pythoncoder]

The problems under discussion are endemic to serial comms and warrant some thought before you start coding.

To set up a robust serial link between two devices usually involves considering the case where one end of the link is power cycled, temporarily disconnected or reset. At the very least the two ends are likely to start up asynchronously. You need a design where synchronisation can be achieved after a disruption: a disruption which may involve the receiver only getting part of a frame. @Sokrates has implemented one approach, having a fixed frame length and a unique start character. Another approach is to use a half duplex request-response protocol where the receiver sends a request character to the transmitter, which responds with a frame. This avoids the need for a unique start character and is the approach I'd recommend if it's possible.

@EasyRider, your approach lacking a start character and relying on timing, is inherently tough to get right in the presence of disruptions. If at all possible I'd use another way.

The other issue is frame length: if it can't be fixed at design time the message format needs a data value (at the start of the message) to specify the total length of data to be transmitted. The other thorny problem is flow control, but hopefully this isn't required here.

[EasyRider]

Been away last week and also cleared my head a bit.

After dissecting the code a bit more and testing timing with the scope.

Problem is definitely in synchronizing the reading of the response bytearray.

With the code below , response would not read the bytearray that I was expecting as discussed in previous posts.

Code: Select all

 uart.write(bytearray(message)) 
 if uart.any():
      response= uart.read(ulen)
 

By introducing a delay before testing for available data to be read has temporarily fixed the problem.
I am still looking at details to fully understand what is going on with timing, and want to get rid of the blocking delay().

Code: Select all

 uart.write(bytearray(message)) 
 pyb.delay(10)
 if uart.any():
      response= uart.read(ulen)
 

pythoncoder has good suggestions and will try that later.

I would also like to set up UART read under interrupt in Micropython.
Something like an interrupt "callback()" rather than testing for read availability" if uart.any():", can't find anything in documentation ?

Regards
John

[dhylands]

uart.any returns if any characters are available. If it executes immediately after the write, its possible that your response may not have arrived yet.

Depending on the how the timeouts are setup the read may not return all of the bytes that you want.

If you want to read ulen bytes, then you'll need a loop.