working with UART and Bytearrays

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
Sokrates
Posts: 15
Joined: Mon Dec 14, 2015 11:24 pm

working with UART and Bytearrays

Post by Sokrates » Tue Dec 22, 2015 7:50 pm

Hi all,

I'm working with bytes; I have to read 25bytes frames from Uart.

At the moment I'm using the following approach and it's working:

1)byte array init:
myFrame = bytearray()

2)read 1 byte from uart:
tmpByte = uart.read(1)

3) add the byte to the bytearry
myFrame.extend(tmpByte)

4) when i get the last byte of the frame, process the content and then destroy the byte array:
myFrame = bytearray()

I would like to adopt another approach in order to avoid destroying the bytearray, so I tried:

1) 25 byte array init:
myFrame = bytearray(25)

2)read 1 byte from uart:
tmpByte = uart.read(1)

3) add the byte to the bytearry in the "frameIndex" location
myFrame[frameIndex]=tmpByte

the problem is that i get the following error on the last operation: can't convert byte to int.

Shouldn't be tmpByte and myFrame[x] already od type byte?

Where is the error?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: working with UART and Bytearrays

Post by pythoncoder » Wed Dec 23, 2015 5:52 am

I suggest you look at uart.readinto(buf[, nbytes])

To answer your question, uart.read(1) returns a bytes object with a single element (a bytes object is like bytearray, but is immutable). So you'd need to write uart.read(1)[0] to get a byte. But readinto() allows you to read a specified number of bytes into a pre-allocated buffer at a stroke and is the way I'd do it.

Code: Select all

myFrame = bytearray(25)
uart.readinto(myFrame) # job done
Peter Hinch
Index to my micropython libraries.

Sokrates
Posts: 15
Joined: Mon Dec 14, 2015 11:24 pm

Re: working with UART and Bytearrays

Post by Sokrates » Wed Dec 23, 2015 11:29 am

pythoncoder wrote:I suggest you look at uart.readinto(buf[, nbytes])

To answer your question, uart.read(1) returns a bytes object with a single element (a bytes object is like bytearray, but is immutable). So you'd need to write uart.read(1)[0] to get a byte. But readinto() allows you to read a specified number of bytes into a pre-allocated buffer at a stroke and is the way I'd do it.

Code: Select all

myFrame = bytearray(25)
uart.readinto(myFrame) # job done
Thanks a lot. I'll definitely follow your advice.

EasyRider
Posts: 94
Joined: Wed Dec 30, 2015 8:17 am

Re: working with UART and Bytearrays

Post by EasyRider » Fri Jan 15, 2016 11:16 am

x
Last edited by EasyRider on Wed Mar 13, 2019 12:53 pm, edited 1 time in total.

EasyRider
Posts: 94
Joined: Wed Dec 30, 2015 8:17 am

Re: working with UART and Bytearrays

Post by EasyRider » Fri Jan 15, 2016 11:23 am

x
Last edited by EasyRider on Wed Mar 13, 2019 12:53 pm, edited 1 time in total.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: working with UART and Bytearrays

Post by dhylands » Fri Jan 15, 2016 4:49 pm

There are 3 things that can cause uart to return.

1 - A timeout occurs. This is controlled by the timeout= parameter to UART.init and defaults to 1000 milliseconds (1 second). The controls the time that the uart waits for the first character to arrive
2 - An inter-character timeout occurs. This is controlled by the timeout_char= parameter. Once the uart has started to receive characters this is the amount of time betweem characters. It looks like there is no way to turn this timeout off. You can pass in large values, like 65535 which will wait for upto 65 seconds between characters.
3 - The desired amount of data is read.

In any of the above scenarios, it will return the actual number of characters read.

When you use readinto it will always start at offset 0 of the buffer you pass in, so if you call uart.readinto(reading) then the newly read data will occupy positions 0 thru the number of the bytes read (minus 1).

If you want to fill the buffer and also deal with partial reads, then you'l need to use a combination of readinto and memoryview. Something like the following:

Code: Select all

import pyb
from binascii import hexlify

u = pyb.UART(1, 115200, timeout_char=500)

buf = bytearray(8)
mv = memoryview(buf)
idx = 0

while idx < len(buf):
    if u.any():
        bytes_read = u.readinto(mv[idx:])
        print('Got {} bytes of data'.format(bytes_read), hexlify(buf[idx:idx+bytes_read], b':'))
        idx += bytes_read
print('Final buffer =', hexlify(buf, ':'))
I used timeout_char of 500 just so that I could type in multiple characters at human speed and get the number of characters I needed. The results I got were:

Code: Select all

Got 1 bytes of data b'61'
Got 3 bytes of data b'62:63:64'
Got 2 bytes of data b'65:66'
Got 2 bytes of data b'67:68'
Final buffer = b'61:62:63:64:65:66:67:68'
With a normal bytearray when you slice it (accessing a sub-range is called a slice) by doing buf[idx:] that winds up creating a new bytearray object with the contents from idx to the end. So if you modify this new bytearray, it doesn't modify the original.

With a memoryview, when you slice it, it creates a sub-memoryview whose indicies start at zero, but whose data is the original data. This allows you to modify the original data. For example:

Code: Select all

>>> buf = bytearray('0123456789')
>>> buf[3:6] = b'abc'
>>> buf
bytearray(b'012abc6789')
>>> mv = memoryview(buf)
>>> bytes(mv)
b'012abc6789'
>>> mv[3:6] = b'abc'
>>> buf
bytearray(b'012abc6789')

EasyRider
Posts: 94
Joined: Wed Dec 30, 2015 8:17 am

Re: working with UART and Bytearrays

Post by EasyRider » Fri Jan 15, 2016 9:38 pm

x
Last edited by EasyRider on Wed Mar 13, 2019 12:54 pm, edited 1 time in total.

EasyRider
Posts: 94
Joined: Wed Dec 30, 2015 8:17 am

Re: working with UART and Bytearrays

Post by EasyRider » Sat Jan 16, 2016 6:24 am

x
Last edited by EasyRider on Wed Mar 13, 2019 12:54 pm, edited 1 time in total.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: working with UART and Bytearrays

Post by dhylands » Sat Jan 16, 2016 6:33 am

To have the the data go into separate bytearrays then you need to limit the size of the reads.

In my experience, trying to do multi-byte reads will inevitably screw up somewhere down the line because the timing won't be exactly what you expect.

I always write my uart receivers to either read one byte at a time and build up whatever data needs to be builtup, or possibly to read a larger buffer but still process it one character at a time.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: working with UART and Bytearrays

Post by pythoncoder » Sat Jan 16, 2016 8:11 am

dhylands wrote:... For example:

Code: Select all

>>> buf = bytearray('0123456789')
>>> buf[3:6] = b'abc'
>>> buf
bytearray(b'012abc6789')
>>> mv = memoryview(buf)
>>> bytes(mv)
b'012abc6789'
>>> mv[3:6] = b'abc'
>>> buf
bytearray(b'012abc6789')
Perhaps not the most convincing demo of the memoryview as assigning to it and the buffer slice resulted in the same outcome ;) This perhaps illustrates it better:

Code: Select all

>>> a = bytearray(b'0123456789')
>>> m = memoryview(a)
>>> z = m[3:6] # z is not a copy
>>> z[:] = b'jkl'
>>> a
bytearray(b'012jkl6789')
>>> 
The point of course being that z is a window into the original bytearray. If z had been declared as a[3:6] it would have been a copy, and assigning to it would have left a unaffected:

Code: Select all

>>> p = a[7:9]
>>> p
bytearray(b'78')
>>> p[:] = b'xy'
>>> p
bytearray(b'xy') # p has changed
>>> a
bytearray(b'012jkl6789') # a has not
>>> 
Peter Hinch
Index to my micropython libraries.

Post Reply