uart.read() is giving accumulated data

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
Duality
Posts: 16
Joined: Tue Jun 09, 2020 2:43 pm

uart.read() is giving accumulated data

Post by Duality » Wed Jan 06, 2021 12:33 pm

Hello All,

I am using micropython on esp32. I have got a query regarding uart.read() function. Here is a basic code for a uart read:

Code: Select all

from machine import UART, Pin
import utime, uselect
u2 = UART(2, 9600, stop=1, tx=23,rx=22, txbuf=512, rxbuf=512, timeout_char=2)
count=0

while True:
    poll = uselect.poll()
    poll.register(u2, uselect.POLLIN)
    poll.poll()
    print('query: ', u2.read(), ' count: ', count, ' buffer: ', u2.any())
    utime.sleep_ms(100)
    count+=1
I am sending 8 bytes every second on these pins (checked with an oscilloscope). I do not get data each second. In fact, I get data of 120 bytes after 15-16 seconds.

Here is the REPL log:
4.JPG
4.JPG (70.19 KiB) Viewed 337 times
Please help.

Regards

Duality
Posts: 16
Joined: Tue Jun 09, 2020 2:43 pm

Re: uart.read() is giving accumulated data

Post by Duality » Thu Jan 07, 2021 5:09 am

Hi All,

This is the updated problem statement. I am doing a loopback in this case (in earlier scenario, I was using a different device which was sending the UART data).

Code: Select all

from machine import UART
import utime, uselect
u1 = UART(1, 9600, tx=17,rx=16, parity=None, stop=1, )
u2 = UART(2, 9600, tx=23,rx=22)

#wait = ((8*8)/9600)

while True:
    u1.write('hello113')
    
    # if(u2.any()!=0):
    # poll = uselect.poll()
    # poll.register(u2, uselect.POLLIN)
    # print(poll.poll())
    print(u2.any(), u2.read())
    #utime.sleep_us(10000)
The REPL shows this repeatedly:
None

Code: Select all

0 None
0 None
0 None
0 None
0 None
0 None
0 None
120 b'hello113hello113hello113hello113hello113hello113hello113hello113hello113hello113hello113hello113hello113hello113hello113'
0 None
0 None
0 None
0 None
0 None
0 None
The buffer sizes are 512 bytes. Also, I see that the HW FIFO size is 128 bytes. I am only writing 8 bytes, hence, there is no overflow. Also, I am reading it as many times as I write. whenever the uart RX FIFO gets any bytes, the function should output it. But, I am only seeing the read output 120 bytes together.

When I put utime.sleep() with wait time above a particular value, then the read function works as expected.
Am I missing something here? The data is not lost, means whatever u2 is receiving, it is getting stored somewhere (may be HW FIFO). But why the uart.read() is not outputting, if the data is stored somewhere ?

Regards

User avatar
pythoncoder
Posts: 4929
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uart.read() is giving accumulated data

Post by pythoncoder » Thu Jan 07, 2021 2:46 pm

From the docs, if .read() is called without an arg
read as much data as possible. It may return sooner if a timeout is reached.
This seems to be exactly what is happening: it's returning with a timeout until it's ready to empty its buffer.

Where possible it's best to send a newline (b'\n') and use .readline(). Alternatively if messages are of fixed length, use .read(length) - however this approach can make synchronisation difficult if the transmitting device starts before the receiver has been set up.

It's also worth looking at uasyncio's StreamReader and StreamWriter objects.
Peter Hinch

Duality
Posts: 16
Joined: Tue Jun 09, 2020 2:43 pm

Re: uart.read() is giving accumulated data

Post by Duality » Fri Jan 08, 2021 6:16 am

Hi Peter,

Thank you for your suggestions,

I put a timeout in uart definition like:

Code: Select all

u2 = UART(2, 9600, tx=23,rx=22, timeout=20)
This has resolved the issue :) . Below 20 ms (critical timeout), it is behaving like previously (accumulated data of 120 bytes). Also if I increase the baud rate to 19200, the code works as expected till I reduce the timeout to half, i.e. 10 ms. Why is this critical timeout exist? Any timeout value above 0, should give at least few characters as output of uart.read(). Right?

I am not familiar with uasyncio module :| , but will try it out.

Regards

User avatar
pythoncoder
Posts: 4929
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: uart.read() is giving accumulated data

Post by pythoncoder » Fri Jan 08, 2021 8:55 am

In my opinion the no-argument read() is virtually useless precisely because you don't know how much data you're going to get: the quantity depends on the exact timing of the transmitting device relative to your receiving task. The way to think about communications is in terms of messages: you want to receive a complete message. In general messages should either be fixed length or delimited with a specific character, typically b'\n'.

These observations are common to almost all communications devices, whether UARTs, sockets or radios. It is necessary to figure out how to synchronise the receiver to a remote transmitter whose timing is usually not under your control.

The only way I could envisage using the no-argument read() is if multiple reads were concatenated into a large buffer which was then subsequently parsed to produce individual messages. This would only make sense if an individual message might be longer than the maximum amount of data returned by read. Despite having programmed UARTs for 45 years I don't recall ever actually doing this.

This link illustrates the uasyncio approach.
Peter Hinch

Post Reply