Async Uart Tx (same as #1642)

[neilh20]

This is an extension to #1642
1533 references UART TX but possibly at "python processing" level.

I've been modifying startup code and figuring how the USB OTG features work.
The MP USB_FS device startup is some amazing code with the way it handles the serial interface into the REPL - and its a key to making the pyboard so easy to plug'n'play over a serial channel.
However, the USB_FS is the main system output, and for the the USB Host I'm using the printf and redirecting it to a UART for better stability during startup monitoring.

This hits another issue that the UART TX results in a blocking write to the hardware.
uart.c:

Code: Select all

STATIC void uart_tx_char(pyb_uart_obj_t *uart_obj, int c) {
    uint8_t ch = c;
    HAL_UART_Transmit(&uart_obj->uart, &ch, 1, uart_obj->timeout);
}

The STM32 USB libs also implement a debug printf in usbd_conf.h and usbh_conf.h - which kills the startup if the output of the printf is tied into usbd as well, as it becomes circular - but that has another solution.

To work in a realtime the debug needs to not change the runtime aspect by much, and a blocking UART write does that.
There are lots of other cases where a blocking uart write is detrimental to system design, one example being.
https://github.com/micropython/micropython/issues/1642
A basic interface which requires a similiar solution has been proposed with
https://github.com/micropython/micropyt ... uart-class uart.write(buf, async=True)

I've recently implemented a solution to this in the context of stm32F429 project, so I'm posting a snippet of some **raw** code that I've wrote to solve the same problem. I can't post the code with #1642 -so attaching it here.

There is also a tutorial on fast serial processing with DMA.
http://www.emblocks.org/wiki/tutorials/ ... sco/serial
I think the DMA is more challenging, and very useful for the high thru put described in the tutorial, and more complex than a simple buffer needed at this stage.

I would like to propose that there be a layer added to the UART TX output processing. This requires a ram buffer to decouple the faster printing from the slower serialization of the UART. The size of the buffer could be an option, with a 0 size buffer having the same blocking behaviour as the current code. I found the debug buffers need to be between 500-3000bytes depending on what type of output is happening.
Just wondering what anybody else might be doing?, is this duplicating some other effort?

[dhylands]

I've been talking with ryannathans (IRC nick) and he said he'd be interested in taking a stab at implementing a tx interrupt driven circular buffer. This isn't quite async Tx, but if the buffer is big enough then it serves the same purpose.

On technique I've used for debug purposes is to implement a high-speed memory logger. It takes a printf-style string and upto N arguments. It stuffs the format string and the arguments into a big circular buffer, along with a high resolution timestamp. It doesn't do any string formatting, just saves the pointers. When the buffer fills up, it just wraps and overwrites old messages. I typically make the code thread safe so that you use the logging from interrupt handlers. This is light weight enough to use for monitoring when interrupts fire and other types of high frequency logging, and has minimal impact on the real-time characteristics of the system.

Then at some later point in time you can pull the log, format the messages and print it. System crash time is often a good time to pull the messages and print them

There are many variations on this theme.

For system crashes, you WANT to use some type of blocking print mechanism because you can't rely on interrupts to fire in order to get the messages printed, so you typically need some type of simple blocking print for this.

[neilh20]

debug buffer: nice idea for the debug buffer that can cope with crashes. I'd be interested in discussing it as well.
I'd would modify it so that it also outputs as it goes - so it can be used as a general purpose debug buffer
Also, for general system stability testing, I've implemented a per module/file 8/16bits variable that can gate the output of the debug.
This can then be modified dynamically when investigating system anomolies to find what path the actual execution is going in.

Code: Select all

uint8 runtimeMask=ERROR;
fn(..) { 
  if (FIXED_CONST & runtimeMask ) {
     dbg_log("ID_String",mon_var_if_any);
   }
}

then through a debug port, with the system running, runtimeMask can be modified as necessary to monitor for module level behaviour.
Its useful for system level debugging, and can also be employed remotely