UART junk characters on startup on CH340 ESP8285

[cefn]

Hi all,

I don't believe this is a Micropython bug, but thought others might have seen this issue before on ESP8266 boards and have ideas what to do.

I have NodeMCU-M boards with ESP8285 running Micropython 1.9.3 (having a CH340 UART chip) and if I connect like `screen /dev/ttyUSB0 115200` or like `serial.tools.miniterm --raw --eol CR --encoding ascii /dev/ttyUSB0 115200` I get access to a functional and stable terminal channel to the REPL.

However, it seems that during startup, a handful of junk characters have already been passed from a buffer or perhaps spit out from a floating pin before the full activation of the UART link has been completed. This is normally 3 or 4 characters for example from recent sessions...

Code: Select all

kH(J
l.bH
l=L
kHH
3Z@N

On some occasions the same identical characters are sent after a power-cycle (a full remove and replug of the USB cable which both powers and provides UART). For example I saw l=L multiple times in succession. At other times, no characters are sent (some kind of race condition? It always seems to be the very first time the serial is used after powerup, so it is associated with the moment the linux driver for that device is first used and the device is first powered. Exiting and relaunching Miniterm/Screen or resetting via CTRL+D or CTRL+B never creates the junk characters after the first occasion.

My worry is having to explain to learners what messages like this mean after they press enter on first connecting...

Code: Select all

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'kHH' is not defined
>>> 

Does anyone have experience of this happening, and can trace it to a particular part of the system?

Is there some kind of workaround if there's a startup timing issue, such as shedding a serial buffer after first powerup? How would I do this from python or is it too late as it's buffered in the REPL already? Just hoping for ideas to avoid confusing people when they power up their boards.

For reference I have already removed the horrifying modemmanager from my Ubuntu 17.10 desktop, which is commonly responsible for this kind of junk startup character.

[jomas]

That is a boot message from the esp8266 rom. See:

https://github.com/espressif/esptool/wi ... ot-ROM-Log

[cefn]

Thanks for the suggestion, but I can't see how that would explain it, for the following reasons;

I think this might work if a signal was coming in on RX, but the boot message is output bytes going out from the ESP8266 over TX, so it wouldn't make sense for the Micropython REPL to see the boot message on the UART output pin as typed characters on the UART RX input pin, unless they were wired to each other somehow.

Also I have an ESP8266 NodeMCUv2 Amica board running exactly the same build and configuration, (but with a CP2102 UART) and it doesn't encounter these stray characters in the REPL on powerup.

[dhylands]

You can probably work around the problem.

main.py executes before you get dropped into the REPL, so you could write some code which reads any data on stdin and tosses it. Then when the REPL runs, the junk will already have been consumed.

I've not run MicroPython on the ESP8266, but you should be able to open the UART and then use .any to determine if characters are available.
http://docs.micropython.org/en/v1.9.3/e ... e.UART.any
Read a character until any returns 0

Then when you exit main.py there should be no characters left for the REPL.

[cefn]

That's really helpful @dhylands. However, I'm not succeeding with this approach yet.

In my boot.py I am importing two boot routines, and keeping the clear uart boot routine until last, as I imagine the bootServeRepl.py will waste enough time for the startup noise generating the characters to have completed.

However, while this routine is running without error, this still doesn't seem to clear the characters. Using boot.py is preferred as I will be guiding learners to replace their main.py

Here is the importing boot.py and the clear uart routine it imports.

boot.py

Code: Select all

# launches webrepl and replserver
import vgkits.util.bootServeRepl
# clears uart
import vgkits.util.bootClearUart

vgkits/util/bootClearUart.py

Code: Select all

from machine import UART
clearUart = UART(0, 115200)
while clearUart.any() > 0:
    clearUart.read()
del clearUart

[jomas]

However, while this routine is running without error, this still doesn't seem to clear the characters. Using boot.py is preferred as I will be guiding learners to replace their main.py

That is because the bootmessages have already been send at 74880 baud. During the sending of the boot message, MicroPyhon, main.py, boot.py, REPL, they are all not loaded yet. Even a completely erased esp8266 will generate this bootmessage.

If you realy want to suppress this you have to do this by hardware. (disable input to your terminal program on startup).

[cefn]

But Jomas...

a) the message is coming into the ESP8266 not going out
b) even if it such a boot message was coming in,
(which I don't believe it is) then clearing the buffer between startup and REPL should prevent it from arriving at the REPL.

[dhylands]

The fact that you're still seeing junk after clearing it suggests one of the following:
1 - it isn't actually getting cleared (wrong UART or other bug in the code)
2 - the data is showing up AFTER the clearing

One thing that could introduce garbage would be adjusting the MUX settings (maybe not applicable to the ESP8285). If the MUX is setup such that your UART was available on multiple GPIO pins, then the garbage could be getting introduced that way.

Putting a logic analyzer on the Rx pin (on the ESP) would let you capture the data and perhaps figure out more (since it might be at a different baud rate or something). And more importantly it would let you determine when the noise shows up.

Which OS are you using as a host? I know under linux, there is a piece of software called modemmanager which may try to send data to some devices. I usually uninstall modemmanager or configure it to ignore the serial port via UDEV rules.

[cefn]

Thanks for the heads up about modemmanager, @dhylands, although I had already eliminated this as a possible reason - it was uninstalled beforehand.

I changed the bootClearUart.py routine to wait on the junk characters as shown below.

Code: Select all

from machine import UART
from time import sleep

waitBefore = True
delayPeriod = 0.1

clearUart = UART(0, 115200)

if waitBefore:
    while not clearUart.any(): # stop for at least one character
        sleep(delayPeriod)

burst = b""
while clearUart.any():
    burst += clearUart.read()
    if not clearUart.any(): # wait for a gap in the burst of at least 0.1 secs
        sleep(delayPeriod)
del clearUart

If waitBefore is False, (you dont' hang on for characters in order to clear them), junk UART characters are visible later when you access the REPL.

However, if waitBefore is True, the routine hangs waiting for characters in bootClearUart.py and it's only after you connect to the REPL that the first character is sent by screen/miniterm and the other junk characters are sent preceding it. Since boot.py was blocked waiting to clear them, the REPL then doesn't see them. However to achieve this boot.py is hung waiting for the first character and main.py is not run until you type something into the REPL.

The burst of characters sent when you type your first character into the REPL can be introspected like this...

Code: Select all

>>> from vgkits.util.bootClearUart import burst
>>> burst
b'\xc0\xc8\x85\xe5\x8a^H\xcc'

Latest suspicion is that they are (non-zeroed?) CH340 buffer contents in the linux driver, or possibly somewhere else in the chain but they are not flushed from the client side until you type your first character in screen/miniterm. Alternatively there is something screwy about stability of the RX which unique to startup on this board layout (and its power arrangements) and leading to transient noise. The board looks like... . Otherwise there should be no difference between these boards and any other 1Meg EP8266.

At this point if I want main.py routines to be run on this board...
* without blocking waiting for the first character
AND
* without causing junk characters to be sent
...it seems like I have two options...

1) Create my own image with a modified main.c with an extra pyexec_frozen_module('bootClearUart.py') after

Code: Select all

mp_reset();

but before

Code: Select all

pyexec_event_repl_init();

This would invoke the (hanging) logic to clear the UART after main.py, and would do no harm except throwing away the first character when you finally decide to run the REPL, since boot.py and main.py are not blocked waiting. Given the latest upip.py cannot be loaded and run on ESP8266 as a source module, and given I plan to distribute a set of modules on the board a bespoke image may be a good idea to defeat memory issues for learners anyway.

2) Accept there is always junk and advise users to always start their session with a CTRL+C to get a prompt (and ignore anything shown on screen). This is now a reasonable fallback as it is consistent, and probably advisable whatever state the REPL is in.

[dhylands]

Some terminal programs send AT commands at the beginning/end of the connection (minicom does this by default).

I used to use picocom (which doesn't do this), but eventually wrote my own little program which auto disconnects/connects when the usb serial device comes and goes (linux only):
https://github.com/dhylands/usb-ser-mon

I normally use

Code: Select all

usb-ser-mon.py -n M

to connect to whichever serial port MicroPython is connected to.