_thread module status/documentation?

[T-Wilko]

Hi all,

I've been super happy using the Loboris esp32 uPy port in that so many of the extra built-in libraries work smoothly and without a hitch. Since development by Boris has stopped on the port, I'm hoping to retain alot of the functionalities of it, while reverting to the main uPy master branch.

What exactly is the current status of the _thread module? The official wiki documentation is scarce and really only points to CPython _thread module without any explanation of current implemented methods etc.

Thanks in advance
Thomas

[tve]

The threads are supported but there's very little around them. I believe that almost everyone is focused on asyncio, which is a much better way to do most of the things threads are used for, IMHO. I've been toying with using threads to solve very focused problems, such as a TLS connect blocking for a full second, but haven't gotten around to really trying that out. Note that currently all python code must run on the same processor on the esp32, so threads don't allow you to use 2 processors.

[T-Wilko]

The threads are supported but there's very little around them. I believe that almost everyone is focused on asyncio, which is a much better way to do most of the things threads are used for, IMHO.

My experience with uasyncio is fairly limited, could you shed a light on why asyncio is better than threading?
Also, does uasyncio allow you to access REPL while a background task is occuring simultaneously, like how threading allows it? There's no clear mention of this that I can find in the docs or CPython docs, but perhaps it's assumed knowledge

[tve]

The threads are supported but there's very little around them. I believe that almost everyone is focused on asyncio, which is a much better way to do most of the things threads are used for, IMHO.

My experience with uasyncio is fairly limited, could you shed a light on why asyncio is better than threading?
Also, does uasyncio allow you to access REPL while a background task is occuring simultaneously, like how threading allows it? There's no clear mention of this that I can find in the docs or CPython docs, but perhaps it's assumed knowledge

The reason asyncio is better is that it is non-preemptive, which means that you don't need to be hellishly careful about atomicity all over the place. It results in code that is simpler to reason about and tends to have fewer race conditions.
With asyncio you cannot run the repl at the same time as the asyncio loop. In MQBoard I end up just calling exec or eval directly to execute code interactively, but it's not the same as typing into the console.

[pythoncoder]

...My experience with uasyncio is fairly limited, could you shed a light on why asyncio is better than threading? ...

For a longer answer see threads are bad and my uasyncio tutorial.

tl;dr Debugging realtime code can be hard. Debugging multi-threaded code is hard**10.

[jcw]

Re: REPL and asyncio. You can do stuff while also listening to the console, e.g.

Code: Select all

serial = pyb.repl_uart() or pyb.USB_VCP()

async def reader():
    stdin = asyncio.StreamReader(serial)
    while True:
        msg = await stdin.readline()
        ...

asyncio.run(reader())

So by adding more tasks, you could use the above to read lines and eval them, creating a basic REPL alongside other tasks.

PS. If the current REPL were refactored to work on asyncio (and offer all its interactivity), this would allow the same use as with threads, and remove the need for a special dupterm mechanism. This is nothing new, it's how Tcl/Tk's "console" has worked for decades (and lets you have a socket-based command line into a faceless server app, for example) - to bring up an old horse ...

[T-Wilko]

Firstly, thanks @tve and @pythoncoder for the explanations and links, they were an interesting read. The very first sentence in that asyncio tutorial section resounded very strongly with me

The initial reaction of beginners to the idea of cooperative multi-tasking is often one of disappointment. Surely pre-emptive is better?

Admittedly, the past few projects involving threading have yielded some peculiar bugs that appeared in such odd, infrequent, unpredictable ways; I wholly understand the exponentiating debugging difficulty.

I still feel a little disappointed that lobo's polished functionality in the _thread module has seemed to go to waste. My first moment of writing a simple function to display live GPS coords on an SSD1306 running as a separate thread in the background to the readily accessible repl was fantastic, and a learning experience in itself for a newcomer coder like myself. Would it not be reasonable to still implement threading, just with a lot of warning labels attached? I'm no C dev level programmer so I can't assess the complexity of lobo's module, but still.

So by adding more tasks, you could use the above to read lines and eval them, creating a basic REPL alongside other tasks.

Thank you for the idea! I'll definitely be using this for simple function calling for testing, but it still seems like the long way around, rebuilding what's already there.

As for my original question, there do appear to be some methods accessible for threading in the default v1.12 but without a userguide, I suspect many people (like me) won't ever be able to use them correctly. Perhaps on purpose in favour of the safer asyncio, I suppose.