MicroPython Forum (Archive)

I'm looking at MicroPython to embed inside a larger (desktop) application for Python expression support. It's currently using CPython, but the GIL is looking to be a bottleneck and I don't think I can wait for isolated subinterpreters. I plan to use an interpreter per thread; there will be a pool of interpreter threads being fed from a single producer thread, but each thread is isolated. The Python scripts themselves will not launch any threads. CPython's GIL destroys any parallelism I might get from that approach, and I really don't want to do multiprocess if I can avoid it.

From the code, it looks like I could make this work with MicroPython by making mp_state_ctx thread-local instead of global and calling mp_init()/mp_deinit() on each of the worker threads. I'll also be extending MicroPython via C to interface with the parent application, but it looks like that's OK as long as I avoid any global state in my extension code.

What I would like to know is if there's a better, built in way to do what I'm trying to do, and if not, if anyone has tried the approach I've outlined to know if it works and found any gotchas that I should watch out for.

There's no built in way I'm aware of, but better doublecheck that mp_state_ctx is the only global which needs to be thread-local. I assume you know there are more init functions than just mp_init (stack/heap/... should also be per thread)?

An alternative which might be less work depending on the situation is building multiple copies of shared libraries and loading each dynamically. Which automatically gives you completely isolated MicroPython instances, but requires some build changes and bookkeeping.