ESP32 multiprocessing

[Damianvv]

Hello,

I want to know if it possible to implement multiprocessing in MicroPython on the ESP32?
I did some research and found the loboris implementation
(https://github.com/loboris/MicroPython_ESP32_psRAM_LoBo),
however this is not true multiprocessing and more like multithreading,
because it still uses only 1 task and 1 GlobalInterpreterLock and it makes both cores work on 1 task.

Maybe ideas like spawning a 2nd MicroPython instance on the 2nd core of the ESP32 is possible, however i can't find information online on how to do this.

Anyone have an idea if its possible to do multiprocessing on the ESP32 using MicroPython?

[jimmo]

Yes, all MicroPython code is pinned to one core on ESP32, due to complexities with managing thread state etc.

I'm not aware that anyone is actively working on this particular issue. However I think the most recent context on this is at https://github.com/micropython/micropython/issues/4895

Even with the ability to run Python code on both cores, you still have to consider the impact of the GIL etc, so like you say, perhaps running two separate VM instances might be better.

[seandepagnier]

Would it be possible to implement standard multiprocessing by using threads in c (already supported for both cores) and each thread runs a complete VM? If these could communicate via pipes. Would the sleep in python waiting on sockets for example actually relinquish the core to a different thread and VM?

[jimmo]

Yes, the ESP32 port already works this way (i.e. it runs MicroPython in a thread). On startup, MicroPython creates a task (which is what FreeRTOS calls a thread), and pins it to one of the cores. Then the MicroPython VM and all MicroPython code runs in that task.

But FreeRTOS is just like any other OS -- so I'm fairly sure that two tasks could be using sockets at the same time.

However (despite what I said earlier), there would be some work to running two concurrent instances of the MicroPython VM. Currently it access all VM state via a global variable (via the MP_STATE_VM, MP_STATE_MEM macros), and thread state via TLS (via MP_STATE_THREAD). You would need to find a way to make MP_STATE_VM/MEM know which VM they're in (which isn't impossible, just needs some thought, probably TLS is the easiest option).

I have some recollection that the choice of core that MicroPython that MicroPython is pinned to isn't an arbritrary choice...so there might be some interaction with IDF / FreeRTOS stuff that makes using that core preferrable (that github thread I linked to might have more context).

[MATTYGILO]

Any progress on this yet?