Run more than one micropython instance

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
jockm
Posts: 13
Joined: Tue Aug 21, 2018 9:46 pm

Re: Run more than one micropython instance

Post by jockm » Tue Aug 21, 2018 9:52 pm

I can tell you why I would like to have multiple micropython instances. I am developing a system that can have multiple user supplied "applets" and I would like to have process isolation. I don't want the code in one applet to be able to see (or conflict with) the code in another applet.

As far as I can tell there is no way to do this with uPy. There is with eLua (though it is a PITA to build it as a library), or with JerryScript (though a pain for other reasons).

User avatar
pythoncoder
Posts: 3244
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Run more than one micropython instance

Post by pythoncoder » Fri Aug 24, 2018 7:59 am

True process isolation needs to cope with the case where someone runs

Code: Select all

while True:
    pass
thereby locking everyone else out. This is only feasible with an underlying OS arbitrating between the MicroPython instances with pre-emptive multitasking. You could doubtless readily do this on a Raspberry Pi running Linux, but I don't think that's the answer you were looking for.
Peter Hinch

jickster
Posts: 611
Joined: Thu Sep 07, 2017 8:57 pm

Re: Run more than one micropython instance

Post by jickster » Sat Aug 25, 2018 1:17 pm

pythoncoder wrote:True process isolation needs to cope with the case where someone runs

Code: Select all

while True:
    pass
thereby locking everyone else out. This is only feasible with an underlying OS arbitrating between the MicroPython instances with pre-emptive multitasking. You could doubtless readily do this on a Raspberry Pi running Linux, but I don't think that's the answer you were looking for.
You could do cooperative multitasking and OS.yield() in between the op-codes.

User avatar
pythoncoder
Posts: 3244
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Run more than one micropython instance

Post by pythoncoder » Sat Aug 25, 2018 5:01 pm

Enforcing that would require a modification to the compiler. But there are numerous ways in which arbitrary apps could conflict. Cooperative multi-tasking is just that: it assumes a set of tasks designed to cooperate in their use of resources.

Regardless of the cooperative/pre-emptive solution, MicroPython is designed primarily for bare metal environments. If two tasks start independently accessing the same peripheral, there will not be a happy outcome. Likewise if two tasks issue uctypes.bytearray_at(addr, size) with the same addr each can scribble over the other's data. Interrupts, the RTC - the sources of potential conflict are legion.

It seems to me that the only hope of task isolation is to run the Unix version and hope that the OS will save you from all but the worst excesses. My knowledge of security issues is limited but I doubt even Linux could protect a "good" MicroPython script from one deliberately crafted to subvert it.
Peter Hinch

oclyke
Posts: 11
Joined: Tue Mar 19, 2019 4:55 am

Re: Run more than one micropython instance

Post by oclyke » Tue Mar 19, 2019 5:33 am

Hello everyone - interesting thread. I wish I had seen it earlier this year.

Summary: if the op (tarun2121) is still pursuing this project it is definitely within reach and I'd like to work together on it. If you can use an underlying scheduler such as FreeRTOS then the rest of the work is minimal thanks to the MP_STATE_XXX(x) macros.

Might as well throw in my two cents, and hopefully learn a little more. Earlier this year I started working toward the same goal (https://github.com/micropython/micropython/issues/4480) and now I have the basics in place. Fortunately for me the ESP32 port is already built within a FreeRTOS task so this problem was reduced to changing references to the mp_state_ctx structure to pointers and making sure that the correct pointer was always selected.

I'm still working on this and I would really love to get it pulled into MicroPython, but there are quite a few questions to answer first. The ones that I can think of off the top of my head are:

- How does this play with the rest of MicroPython?
- What is the common interface that any port can share and what needs to be reduced to a port-specific implementation?
- How can we update/improve the garbage collector and general heap management to be efficient and dynamically extendable?
- How much effort might it take to ensure that hardware peripherals all have some sort of lock to prevent simultaneous access?

At least for this first point it's tempting to say that we could relatively easily install the foundation (pointer access to the state) into MicroPython without impacting others. Also I think it's fair to say that this would not step on the toes of uasyncio because it serves a different audience and simply putting in the foundation hardly bogs down the codebase at all (IMO that is someone who might want to add additional code to execute without worrying about knowing what code is currently on the board.)

The other points will be significantly more work and I surely would not come up with the best solution on my own. Anyone interested and willing to collaborate? I'll be working on this here so please join me! https://github.com/oclyke/micropython

Post Reply