MicroPython Forum (Archive)

You'd think the that question can be answered with confidence with a five minute google search, at most.

Well, my google skills seem to be lacking.

Is MicroPython hard realtime / deterministic and with what assumptions?

How do I determine if a particular time constraints can be met?

How do I determine if a particular time constraints are met with my code?

If Python alone cannot do it, is it possible run a separate C task (timer interrupt or something) to do the realtime stuff without the Python interfering with that (for example, does the GC disable interrupt, does anything in the MicroPython disable interrupts)?

For example could I generate step pulses for five stepper motors in the 10 kHz pulse rate region with less than 10% guaranteed jitter?

At that pulse rate a single pulse delayed too much will cause the stepper to loose position or stall.

That is just an example of what kind of questions I want find answers to.

cheers Kusti

nyholku wrote: ↑

Thu Jul 08, 2021 6:22 am

Is MicroPython hard realtime / deterministic and with what assumptions?

In general, no, because of the garbage collector, but in some specific-but-achievable cases, yes. It also depends on the hardware too -- for example on more sophisticated architectures memory hierarchy becomes more relevant (i.e. memory bus stalls due to reading code from XIP flash, etc).

It is possible, with varying degrees of difficulty, to write code that does not allocate, and therefore does not trigger the GC.

Some key things to be aware of:
- MicroPython has both "bare-metal" (e.g. STM32) and "OS" (e.g. ESP32, Linux) ports. On the bare metal ports, when a hardware interrupt fires, the Python VM is executed in the context of the hardware interrupt (i.e. no delay). Hardware interrupts are not blocked by the garbage collector, and hard interrupt handlers may not allocate.
- On OS ports, interrupts happen in "scheduler" context which may have a delay as the VM only runs the scheduler every N instructions, and the VM is blocked while the GC is running.
- Timer interrupts may be pre-empted by other higher-priority interrupts, so it's important to ensure that you don't have other peripherals active (e.g. USB) or you need to account for that.
- Non-interrupt code on bare-metal ports will have non-deterministic timing if the GC is active (allocations take a variable amount of time because they involve scanning the heap), and obviously allocations can trigger collection. You can use gc.lock() (and gc.unlock()) to force your code to not allocate (will raise MemoryError instead). Other than that, the VM is deterministic (i.e. it doesn't have, for example, optimisations that kick in automatically and change behavior).

nyholku wrote: ↑

Thu Jul 08, 2021 6:22 am

If Python alone cannot do it, is it possible run a separate C task (timer interrupt or something) to do the realtime stuff without the Python interfering with that (for example, does the GC disable interrupt, does anything in the MicroPython disable interrupts)?

Yes. In the STM32 port for example there are background C tasks that are run from SysTick. (Some directly, and some via PENDSV).

nyholku wrote: ↑

Thu Jul 08, 2021 6:22 am

How do I determine if a particular time constraints can be met
How do I determine if a particular time constraints are met with my code?

These are difficult questions. Not my area of expertise, but as a starting point, on bare-metal ports MicroPython does very little in the background, especially if you don't enable any peripherals. Obviously "measuring it and see" isn't sufficient to eliminate concern of unknown causes of jitter (unless you do a _lot_ of testing).

nyholku wrote: ↑

Thu Jul 08, 2021 6:22 am

For example could I generate step pulses for five stepper motors in the 10 kHz pulse rate region with less than 10% guaranteed jitter?

FWIW, this talk might be useful just to understand limitations around how fast you can toggle a pin -- https://www.youtube.com/watch?v=hHec4qL00x0

Two comments.

Firstly, realtime performance is highly dependent on hardware. You are more likely to achieve your goals with a Pyboard with its high performance hardware timers than with (say) an ESP32. So you need to study the various hardware platforms. You may also want to consider offloading some tasks onto dedicated hardware.

Secondly we have a lot of documentation answering your questions. For example https://docs.micropython.org/en/latest/ ... rules.html, https://docs.micropython.org/en/latest/ ... ython.html and https://docs.micropython.org/en/latest/ ... ained.html.

There are no quick answers to questions about realtime programming (in any language). You have to analyse the problem, then do your research.

Good luck

@jimmo

Thanks! That was a great answer.

Pretty much what I had gathered from reading but was surprised that this is not spelled out.

@pythonencoder

I've see those and I don't think they really answer the question(s).

My takeaway from this is that because of the (way it is implement) GC prevents RT in µPy.

Not a surprise but slightly disappointing.

Let me qualify above. I would think that for a given configuration GC probably has an absolute upper bound that in theory could be computed.That upper bound would probably be in the range 100 of msec or some seconds.

And that would be hard realtime, which as we know it, is not about performance per se, but meeting absolute deadlines.

But at the same time, many systems that require hard realtime have requirements in the sub 10 msec range.

Avoiding GC is a problem because GC is one of the key features of Python and making sure any of the libraries used are free from memory allocation (if any such libraries exist) is a chore and takes away the other key attraction of Python: libraries.


I've done embedded programming starting from 1979 with 1802/8085/6800/6502 professors and ever since C became a thing for µC development in the 80's I've been on the look out for something better. Not that I have much to complain about C.

For a while C++ looked promising but it is such a monster of a language that I've not bought into it even if I use it in some projects.

So I guess I will keep looking out for the replacement for C for the time being.

This is not to put µPy down, I'm sure it fills a niche for a group of people, but it seems I'm not in that group.

The length of time taken by GC depends on the platform and other factors. On an ESP32 with SPIRAM it can take >100ms. Which is grim.

More typical figures are 3-10ms or so. On a Pyboard and probably other baremetal platforms it can be kept down to ~1ms by running a task which periodically performs an explicit GC. This works better than waiting for the runtime to decide it is necessary: you substitute relatively frequent brief GC periods for less frequent but longer ones.

Response times to IRQ's on baremetal platforms are on the order of 10-15μs.

Hard RT applications can be done with MPY under the following restrictions:
- Use a bare-metal implementation.
- Carefully monitor dynamic memory handling. GC usually isn't a big deal. GC uses less than 10ms on a PYBD and there is an upper limit.
Of course it might be difficult to design an RT application that uses dynamic memory handling, but this isn't limited to MPY.

Typical professional RT applications we did using MPY achieved IRQ response times in the sub-us range.
With appropriate layout of the dyn. memory system hard RT behavior down to 5 us (< 20 ns jitter) at 100 Mhz SYSCLK (PYBD) can be achieved.