how to write coroutine in C

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
jickster
Posts: 532
Joined: Thu Sep 07, 2017 8:57 pm

how to write coroutine in C

Post by jickster » Mon Jun 25, 2018 2:10 pm

Environment: cooperative RTOS and micropython VM is one of the tasks.
To make the VM not block the other RTOS tasks, I insert RTOS_sleep() in vm.c:DISPATCH() so that after every bytecode is executed, the VM relinquishes control to the next RTOS task.

I created a uPy interface to asynchronously obtain data from a CAN bus, similar to producer-consumer design pattern. The abstraction I chose is a queue, similar to https://github.com/peterhinch/micropyth ... d#35-queue

Usage in uPy:

Code: Select all

can_q = CANbus.queue()
message = can_q.get()
The implementation in C is such that .get() does NOT block the RTOS: it polls a C-queue and if message is not received, it calls RTOS_sleep() to give another task the chance to fill the queue. Things are synchronized because the c-queue is only updated by another RTOS task and RTOS tasks only switch when RTOS_sleep() is called.

The implementation is basically (C-code):

Code: Select all

while(c_queue_empty() == true) RTOS_sleep(); // gives chance for c-queue to be filled

return c_queue_get_message();
While can_q.get() does not block the RTOS, it does block the uPy script but I'd like to use my can_q.get() with async def.
I cannot directly use micropython-uasyncio.queues because my producer is asynchronous to the uPy script since it occurs at the C-level.

How do I implement can_q.get() so that I can await on the call to can_q.get() so can_q.get() doesn't block uPy?

User avatar
pythoncoder
Posts: 3068
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: how to write coroutine in C

Post by pythoncoder » Tue Jun 26, 2018 7:39 am

You might like to look at the updated version of my uasyncio tutorial section 5.4, and implement a stream reader device. Your device driver would have an ioctl method. This is repeatedly called by uasyncio event loop. Your C interface would need to return a ready status for ioctl: this would yield to the RTOS, check whether data has been received and return a boolean. Your get method would then only be called if data were available and would be designed to return that data quickly.

So nothing blocks.
Peter Hinch

jickster
Posts: 532
Joined: Thu Sep 07, 2017 8:57 pm

Re: how to write coroutine in C

Post by jickster » Tue Jun 26, 2018 2:03 pm

pythoncoder wrote:
Tue Jun 26, 2018 7:39 am
You might like to look at the updated version of my uasyncio tutorial section 5.4, and implement a stream reader device. Your device driver would have an ioctl method. This is repeatedly called by uasyncio event loop. Your C interface would need to return a ready status for ioctl: this would yield to the RTOS, check whether data has been received and return a boolean. Your get method would then only be called if data were available and would be designed to return that data quickly.

So nothing blocks.
Excellent. Thanks :)
I was reading that tutorial but didn't get far enough I guess.

For completeness, is ioctl how CPython implements this?

User avatar
pythoncoder
Posts: 3068
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: how to write coroutine in C

Post by pythoncoder » Wed Jun 27, 2018 10:13 am

See ports/stm32/uart.c.
Peter Hinch

Damien
Site Admin
Posts: 581
Joined: Mon Dec 09, 2013 5:02 pm

Re: how to write coroutine in C

Post by Damien » Thu Jun 28, 2018 2:26 am

To make the VM not block the other RTOS tasks, I insert RTOS_sleep() in vm.c:DISPATCH() so that after every bytecode is executed, the VM relinquishes control to the next RTOS task.
It would probably be more efficient (more bytecodes executed per second, less context switching) to yield from the VM using the MICROPY_VM_HOOK_xxx macros. These will be called either on return, or whenever a jump opcode is executed. This guarantees that they will eventually be called no matter what the script does (even in an infinite loop). The esp8266 port uses these macros to cooperatively yield to the underlying OS; see ports/esp8266/mpconfigport.h.
How do I implement can_q.get() so that I can await on the call to can_q.get() so can_q.get() doesn't block uPy?
Making a C function yield (in the Python sense) is currently not supported by the runtime (only bytecode can yield), although it would be possible to add such a feature. In the meantime, pythoncoder's suggestion to implement ioctl at the C level is a good one.
For completeness, is ioctl how CPython implements this?
AFAIK, no, CPython doesn't have ioctl. It uses file descriptors only and relies on the underlying OS to poll files efficiently. MicroPython allows any object (implemented in C or Python) to be polled via ioctl.

jickster
Posts: 532
Joined: Thu Sep 07, 2017 8:57 pm

Re: how to write coroutine in C

Post by jickster » Fri Jun 29, 2018 3:21 pm

Damien wrote:
Thu Jun 28, 2018 2:26 am
To make the VM not block the other RTOS tasks, I insert RTOS_sleep() in vm.c:DISPATCH() so that after every bytecode is executed, the VM relinquishes control to the next RTOS task.
It would probably be more efficient (more bytecodes executed per second, less context switching) to yield from the VM using the MICROPY_VM_HOOK_xxx macros. These will be called either on return, or whenever a jump opcode is executed. This guarantees that they will eventually be called no matter what the script does (even in an infinite loop). The esp8266 port uses these macros to cooperatively yield to the underlying OS; see ports/esp8266/mpconfigport.h.
I need the timeslices to be as granular as possible. uPy scripting is secondary functionality on my embedded product so I need to be as conservative as possible with respect to the timeslices so the primary functionality is not degraded.

Yes I know list.sort() and other methods could take "very long" and that's another issue for me to solve in my cooperative OS.

Post Reply