Trying to use timers to avoid a blocking loop

[pythoncoder]

I suspect you have substantially more experience in this field than me. But it strikes me that some of your comments are more relevant to preemptive multitasking than my strictly cooperative model. With no underlying OS I'm not sure that the concept of a blocked thread is relevant. If presented with a piece of hardware which takes time to respond the options I envisage are either to write a thread which polls the device (and yields in each iteration) or to use an interrupt with a simple handler and wait on its execution. In each case other threads get a look-in. As far as I can see we're talking of embedded systems where, if you need a device driver, you write it - as with my LCD and switch classes.

I take your point regarding priority. At the moment the scheduler is designed on the KISS principle with its main loop consisting of 19 lines of code but I'll seriously consider implementing a better priority mechanism along the lines you suggested.

As for more advanced concepts like semaphores, mutexes and suchlike I have no plans to implement these as I feel they go beyond the kind of simple applications I envisage. Many of the nastier aspects of realtime programming are reduced in a cooperative enviroment. That said, Micropython's support for interrupt handlers does imply true concurrency and hence offer justification for such objects. I think, however, it might be prudent for me to leave those aspects to others with more experience.

To put the discussion into context, I'm a programmer and hardware developer who retired a decade ago and am now purely a hobbyist. But many of my projects have used simple cooperative "concurrency" to sensibly manage user I/O and hardware interfaces. My aim is to provide a small, simple solution for the micropython board. If I tried to be more ambitious here I'd probably end up with egg on my face

Regards, Pete

[dhylands]

To me a blocked process is just a process thats waiting for an even to happen.

So the purpose behind introducing something like a semaphore is really performance related. If a thread is "blocked" that means its on a wait queue and isn't consuming any cpu cycles at all.

As an example: I have a thread which wants to process data from the serial port. The simple approach would be to do something like this:

Code: Select all

def SerialThread:
    while True:
        if serialPort.dataavailabe():
            ch = serialPort.getchar()
            processChar(ch)
        yield None

Using some type of blocking primitive the code would look like:

Code: Select all

def SerialThread:
    while True:
        blockOnSerialDataAvailable()
        ch = serialPort.getchar()
        processChar(ch)

blockOnSerialDataAvailable would do something like this:

Code: Select all

blockOnSerialDataAvailable:
    decrement semaphoreCount
    if semaphoreCount < 0:
        move yourself from the ready-to-run queue onto the waiting queue
        yielld None

and the ISR routine for the serial port would signal the semaphore, which would move it from the wait queue back onto a ready-to-run queue.

With the polling approach you wind up spending lots of time polling for stuff and the time spent goes up as you add more threads. With the blocking approach you spend very little time polling, and in general your code to process new data will run more quickly (i.e. reduced latency).

So this is why blocking is still a very useful technique, even for cooperative multi-threading.

[pythoncoder]

Ah, thanks for that - I'd misunderstood you point about blocking threads. My scheduler does enable the user to write blocking device drivers. For a serial device the user would proceed as follows.

There would be a custom written interrupt service routine which would perform actions requiring immediate attention, such as putting characters in a (pre-allocated) buffer and otherwise dealing with the UART hardware. Its final action before terminating would be to call a generic interrupt handler imported from my module.

The generic handler vectors the interrupt to run a callback function on the appropriate Waitfor instance: this simply increments a counter which is a property of the instance. I'm hoping this should be pretty fast but I haven't yet quantified it.

The thread which is to block instantiates a Waitfor object passing it the IRQ number and a timeout value (which can be infinite). When it wants to block it yields the Waitfor instance. It will then consume no resources until the interrupt occurs or the timeout expires.

The scheduler, whenever it gains control, builds a list of the threads which are ready to run. It does this as follows. When a thread submits control to the scheduler it yields a Waitfor object. The scheduler queries each of these and receives a tuple containing three integers. In the case of a Waitfor which is waiting on an interrupt, and where at least one interrupt has occurred, the tuple will contain (No. of interrupts, 0,0). (At this point the Waitfor instance zeroes the counter and herein lies a potential concurrency issue which may be worth discussing).

The scheduler determines which thread to run by finding the tuple with the maximum value, which prioritises threads waiting on interrupts. The priority order is
1. Threads waiting on interrupts
2. Threads waiting on polling functions
3. Those waiting on times (most overdue first). (As discussed, this may change)

The tuple containing the interrupt count is passed back to the thread so it can determine the number of interrupts which have occurred since it last ran.

As far as I can see I've implemented the required functionality you've identified. I do need to use my oscilloscope to quantify the time overhead this mechanism imposes on the interrupt handler as I'm very conscious of performance.

One issue with Micropython is the fact that interrupt handlers can't accept arguments. This means (as far as I can see) that they can't be class members. The way I employ to vector interrupts is using a global vector dictionary (private to the module) indexed by IRQ number and containing function pointers to the relevant instance method. I'm sure you'll agree this isn't ideal. If you can think of a better way, or can influence the relevant devs to consider the issue, it would be good.

Here is an example of some code which tests this. The ISR pulses Y10 and the thread pulses Y9 when it responds and a count is maintained on my LCD display.

Code: Select all

# Test of hardware interrupt. Run oscillator thread and patch output (X7) to input (X8) to test
testpin = pyb.Pin(pyb.Pin.board.Y10, pyb.Pin.OUT_PP)
def mycallback(irqno):                                      # Custom callback pulses a pin when interrupt received
    testpin.high()                                          # Should see pulse of 6.8uS on Y10 after each 
    testpin.low()
    intcallback(irqno)                                      # Must run default callback 

def irqtest1(mylcd):
    outpin = pyb.Pin(pyb.Pin.board.Y9, pyb.Pin.OUT_PP)     # Push pull output pin
    mypin = pyb.Pin.board.X8
    extint = pyb.ExtInt(mypin, pyb.ExtInt.IRQ_FALLING, pyb.Pin.PULL_NONE, mycallback)
    wf = Waitfor(irq = extint, forever = True)  # On yield the thread will block pending an interrupt
    count = 0
    while True:
        yield wf
        outpin.high()                                       # Toggle Y9 to test speed of toggle
        outpin.low()
        mylcd[0] = "Interrupt count: {:6d}".format(count)
        count += 1
#        print("Interrupt received {:2d}".format(wf.pinstate))

def oscillator():
     outpin = pyb.Pin(pyb.Pin.board.X7, pyb.Pin.OUT_PP)     # Push pull output pin
     wf = Waitfor()
     tim = seconds(0.1)
     while True:
        outpin.low()
        yield wf.retrig(tim)
        outpin.high()
        yield wf.retrig(tim)

Regards, Pete

[dhylands]

IRQ Callbacks can most definitely be class members. The self pointer winds up being bound together with the function. Here's an example:

Code: Select all

import pyb
import micropython

class Heartbeat(object):

    def __init__(self):
        self.tick = 0
        self.led = pyb.LED(4) # 4 = Blue
        tim = pyb.Timer(4)
        tim.init(freq=10)
        tim.callback(self.heartbeat_cb)

    def heartbeat_cb(self, tim):
        if self.tick <= 3:
            self.led.toggle()
        self.tick = (self.tick + 1) % 10

micropython.alloc_emergency_exception_buf(100)
Heartbeat()

If you're playing with interrupts, you'll want the call to alloc_emergency_exception_buf. It allows traceback information to be collected for exceptions which occur in the IRQ handler.

[pythoncoder]

Ooops! I must have been suffering from finger trouble when I concluded they didn't work. Thanks for the pointer.

Regards, Pete

[pythoncoder]

Thanks for that: now working fine without a global vector table. Much better!

I'd allowed myself to be misled by this in the docs:

callback is the function to call when the interrupt triggers. The callback function must accept exactly 1 argument, which is the line that triggered the interrupt.

https://micropython.org/doc/module/pyb/ExtInt

I'd wrongly taken this to imply that class methods were precluded since a class method appears to have a first argument self.

Regards, Pete