Page 1 of 2

Multithreading-like ideas?

Posted: Wed Mar 15, 2017 12:42 am
by jcea
I need to read data from the ADC as fast as possible (currently about 2.5K samples per second), do a small calculation with the data read and post it somewhere via HTTP or MQTT. The posting is not time critical and I can accumulate several readings per publication.

As far as I know, Micropython doesn't support threads but I would like to avoid stopping the ADC readings while I am posting the results via WIFI. My samplerate is not critical and jitter is not a problem as far as I can keep a sample rate > 500 samples per second.

My current code records data for 60 seconds and then stop sampling while posting the data online (it takes 3-5 seconds). I would like to avoid losing data for that long. Increasing the posting interval is possible, RAM usage need to be evaluated, but inconvenient and only improve the symptoms, not solving the real issue.

Do you have any suggestion? I could do Xtensa ASM if the code is short enough. Can I call the ADC reading from a interrupt function (maybe coded in asm)?. Does micropython allows frequent (> 500 hz) periodic functions calls in parallel with main thread?.

Please, if you have an idea or suggestions, let me know!.

Thanks in advance for your time!.

Re: Multithreading-like ideas?

Posted: Wed Mar 15, 2017 8:23 am
by pythoncoder
You should be able to read the ADC using a timer interrupt. But these rates are quite fast for an ESP8266 so the interrupt handler needs to be minimal. I'd suggest putting the raw data into a circular buffer. The main loop would periodically read all available data from the buffer and do the calculations; clearly this would need to be done relatively frequently to prevent overruns. Every minute it would post the results.

The size of the circular buffer is determined from the sample rate and the maximum length of time the main loop is unable to read from it. I assume the time taken to post the results.

You may need to run the ESP8266 at 160MHz.

Re: Multithreading-like ideas?

Posted: Wed Mar 15, 2017 8:54 am
by Roberthh
I have an application which reads data from the ADC in a timer ISR and puts it into an circular buffer. The granularity of the time interrupt is ms. Therefore the fastest rate you can achive is 1 kHz. If the ISR just takes the value from the ADC and puts it into a cicrular buffer, it's execution time is like 300 µs. You can use the @micopython.native decorator to make it a little bit faster. A skeleton class could look like:

Code: Select all

import machine
import array

_BUFFERSIZE = const(32) # Size of the Ringbuffer
_PERIOD = const(2)  # Period; here 2 ms == 500 Hz

class acquire:

    def __init__(self):
        self.adc_read = machine.ADC(0).read
        self.data = array.array("H", [0] * _BUFFERSIZE)
        self.reset_index()

    @micropython.native
    def sample(self, x):
        if not self.irq_busy:
            self.irq_busy = True
            self.data[self.index_put] = self.adc_read()
            self.index_put = (self.index_put + 1) % _BUFFERSIZE
            self.irq_busy = False

    def reset_index(self):
        self.index_put = 0
        self.index_get = 0
        self.irq_busy = False
#

acq = acquire()
tim = machine.Timer(-1)
tim.init(period=_PERIOD, mode=machine.Timer.PERIODIC, callback=acq.sample)

Re: Multithreading-like ideas?

Posted: Wed Mar 15, 2017 2:55 pm
by jcea
Great, so I can read the ADC from a periodic timer.

Do these timers have the same limitations as interruptions? That is, no python object creation, no floating point, etc.

Re: Multithreading-like ideas?

Posted: Wed Mar 15, 2017 3:04 pm
by Roberthh
The timer callbacks have no limitations. So you can use floating point op's and allocate memory. However it is highly reccomended that they finish as fast as possible, and obviously within the timer period, best within 500µs. The best approach is to perform in the callback only the minimal needed tasks, like storing the data, and do everything else in the main thread. Timer callbacks can be interrupted by the same callback again. For this reasing there is the locking mechanism with self.irq_busy present. Otherwise you may get strange results.

Just to give you an impression of times required:
- calling and leaving the callback: ~100 µs
- reading the ADC: ~120 µs
- storing a value into the ring buffer: ~20 µs

The esp8266 is not really built for real time operations.

Re: Multithreading-like ideas?

Posted: Wed Mar 15, 2017 3:58 pm
by jcea
Roberthh, many thanks. I was confused because in https://docs.micropython.org/en/latest/ ... Timer.html timers are documented as having the same limitations that regular interruptions.

My code is almost ready. Results are good so far.

Thanks!.

Re: Multithreading-like ideas?

Posted: Thu Mar 16, 2017 9:19 am
by pythoncoder
Roberthh wrote:The timer callbacks have no limitations...
I think Damien advised against allocating memory in an ESP8266 interrupt handler. Alas I haven't managed to find the reference. My recollection of his advice is that allocations usually work but the behaviour is not guaranteed. Hence the note in the official docs.

Re: Multithreading-like ideas?

Posted: Thu Mar 16, 2017 9:52 am
by Roberthh
My experience about the actual behaviour:
- In a timer callback you can allocate memory, even if that may not be most timing efficient. That callback seems not act like an ISR.
- In a PIN interurrupt handler you must not allocate memory.

Re: Multithreading-like ideas?

Posted: Fri Mar 17, 2017 7:04 am
by pythoncoder
Perhaps @Damien can provide a definitive answer.

Re: Multithreading-like ideas?

Posted: Fri Mar 17, 2017 4:10 pm
by jcea
My timer code was behaving in strange ways until I rewrote it to be "restricted".

I would like very much to have documentation about this.