Page 2 of 3

Solution architecture

Posted: Tue Mar 14, 2017 8:37 am
by pythoncoder
Perhaps you guys might like to comment on the solution I referenced ... This is based on a simple state machine designed to minimise the ISR overhead.

It doesn't use a ringbuffer, just an array indexed from 0. If the array fills up an overrun has occurred and an error is recorded: the NEC protocol (which has a fixed number of edges) has not been correctly received. The ISR simply records in the array the arrival time of an edge. This makes the ISR extremely simple and (hopefully) fast.

The arrival of the first edge triggers a software timer of 73ms. When this times out an entire NEC block should have been received and the decoding process begins. Note that the decode does not occur in an ISR context, so efficiency is not paramount.

At the end of a decode a user supplied callback is triggered with the decoded value, and the state machine is reset ready for the next burst.

I will test the code on the ESP8266 and report back.

Re: IR receiver/transmitter for ESP8266?

Posted: Tue Mar 14, 2017 9:03 am
by Roberthh
Hi Peter, the contribution I made was for the comparison of viper vs. native code, and a demonstration of direct port access. In my project, I'd need a ringbuffer, and this buffer access is relatively slow, making the speed improvement of viper neglegible.
For the IR receive with a limited length, this is obviously not needed, and also the polarity is not really an issue, because when you look at edges, you will always have alternating polarities (what goes up, must come down). So just collecting a fixed set of transitions in a limited time is valid, which simplifies the ISR a lot (no port reading, of sign calculation). And also in the example, the debug output is not needed, although the time penalty with direct port access is only a few µs. What I still find astonishing is the long time between the interrupt event and the first line of the ISR execution.

Re: IR receiver/transmitter for ESP8266?

Posted: Tue Mar 14, 2017 10:03 am
by pythoncoder
I've now tested it on the ESP8266 and the results are disappointing. It works most of the time, but sometimes it evidently misses a large number of interrupts reporting a bad block. The NEC protocol mandates 68 edges. On occasion, in the time window of 73ms from the first edge, it can miss half of these. This makes me question the interrupt mechanism on the ESP8266.

Robert: those times are indeed astonishing. It might be interesting to characterise the worst-case response time over a few thousand interrupts. On occasion my solution gets a "bad block" and fails to respond to further IRQ's. I suspect (but haven't yet proved) that on occasion the response time is so slow that even at the NEC protocol rate I'm getting re-entrancy.

The ESP8266 has a decidedly lethargic attitude to life on occasion :(

Re: IR receiver/transmitter for ESP8266?

Posted: Tue Mar 14, 2017 10:33 am
by Roberthh
Hi Pete,
in my project of the 'lie detector' I spent quite some time on ISR or timer callbacks and ISR execution time. I used an ADS1115 ADC either sampled by time callbacks or by the hardware interrupt generated by that ADC. In the ISR/Callback I read out tick_us() for timing. As signal I used a slow sinewave generated by an external source. My non-scientific experiance so far:

- calling an empty function on EPS8266 takes about 100 µs @ 80 MHz clock. About half of that should be the minimum response time of an ISR. B.T.W: on Pyboard the ISR response took 18 µs, so even that is not fast.
- The timer callback has an jitter of about 1 ms, with the average time being constant. If the callback takes longer than 500 µs, it is called again. So a blocking mechanism for nested calls is required. Not sure whether is is a bug or a feature.
- The hardware ISR showed a smootly distributed jitter of +/- 100 µs with rare larger delays of 400µs. The ADC has a jitter itself, but much smaller, like in the range of a few µs. I forgot how much, so I have to loik at it again.

For my purpose that was OK, since the signals I look at are <100Hz. Only the jitter reduces the digital filterung efficiency of mains noise (50Hz/60Hz). But since the ESP8266 clock is also not precise, anyhow not much can be achieved.

Edit: I added a graph showing the result, from top to down
- sampled sine
- first order derivate
- second order derivate
- ticks_us difference between adjacent samples
(121.68 KiB) Downloaded 175 times
Edit: The jitter of the ADC is below far 1 µs, more like 200 ns.

ESP8266 interrupt latency

Posted: Tue Mar 14, 2017 7:45 pm
by pythoncoder
I tested with this script (link GPIO pins 4 and 5):

Code: Select all

from machine import Pin, freq, disable_irq, enable_irq, Timer
from utime import ticks_us, ticks_diff, sleep

pin = Pin(5, Pin.IN)
pout = Pin(4, Pin.OUT)
t = ticks_us()
tmax = 0
count = 0

def phan(line):
    global tmax
    tmax = max(tmax, ticks_diff(ticks_us(), t))

def than(timr):
    global t
    t = ticks_us()

pin.irq(handler = phan, trigger=Pin.IRQ_RISING)
tim = Timer(-1)
tim.init(period=20, mode=Timer.PERIODIC, callback=than)

while True:
    print(count, tmax)
    count += 1
The outcome, at the standard clock rate, was a maximum latency of 867us with a typical value around 200us. At 160MHz the max latency was 643us - this corresponds to 102880 clock periods. Is this is a true hardware interrupt or some kind of soft IRQ mediated by the underlying OS?

Re: IR receiver/transmitter for ESP8266?

Posted: Tue Mar 14, 2017 8:40 pm
by Roberthh
I cannot tell. The figures you show are not good, but not much different to my sine wave test. For the timer, it's truly a software callback. You can allocate memory in it, and interrupts are still enabled. For the Port IRQ I had the impression of a hardware interrupt, because other interrupts were disabled and memory allocation failed; but i could run an I2C communication.
But there is a discussion to make interrupts soft:
That would make things even worse.

Re: IR receiver/transmitter for ESP8266?

Posted: Wed Mar 15, 2017 2:33 am
by mmh
I was able to consistently receive in a python circular buffer several sets of NEC code samples perfectly it felt like I could capture into the circular buffer for ever. The problem for me is beyond once anything else happens that causes memory allocation to occur I think my code suffers from huge time delays because of gc (still a slight question mark). I only suspect its GC. So to make my code really work I believe that I need to push my buffering into two sections bottom half fifo managed by isr, and top half decoder buffer which copies the samples from the lower half using a memory view copy. This seems way too complex to me managing such a tight timeline in python which is so sensitive to jitter. I guess filtering is reasonable but its hard to filter this when you get a glitch of time which is almost double what we are expecting in terms of time. I do need to figure out how to capture this condition on my logic analyzer. So I could change my decoder logic but then the problem will more than likely become a problem of what else I stack up on the esp8266 in python and I would like to run much more or I would just use the thing as a plain old Arduino with wifi.

I think the collection of transitions needs to be done in the internal gpio driver in C, does anyone know how that is done for (pyboard) ST part seems like they have a very nice solution there. I looked at the code its a lot more complex than what the ESP8266.

Peter I study the aremote code Esp8266 is missing the event stuff, I like the loop you wrote but not all the hooks are in place here to make that fly, but I have to admit I could be missing something.

Optional soft interrupts would be ok and a great feature but I think we would still need better performance for these low latency interfaces and now that Robert has taught me viper and native the picture gets nicer for dealing with realtime stuff.

Re: IR receiver/transmitter for ESP8266?

Posted: Wed Mar 15, 2017 7:29 am
by pythoncoder
@mmh Apologies, the asyn library with the Event class is here with my tutorial on uasyncio:

I have produced a variant of that runs on the ESP. However it has two issues. Firstly it produces some 'bad block' results. And secondly it occasionally gets into a mode where it fails to respond to interrupts and effectively hangs. It is still running, but it does not respond to IR signals until interrupted and restarted. Hanging never occurred on the Pyboard, and bad blocks were only reported when I deliberately presented malformed IR streams.

The measurements I've done on interrupt latency cause me to question whether any solution based on interrupts will be truly reliable on the ESP. Depending on the application occasional 'bad block' errors may be acceptable - if a device fails to respond to a remote, my response is to try again. Hanging is a total fail and I'll try to get to the bottom of this. If I can fix it I'll post some code.

Re: IR receiver/transmitter for ESP8266?

Posted: Wed Mar 15, 2017 8:22 am
by Roberthh
@Peter: Reading the docs, I came across the priority=x argument of the pin object constructur. Did you try that, or is is not available on ESP8266?

Re: IR receiver/transmitter for ESP8266?

Posted: Wed Mar 15, 2017 9:07 am
by pythoncoder
I tried this yesterday but it's not available.

On further testing the problem may be electrical. On both the Pyboard and the ESP8266 I occasionally get invalid decodes which occur without my using the remote; these may be in response to spurious IR sources. However these are rare on the Pyboard and frequent on the ESP8266. So there is a problem specific to the ESP. The physical test setups are as near identical as possible, and I'm using the same decoder. The chip is mounted on a tiny piece of stripboard with a decoupling cap and connected to the target with short leads.

Time to take a look with the scope. Inevitably with the ESP the decoder is mounted close to a 2.5GHz transmitter aerial...