MicroPython Forum (Archive)

Hi guys,
I'm working on a Neopixel project, and have goten quite far using 8266, which is working out nicely. The project is nothing extreme, I'm aiming at around 200 pixels, it's three 8x8 grids and a few single pixels.
However, while the 8266 works well, it's a bit slow, and I'd like to be able to do a bit more stuff on it, which slows it down a bit. So I decided to step up to the esp32, which I figured would be faster and have more memory. While it definitely is faster, it also has problems with "twitching" pixels. It seems common, and I've read all internet has to say about it (I think). But I just want to make sure I didn't miss anything recent, most info seem quite old, is there a solid way to use 200+ neopixels from an ESP32 at the moment?

I've tried the undocumented "timing=True"-flag:
np = neopixel.NeoPixel(Pin(26), 128, timing=True)
I've tried the driver from: https://gist.github.com/nevercast/9c485 ... 4a22062795 which gets rid of the twitching, but does not take all of my 200 pixels without memory issues.

Is the best option right now to use the 8266, or did I miss anything for the esp32 that makes it usable and stable?

Thanks!

Unfortunately, I believe that is the current state of affairs.

I went into a little more detail in the March-April News Roundup. There are a few promising ways we can resolve the issue but, right now, I don't know of any great way to drive that many pixels with the ESP32 from MicroPython.

mattyt wrote: ↑

Sat Jun 06, 2020 2:28 am

Unfortunately, I believe that is the current state of affairs.

I went into a little more detail in the March-April News Roundup. There are a few promising ways we can resolve the issue but, right now, I don't know of any great way to drive that many pixels with the ESP32 from MicroPython.

A question for you Mattyt. I believe the reason RMT is unable to maintain perfect timings for Neopixels is due to RTOS getting in they way. Is it only the ESP32 port that uses RTOS?? Why use RTOS, is it because the ESP32 is dual core and RTOS is used for Multiprocessing??

OutoftheBOTS_ wrote: ↑

Sat Jun 06, 2020 9:40 pm

A question for you Mattyt. I believe the reason RMT is unable to maintain perfect timings for Neopixels is due to RTOS getting in they way.

I'm not aware of timing issues when using RMT. It's dedicated hardware so interrupts to the micro shouldn't interfere.

I'm glossing over some complexity here; from MicroPython a block of data is handed to the ESP-IDF and it's the IDF that takes care of breaking it up into smaller portions (if necessary) and sending it to the RMT HW. This process involves an interrupt and the micro's intervention - but that shouldn't be affected by the RTOS.

That's my understanding; it could be wrong. If anyone has a reproducible RMT glitch issue I'd like to see it.

No, the problem with RMT is that the memory has to be pre-allocated and it can be large.
We can work around this by pushing interrupts all the way through to MicroPython - so the data is generated on the Python side on-demand but this adds a lot of complexity and pushes timing constraints on to the user.

The RTOS does interfere when trying to generate a glitch-free output from the micro itself - for example with the typical GPIO bit-bang implementation.

OutoftheBOTS_ wrote: ↑

Sat Jun 06, 2020 9:40 pm

Is it only the ESP32 port that uses RTOS?? Why use RTOS, is it because the ESP32 is dual core and RTOS is used for Multiprocessing??

I'm not sure if it's the only port that uses RTOS but it's the only ports I commonly use with an RTOS. STM32 and nRF don't employ an RTOS.

The short answer as to why an RTOS is used is that the ESP-IDF that Espressif provides to give access to the ESP32 capabilities is built on top of FreeRTOS. MicroPython is built on top of the ESP-IDF so we're also using the RTOS. It's technically possible to build MicroPython against the bare-metal but it would be a much bigger porting effort, requiring the duplication of a lot of features that the IDF provides.

Why did Espressif choose to use an RTOS? Obviously I don't speak on their behalf but it would be for simplicity and flexibility. The ESP32 is a complex device and the RTOS provides a good level of abstraction (scheduler, threading primitives etc) to build on. It's particularly helpful when considering that there are a number of hardware modules like Wifi and Bluetooth where an RTOS can help with their asynchronous nature.

mattyt wrote: ↑

Sun Jun 07, 2020 5:10 am

No, the problem with RMT is that the memory has to be pre-allocated and it can be large.
We can work around this by pushing interrupts all the way through to MicroPython - so the data is generated on the Python side on-demand but this adds a lot of complexity and pushes timing constraints on to the user.

Ahh ok the problem with ESP32 using RMT for large number of neopixels has to do with the fact that RMT uses 32bits of RAM for every bit of data that is sent so for large number of Neopixels you can't setup the whole transmission due to the amount of RAM required so your trying to update RAM mid transmission.

Considering this problem maybe ESP32 might be better using the SPI method for sending the signal train to Neopixels rather than the RMT method? RMT requires 32bits for every bit sent and SPI requires 3 bits for every bit sent, so SPI requires dramatically less RAM. I have only used Micropython when playing with ESP32 and haven't played with the lower level functionality but hace played with bare metal functionality of STM32. With STM32 it has a double buffered circular buffer for DMA that allows you to setup the second buffer while the first is being transmitted, STM32 also has a DMA interrupt for both half transmission and full transmission so when using circular mode even with smaller buffer you can still be setting 1 half of the buffer while DMA is transmitting the other half.

OutoftheBOTS_ wrote: ↑

Sun Jun 07, 2020 7:26 am

Ahh ok the problem with ESP32 using RMT for large number of neopixels has to do with the fact that RMT uses 32bits of RAM for every bit of data that is sent so for large number of Neopixels you can't setup the whole transmission due to the amount of RAM required so your trying to update RAM mid transmission.

Exactly. RMT is very accurate and flexible but the cost is memory, particularly for many pulse transitions.

OutoftheBOTS_ wrote: ↑

Sun Jun 07, 2020 7:26 am

Considering this problem maybe ESP32 might be better using the SPI method for sending the signal train to Neopixels rather than the RMT method? RMT requires 32bits for every bit sent and SPI requires 3 bits for every bit sent, so SPI requires dramatically less RAM. I have only used Micropython when playing with ESP32 and haven't played with the lower level functionality but hace played with bare metal functionality of STM32. With STM32 it has a double buffered circular buffer for DMA that allows you to setup the second buffer while the first is being transmitted, STM32 also has a DMA interrupt for both half transmission and full transmission so when using circular mode even with smaller buffer you can still be setting 1 half of the buffer while DMA is transmitting the other half.

Yes, the SPI method could work - and, wouldn't you know it, Nicko has written a MicroPython library to drive WS281B's with SPI. Anyone able to try it out?

Incidentally, RMT allows those paging operations much like the circular buffer you described. I had intended to expose that all the way through to Python but it did occur to me that we could accept the colour data and manage RMT on the C-side, only consuming a buffer at a time. It would mean more C code but it ought to be efficient...

mattyt wrote: ↑

Thu Jun 11, 2020 10:04 am

Yes, the SPI method could work - and, wouldn't you know it, Nicko has written a MicroPython library to drive WS281B's with SPI. Anyone able to try it out?

Incidentally, RMT allows those paging operations much like the circular buffer you described. I had intended to expose that all the way through to Python but it did occur to me that we could accept the colour data and manage RMT on the C-side, only consuming a buffer at a time. It would mean more C code but it ought to be efficient...

SPI vs RMT

SPI disadvantage is it can only do timing of duty width of 1/3 and 2/3. This tends to be close enough and be within the wiggle room of most addressable RGB LEDs. The total frequency (1/period) is set only by the pre-scaler of the SPI so can't get as exact timings for freq as RMT or timers that use both a pre-scaler and counter to achieve the freq. Non precise timings is the biggest cause of problems with these addressable RGB LEDs

SPI advantage is that it only require less than 1/10 of RAM compared to RMT meaning it is easier to have a larger buffer meaining it has a longer time to update the buffer.

RMT disadvantage is that it uses a crazy amount of RAM witch will need faster updates due to the fact less data is being buffered.

RMT advantages is can create very precise timing of any freq and any duty width.

There is also the timer method that I have developed and tested (tested by both me and the Ardunio boys, yes the ardunio manged to break it at first till changed a few things and now it works flawlessly) on STM32F4 that I hadn't seen anyone else use. https://github.com/OutOfTheBots/STM32_NEOpixels_timer

Timer advantages is that it doesn't use any extra RAM to buffer anything and doesn't use DMA. Can create perfect timings for both freq and duty width.

Timer disadvantageous is it requires fast interrupts to update the timer for each bit sent. Not sure this is possible with ESP32

BTW 1 aurdinuos guys took my timer method and put it on steroids so when the interrupt fires it sets all 4 channels of the timers so that 4 bits are sent in parallel and then used 4 of these 8x32 matrix https://www.aliexpress.com/item/3298303 ... 9976%23233 stacked on top of each other to create a display where each channel drives a matrix in parallel.

Then another 1 of the ardunio guys took it even further and used 1 timer as a master timer and a second timer as a slave timer so when the interrupt fires it updates 8 channels (4 for each timer) to be able to send 8 bits in parallel.

My NeoPixels (60 LEDs) had the "twitching" pixels problem.

Then I found https://blog.adafruit.com/2017/05/03/ps ... ode-tweak/ . Holding data line low for 300 us is required after each np.write().

With MicroPython v1.15, the low-holding time after np.write() can be as short as 176 us, as shown in https://ibb.co/X8qBgQB

So after each np.write(), I simply hold the data pin low for 300 us (and leave it low).

The problem is gone.

In the next Version of Micropython (1.18?) should be a solution to this.

You find more Information here: https://github.com/micropython/micropython/pull/7985