"Big" array in sram | DMA/PIO

RP2040 based microcontroller boards running MicroPython.
Target audience: MicroPython users with an RP2040 boards.
This does not include conventional Linux-based Raspberry Pi boards.
Post Reply
kreasteve
Posts: 7
Joined: Tue Jan 25, 2022 7:48 am
Location: Germany

"Big" array in sram | DMA/PIO

Post by kreasteve » Tue Jan 25, 2022 8:55 am

Hello!

I'm trying to make a I2S sound generator with DMA and PIO.
The PIO works so far.

If I generate a array.array with a length above 8 words I get in trouble with ... mhm "fragmentation"???

Here is how my code looks like:

Code: Select all

ring_buffer_size = 8
ring_buffer = array.array('L', (0x80000000 for _ in range(ring_buffer_size)))
time.sleep_ms(100) #wait for gc
ring_buffer_address = uctypes.addressof(ring_buffer)

#next lines are abstract to give you an idea
dma_init(ring_buffer_address, ring_buffer_size)
pio_init()
pio_start(i2s_config)
time.sleep_ms(1)
dma_start()

while True:
	pass
With a buffer size of 8 words I get all words out in serial correctly. With 16 words or more I get the first 8 right but the others are not what I put into the array. I can't get the reason for this behavior. I allready played around with some delays to make shure the gc has time enough to allocate the memory.

My Question is, how I can make an array wich is represented by a compact bunch of words in SRAM followed by each other?

If you need more of my code for understanding it, let me know.

Best
Steven

kreasteve
Posts: 7
Joined: Tue Jan 25, 2022 7:48 am
Location: Germany

Re: "Big" array in sram | DMA/PIO

Post by kreasteve » Tue Jan 25, 2022 10:48 am

Ohh, the allocation is not the problem.

After hours of playing around I located the problem at the RING_SIZE setting of the DMA channel.

For now I can say:
  • The allocation is done right and quickly
  • for a ringbuffer size of 2**5 (8 Words) all is fine
  • for a ringbuffer size of 2**6 and above the error occurs
I'm going to find a solution and give you updates here.

For now I can allocate 2**13 words and shift them out correctly. But only once, not with the ring buffer function of the DMA.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: "Big" array in sram | DMA/PIO

Post by pythoncoder » Tue Jan 25, 2022 11:47 am

Have you looked at machine.I2S?

I'm surprised that you're using unsigned 32 bit integers. For audio applications 16 bits are plenty (CD quality). Does your output device have better than 16-bit resolution?

Re buffers the key is to instantiate them early and to ensure that they remain in scope - perhaps defined as a global. It is certainly possible to create buffers of ~50KiB in size on the Pico. If a large buffer goes out of scope and has to be re-created, you will encounter fragmentation problems.
Peter Hinch
Index to my micropython libraries.

kreasteve
Posts: 7
Joined: Tue Jan 25, 2022 7:48 am
Location: Germany

Re: "Big" array in sram | DMA/PIO

Post by kreasteve » Tue Jan 25, 2022 3:42 pm

Hi Peter,

the machine.I2S looks interesting and I'll give it a try.

Yes, I'm currently using 32 bit. This is because my PIO statemachine can handle up to 32 bit for each channel.

My final goal is to get more experience in using the DMA ring buffer.
In this project I want to be able to generate different I2S signals (justify, resolution, ...), but also PCM, properitary audio signals and maybe others.

The I2S PIO is working fine, I have only trubble with the ring buffer function of the DMA.

kreasteve
Posts: 7
Joined: Tue Jan 25, 2022 7:48 am
Location: Germany

Re: "Big" array in sram | DMA/PIO

Post by kreasteve » Tue Feb 01, 2022 7:48 am

pythoncoder wrote:
Tue Jan 25, 2022 11:47 am
Re buffers the key is to instantiate them early and to ensure that they remain in scope - perhaps defined as a global. It is certainly possible to create buffers of ~50KiB in size on the Pico. If a large buffer goes out of scope and has to be re-created, you will encounter fragmentation problems.
It is not easy to remain it in scope. Only a single python command anywhere in the code (e.g. time.sleep(0.1)) can collapse the array. It feels like Schrödingers Cat. Is it dead or not? What if you look in?
It might be that I don't fully understand what exactly python is doing.

I'm going to try to change the ram address (and length) in micropythons source code to give me a memory which isn't affected by gc or other python magic.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: "Big" array in sram | DMA/PIO

Post by pythoncoder » Tue Feb 01, 2022 10:26 am

kreasteve wrote:
Tue Feb 01, 2022 7:48 am
...
It is not easy to remain it in scope. Only a single python command anywhere in the code (e.g. time.sleep(0.1)) can collapse the array. It feels like Schrödingers Cat. Is it dead or not? What if you look in?
It might be that I don't fully understand what exactly python is doing...
I'm not entirely sure what you mean. My GUI applications typically declare a large global array for the frame buffer. The code is arranged so that this occurs as early as possible, before other modules are loaded, when fragmentation has not occurred. This array exists until the application terminates: GC doesn't touch it.

Globals are normally to be avoided but large arrays are an exception. You can always avoid the performance hit from accessing globals by creating a local reference.
Peter Hinch
Index to my micropython libraries.

kreasteve
Posts: 7
Joined: Tue Jan 25, 2022 7:48 am
Location: Germany

Re: "Big" array in sram | DMA/PIO

Post by kreasteve » Tue Feb 01, 2022 11:38 am

Hi Peter, makes sense. Thank you.
Now, I attached a example code.

The dma is just copying the ring_buffer 4 times into the target_buffer.

Code: Select all

blablabla...
...
dma_channel0.CTRL_TRIG.RING_SIZE = 7 #32 words
dma_channel0.CTRL_TRIG.INCR_READ = True #True: after read the adress is incremented by one
dma_channel0.CTRL_TRIG.INCR_WRITE = True
dma_channel0.CTRL_TRIG.DATA_SIZE = dma_dev.SIZE_WORD #size of each transfer

dma_channel0.CTRL_TRIG.EN = 1 #runs the channel
#dma is copying
If you append this line:

Code: Select all

print(target_buffer)
The array will be printed out correctly. Like "print(list(range(32))*4)" does

But if you append also this line:

Code: Select all

a=[x for x in range(10)]
The array looks like this:
... ,12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 42, 536907792, 0, 0, 0, 1, 2, 3, ...
The last 8 words are wrong.

Why? What I'm doing wrong?
Attachments
ringBufferTest.zip
(3.17 KiB) Downloaded 82 times

Post Reply