Organizing and storing arrays

[sushyoshi]

Hello everyone,

I am trying to find out what is a good way to create many arrays and access them (maybe through a list), while being the most memory effecient. So far this is what I have come to:

Code: Select all

def sine_wave(TONE_FREQUENCY_IN_HZ):
    
# create a buffer containing the pure tone samples
    global samples_per_cycle
    samples_per_cycle = SAMPLE_RATE_IN_HZ // TONE_FREQUENCY_IN_HZ
    sample_size_in_bytes = SAMPLE_SIZE_IN_BITS // 8
    samples = bytearray(samples_per_cycle * sample_size_in_bytes)
    volume_reduction_factor = 32
    srange = pow(2, SAMPLE_SIZE_IN_BITS) // 2 // volume_reduction_factor

    if SAMPLE_SIZE_IN_BITS == 16:
        format = "<h"
    else:  # assume 32 bits
        format = "<l"

    for i in range(samples_per_cycle):
        sample = srange + int((srange - 1) * math.sin(2 * math.pi * i / samples_per_cycle))
        struct.pack_into(format, samples, i * sample_size_in_bytes, sample)
     

    
    return (samples)


# ======= CREATE SAMPLES ===================================================================

sine_sample = [[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
               [sine_wave(55), sine_wave(58), sine_wave(62), sine_wave(65), sine_wave(69), sine_wave(73), sine_wave(78), sine_wave(82), sine_wave(87), sine_wave(93), sine_wave(98), sine_wave(104)],
               [sine_wave(110), sine_wave(117), sine_wave(124), sine_wave(131), sine_wave(139), sine_wave(147), sine_wave(156), sine_wave(165), sine_wave(175), sine_wave(185), sine_wave(196), sine_wave(208)],
               [sine_wave(220), sine_wave(233), sine_wave(247), sine_wave(262), sine_wave(277), sine_wave(294), sine_wave(311), sine_wave(330), sine_wave(349), sine_wave(370), sine_wave(392), sine_wave(415)],
               [sine_wave(440), sine_wave(466), sine_wave(494), sine_wave(523), sine_wave(554), sine_wave(587), sine_wave(622), sine_wave(659), sine_wave(698), sine_wave(740), sine_wave(784), sine_wave(831)],
               [sine_wave(880), sine_wave(932), sine_wave(988), sine_wave(1047), sine_wave(1109), sine_wave(1175), sine_wave(1245), sine_wave(1319), sine_wave(1397), sine_wave(1480), sine_wave(1568), sine_wave(1661)],           
               [sine_wave(1760), sine_wave(1865), sine_wave(1976), sine_wave(2093), sine_wave(2217), sine_wave(2349), sine_wave(2489), sine_wave(2637), sine_wave(2794), sine_wave(2960), sine_wave(3136), sine_wave(3322)]
               ]
               
tone_sample = sine_sample[4][0]               

So with this code I am creating a list containing the sine waves of different frequencies. Then I can easily access the array.
The code works, however I am not able to create any more of these list of arrays ( for example for square wave and triangle wave) because of the limit memory of the Pico.
Is there a way to be able to use less memory, or storing the arrays in the Pico flash memory?

[OutoftheBOTS_]

Looking at your code (not totally understanding it) it seems the amount f RAM your using is simple math. Number of samples multiplied by the size of each sample (2 bytes or 4 bytes) packed in to a bytearray.

looking at it you have 3 options:

1. choose a MCU with more RAM

2. either reduce the number of samples or reduce the accuracy of the samples down to 1 byte.

3. store all the different byte arrays in a file and just load only 1 at a time to use.

[karfas]

Ideas to reduce the RAM amount (and the amount of samples required):

1) sample the (sine!) wave only to 180°(=pi) and calculate the wave until 360°(2*pi) on-the fly when you give the waveform to your DAC.
This is a simple addition or subtraction and should therefore be fast enough.
Will save ~1/2 of current RAM usage.

2) Provided you can feed your DAC using different bitrates (or use a timer), you can also "sample" one sine wave and create different frequencies by switching the DAC feed bitrate.
Will save most of the RAM and might reduce the number of samples to one (in your example).

Regards,
Thomas

[pythoncoder]

In principle you only need 0-90° with symmetry about the X and Y axes providing the rest. However doing this on the fly isn't always practical, depending on the frequency.

A compromise is to store a single cycle in a circular buffer and adjust the DAC clock rate to produce the required frequency.

[sushyoshi]

Thank you all for the suggestions.

Yes. I also realized that I only need 1 period of a wave sample for sine ( or any other symmetric wave format). I am using the I2S module for this, however the module is still under development and not many "user friendly" features are available, like changing the frequency through the DAC clock rate.
Using my current (very limited) knowledge and the I2S module, the only solution to change the frequency of a wave is to create arrays and store them in the RAM at the initialization. This approach is very limited and I am not 100% happy with it. I do wish further development on the I2S module, but in reality I believe that I might need to take a look at the code and try t understand what is happening and adapt it to my needs.

Thanks everyone for the advise. I will try to focus on creating a module for that exact purpose:

Change frequency of a 1 period wave sample with an encoder.

As always, suggestions are appreciated.

[pythoncoder]

The I2S module does output data at a continuous rate, however that need not be a problem.

Consider my music player demo. This reads a stream of data from a disk file, buffering it for output to the I2S driver. The task is to populate the buffer with data from a single cycle sample, with the data resampled so that the buffer contains an integer number of cycles at the correct frequency. Once play begins, the driver repeatedly streams data from that buffer to the I2S device.

The buffer size would be determined by the lowest frequency you wanted to output, along with the output sample rate. So, if your lowest frequency is 20Hz and sample rate is 48KHz with 16 bit mono samples, your buffer would be a manageable

48000*2/20 = 4800bytes

A point of detail: note how the audio demo uses a memoryview object. This can be used to do allocation-free slicing of the buffer to make its apparent size match an integer number of cycles.

[pythoncoder]

You could try something like this (adapted for your hardware). I've tested it between 25Hz and 10KHz.

Code: Select all

from machine import I2S
from machine import Pin
import uasyncio as asyncio
from math import sin, pi

FMIN = const(20)  # Minimum output freq
TS = 1 / FMIN  # Duration of sample set
FS = const(48000)  # Sampling freq
NS = round(FS * TS)  # No. of samples
WAVSIZE = round(2 * NS)  # 16 bits/sample
samples = bytearray(WAVSIZE)
samples_mv = memoryview(samples)
SCALE = const(32767)

# ======= I2S CONFIGURATION =======

BUFSIZE = 10_240  # Internal I2S buffer
I2S_ID = 1

config = {
    'sck' : Pin('W29'),
    'ws' : Pin('W16'),
    'sd' : Pin('Y4'),
    'mode' : I2S.TX,
    'bits' : 16,  # Sample size in bits/channel
    'format' : I2S.MONO,
    'rate' : 48_000,  # Sample rate in Hz
    'ibuf' : BUFSIZE,  # Buffer size
    }

go_pin = Pin(Pin.board.Y10, Pin.IN, Pin.PULL_UP)
audio_out = I2S(I2S_ID, **config)
swriter = asyncio.StreamWriter(audio_out)

# ======= WAVEFORM GENERATORS =======
# Arg 0 < x < 0.5 is the fraction of a cycle increment of each iteration

waves = {
    "saw": lambda x : round(SCALE * (1 - 2 * x)),
    "square": lambda x : round(SCALE if x < 0.5 else -SCALE),
    "sine": lambda x : round(SCALE * sin(x * 2 * pi)), 
    }

def wavegen(dx, f):
    x = 0
    while True:
        yield f(x)  # 0 .. 1
        x = (x + dx) % 1

def populate(fo, wname):
    nc = int(fo * TS)  # No. of complete cycles to buffer
    ns = round(nc * FS / fo)  # No. of actual samples (effective buffer length)
    g = wavegen(fo / FS, waves[wname])
    i = 0
    for _ in range(ns):
        s = next(g)
        samples[i] = s & 0xFF
        i += 1
        samples[i] = s >> 8
        i += 1
    return ns  # Effective size of buffer in 16-bit samples

async def play(fo, wname):
    buflen = 2 * populate(fo, wname)
    while go_pin():
        # HACK awaiting https://github.com/micropython/micropython/pull/7868
        swriter.out_buf = samples_mv[:buflen]
        await swriter.drain()

def run(f, wname):
    try:
        asyncio.run(play(f, wname))
    finally:
        audio_out.deinit()
        _ = asyncio.new_event_loop()

Run with e.g.

Code: Select all

import awg
awg.run(1000, "sine")

[sushyoshi]

Thanks everyone for your suggestions.

Peter:
I tried your code and it does work well, however I dont really understand it. I am working on it to see if I can understand it better and incorportate in my project.
Here are some questions related to it:

1 - Is it possible to change the wave amplitude? I did it like this:

Code: Select all

volume_mod = 0.01
waves = {
    "saw": lambda x : round(SCALE*volume_mod * (1 - 2 * x)),
    "square": lambda x : round(SCALE*volume_mod  if x < 0.5 else -SCALE*volume_mod),
    "sine": lambda x : round(SCALE*volume_mod * sin(x * 2 * pi)), 
    }

Do you think I can change the amplitude of the waves on the fly with an enoder?

2 - Is it possible to run this in a while True loop ? I want my wave array to keep changing according to user input on the fly. Since the pico has 2 cores, I was thinking of the 2nd core would only be running the audio output and the main core would take all the user inputs and modify the array and other attributes.

Thanks again for the help. I am still very new at coding so understanding your code is a real challenge.

[pythoncoder]

The I2S.shift method handles volume changes. Note that its arg is a number of bits: this gives it a logarithmic response which is what you need for a volume control. My audio demo has a volume control driven by an encoder or pushbuttons. It uses I2S.shift and works well. A second advantage over your approach is that volume can be changed on the fly while a wave is playing, with no need to re-populate the buffer.

It is easy to change the wave on the fly. There is no need to use the second core: you just need to set a flag to cause play to stop (in my sample above I use a pin, but it could just as easily be a boolean). Once play has stopped, re-populate the buffer with the new wave and create a new instance of the play coroutine.

You may want to look at my uasyncio tutorial. Asynchronous coding may be unfamiliar to you, but it is an essential skill for firmware applications.

[sushyoshi]

The I2S.shift method handles volume changes. Note that its arg is a number of bits: this gives it a logarithmic response which is what you need for a volume control. My audio demo has a volume control driven by an encoder or pushbuttons. It uses I2S.shift and works well. A second advantage over your approach is that volume can be changed on the fly while a wave is playing, with no need to re-populate the buffer.

It is easy to change the wave on the fly. There is no need to use the second core: you just need to set a flag to cause play to stop (in my sample above I use a pin, but it could just as easily be a boolean). Once play has stopped, re-populate the buffer with the new wave and create a new instance of the play coroutine.

You may want to look at my uasyncio tutorial. Asynchronous coding may be unfamiliar to you, but it is an essential skill for firmware applications.

That is really a great and elegant solution for the volume problem.

Thank you for leading me into the uasyncio world. I didnt know about it, although I always asked myself what if we could schedule tasks to run instead of running the full program in a while true loop. I am reading through your explanation and everything is starting to make more sense with asynchronous tasks.