reducing memory

[pythoncoder]

So, to answer the question from @JimTal001, the key is to use the array module.

An array of 16*288 floats will use a little over 16*288*4 = 18,432 bytes. An array of 32 bit integers will use (give or take a few bytes) the same amount of storage. To save RAM the solution is to use an array of half words or even bytes (a bytearray). Whether this is feasible depends on the range of data values and the precision required.

There are other tricks which can be employed, depending on the nature of the data. But these only work if the data is non-random, and they trade reduced RAM space for increased code complexity. For example, slowly changing data can be stored in half words or bytes as deltas. The other obvious (but slow) option is to store the data on a disk device i.e. SD card.

[JimTal001]

Thanks everyone for your assistance here.

Currently, I'm using list. This is the way I initialize the list:

Code: Select all

data = [[0]*lineCnt for x in range(depthCnt)]

where lineCnt is typically 288 (5 minute readings for 24 hours)
and depthCnt is 7

The values are Temperature in deg C. One point precision is sufficient, but there will be negative values. The temperature range is (-20 to 50 deg C)

I am loading the list as I read through each line:

# read line in file
# parse line into 7 floating point values stored in args[]
...

Code: Select all

for x in range(0, depthCnt):
	dataArray[x][lineCnt] = float(args[x])
			
lineCnt += 1

...

What would you suggest in this case?

[Roberthh]

I think the conclusion was to use the array module to preallocate the data. because then it will stay packed. The code for a single block looks quite similar:

Code: Select all

from array import array
lineCnt = 288
depthCnt = 16
data = [ array("f", [0.0 for _ in range(lineCnt)]) for _ in range(depthCnt)]

And then, you can go ahead as previously. If possible, preallocate the arrays early in your code. The total space allocated is about 20 k, which should not be a problem on a PyBoard. If your code is large and the load fails during import with memory error, you could pre-compile it (using mpy-cross) or put it into flash.