Memory allocation for lists of different types
Posted: Thu Jun 20, 2019 10:02 am
Hello!
I'm working on a port of MicroPython for the CC1312 chip. MicroPython is used as an embedded scripting engine: I compile some Python code using cross compiler and lauch it in a separate thread running MicroPython VM. I need to pass some data between the python environment and the main C program, so I'm looking into micropython data types implementation.
In classic python lists store references to the elements, and the elements of list are stored somewhere separately, as described in https://www.laurentluce.com/posts/pytho ... mentation/ . Possibly, small integers may be an exception because python may pre-allocate them in constant memory.
So here comes the question:
The following code creates a list of large integers:
[code]
lst = [1000000] * 200
for i in range(0,200):
lst[i] = i + 1000000
print(lst[0])
[/code]
and this code requires 1200 bytes of heap memory, which means the 200-element list uses only 800 bytes (4 bytes per element). This should mean, if I'm correct, that the integer values are somehow stored directly in list, because if list was storing references, the required space must have been bigger, at least by sizeof(reference)*N.
If i modify the code to use floats:
[code]
lst = [1000000.0] * 200
for i in range(0,200):
lst[i] = i + 1000000.0
print(lst[0])
[/code]
code uses 4448 bytes, which means lists uses 4000 bytes to store data, and single value needs 4000/200 = 20 bytes. This is probably size of references in the list and the actual values stored separately, which seems to be the expected result.
memory usage is provided by gc debug output
So, how are lists of different types stored in memory in MicroPython?
I'm working on a port of MicroPython for the CC1312 chip. MicroPython is used as an embedded scripting engine: I compile some Python code using cross compiler and lauch it in a separate thread running MicroPython VM. I need to pass some data between the python environment and the main C program, so I'm looking into micropython data types implementation.
In classic python lists store references to the elements, and the elements of list are stored somewhere separately, as described in https://www.laurentluce.com/posts/pytho ... mentation/ . Possibly, small integers may be an exception because python may pre-allocate them in constant memory.
So here comes the question:
The following code creates a list of large integers:
[code]
lst = [1000000] * 200
for i in range(0,200):
lst[i] = i + 1000000
print(lst[0])
[/code]
and this code requires 1200 bytes of heap memory, which means the 200-element list uses only 800 bytes (4 bytes per element). This should mean, if I'm correct, that the integer values are somehow stored directly in list, because if list was storing references, the required space must have been bigger, at least by sizeof(reference)*N.
If i modify the code to use floats:
[code]
lst = [1000000.0] * 200
for i in range(0,200):
lst[i] = i + 1000000.0
print(lst[0])
[/code]
code uses 4448 bytes, which means lists uses 4000 bytes to store data, and single value needs 4000/200 = 20 bytes. This is probably size of references in the list and the actual values stored separately, which seems to be the expected result.
memory usage is provided by gc debug output
So, how are lists of different types stored in memory in MicroPython?