Create read-only array in code memory, not RAM?
-
- Posts: 20
- Joined: Wed Nov 01, 2017 8:00 pm
Create read-only array in code memory, not RAM?
I have a need to define some two dimensional arrays which, at run-time, will be read-only. I would therefore prefer them to be created in code memory rather than in precious RAM.
In 'C' I would force them to be created in ROM by declaring the array as CONST unsigned char myarray=.... etc, is there an equivalent way to force a read only array to be created in code memory rather than in RAM in Micropython?
In 'C' I would force them to be created in ROM by declaring the array as CONST unsigned char myarray=.... etc, is there an equivalent way to force a read only array to be created in code memory rather than in RAM in Micropython?
-
- Posts: 847
- Joined: Mon Nov 20, 2017 10:18 am
Re: Create read-only array in code memory, not RAM?
Mico Python has a const() fuction that does the same thing
see http://docs.micropython.org/en/latest/w ... ained.html
see http://docs.micropython.org/en/latest/w ... ained.html
-
- Posts: 20
- Joined: Wed Nov 01, 2017 8:00 pm
Re: Create read-only array in code memory, not RAM?
Thank you, I read that, but I still don't get it. I think I must have a blind spot for Python because, try as I might, I can rarely find a simple defined example which shows exactly what I think I want to do when looking through Python related docs. That document shows how to declare a simple variable as a CONST but doesn't seem to give examples for more complex data structures.
Take an actual fragment of code:
This defines and fills a two dimensional array, or what I call an array. (The actual values are irrelevant in this example).
By default this array is in RAM and can be written to as well as read from. I want to declare this to be a read-only array stored in the Flash rather than a read / write array stored in RAM.
What's the actual syntax used to make that distinction?
Take an actual fragment of code:
Code: Select all
MyArray = [[1,2,3,4,5],
[5,4,3,2,1],
[2,4,6,8,0],
[1,3,5,7,9],
[0,0,0,0,0]]
By default this array is in RAM and can be written to as well as read from. I want to declare this to be a read-only array stored in the Flash rather than a read / write array stored in RAM.
What's the actual syntax used to make that distinction?
Re: Create read-only array in code memory, not RAM?
No it does not. I looked at that C implementation for the Python function `const` and it does absolutely nothing. Literally. It merely returns its input.OutoftheBOTS_ wrote:Mico Python has a const() fuction that does the same thing
see http://docs.micropython.org/en/latest/w ... ained.html
Create read-only array in code memory, not RAM?
You CANNOT put something in flash memory simply by using “const” qualifier like you can in C.Siriushardware wrote:Thank you, I read that, but I still don't get it. I think I must have a blind spot for Python because, try as I might, I can rarely find a simple defined example which shows exactly what I think I want to do when looking through Python related docs. That document shows how to declare a simple variable as a CONST but doesn't seem to give examples for more complex data structures.
Take an actual fragment of code:
This defines and fills a two dimensional array, or what I call an array. (The actual values are irrelevant in this example).Code: Select all
MyArray = [[1,2,3,4,5], [5,4,3,2,1], [2,4,6,8,0], [1,3,5,7,9], [0,0,0,0,0]]
By default this array is in RAM and can be written to as well as read from. I want to declare this to be a read-only array stored in the Flash rather than a read / write array stored in RAM.
What's the actual syntax used to make that distinction?
When C code is compiled, the resulting binary is meant to be executed on hardware which has the concept of readonly and readwrite memory.
When .py is compiled to bytecode, it’s meant to run on a “virtual” machine for which there is no notion of readonly vs readwrite memory.
With current state of micropython, If you want to put something in a .py in flash, you have to freeze the .py and build it as part of the firmware. This is very cumbersome and in some cases not viable. For example, if your system is deployed in the field and accepts an arbitrary .py file to execute from some user, you cannot just rebuild the firmware every time a new .py file is uploaded.
If you want to move an object from RAM to readonly memory from with a .py, it’s not hard but you have to know the underlying storage mechanism - flash vs OS-provided filesystem - and C-programming to implement it.
Honestly this sounds like a good thing to implement as part of micropython; I’ll create an issue.
https://github.com/micropython/micropython/issues/4067
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Create read-only array in code memory, not RAM?
The const() keyword applies only to integers (or expressions which can evaluate to integers at compile time). It is a signal to the MicroPython compiler which replaces the symbol with a number. This saves time by eliminating a variable access. If the symbol begins with an underscore it also saves RAM. This prevents another module from importing the name so the compiler can skip allocating RAM for it.
At a 'C' level it does indeed return its contents. This allows code using the const keyword to run under CPython.
Storing data in read-only form is difficult. Python copies mutable objects into RAM (because they can be changed in code). So read-only objects must be immutable (bytes or str instances). Even then you have to apply attention to detail in your code to avoid inadvertent RAM use. As soon as you use methods like string join or format, copying to RAM takes place.
So, as far as I can see, lists or arrays can never be stored in read-only form in a way that actually saves RAM. It can be done, with care, using immutable objects and frozen bytecode. See this example with font files.
At a 'C' level it does indeed return its contents. This allows code using the const keyword to run under CPython.
Storing data in read-only form is difficult. Python copies mutable objects into RAM (because they can be changed in code). So read-only objects must be immutable (bytes or str instances). Even then you have to apply attention to detail in your code to avoid inadvertent RAM use. As soon as you use methods like string join or format, copying to RAM takes place.
So, as far as I can see, lists or arrays can never be stored in read-only form in a way that actually saves RAM. It can be done, with care, using immutable objects and frozen bytecode. See this example with font files.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Create read-only array in code memory, not RAM?
Python copies ALL objects into RAM because when `.make_new` is called, memory is allocated from the heap.pythoncoder wrote: ↑Tue Aug 21, 2018 8:23 amPython copies mutable objects into RAM (because they can be changed in code). [/url].
If you create a tuple - which is immutable - it will still exist in RAM.
-
- Posts: 20
- Joined: Wed Nov 01, 2017 8:00 pm
Re: Create read-only array in code memory, not RAM?
I would not expect this to ever be implemented.Siriushardware wrote: ↑Tue Aug 21, 2018 8:00 pmAlthough disappointing, that is essentially what I needed to know, for now. Thanks for taking the steps to suggest it to be implemented.
I fleshed out what an implementation would require
https://github.com/micropython/micropython/issues/4067
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Create read-only array in code memory, not RAM?
True, but you can prevent the copy to RAM using a memoryview, as per the frozen font example I cited above.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.