Micropython data stored in Heap
Micropython data stored in Heap
Does Micropython stores all local and global variables in Heap?
Re: Micropython data stored in Heap
MicroPython stores most python objects in the heap. There are a number of objects which are stored in flash (like all of the pin objects).
Small ints (anything upto 31 bits) don't take up any space in the heap.
Small ints (anything upto 31 bits) don't take up any space in the heap.
Re: Micropython data stored in Heap
@dhylands: Thank you for clearing my doubt. I will really appreciate it if you could help me to visualize this thing.
I wants to store all local and global variable in the flash so if I exit from my current program and rerun it than it should have all the locals and global variable from my previous session.
Anything on this topic would be a great help.
I wants to store all local and global variable in the flash so if I exit from my current program and rerun it than it should have all the locals and global variable from my previous session.
Anything on this topic would be a great help.
Re: Micropython data stored in Heap
Objects stored in flash never change (i.e. are read-only).
locals lose their value as soon as the function they're defined in exit.
If you want to persist values, then you'll need to write the code to do that yourself, and store the data someplace (like on the filesystem, or in some battery backed RAM).
locals lose their value as soon as the function they're defined in exit.
If you want to persist values, then you'll need to write the code to do that yourself, and store the data someplace (like on the filesystem, or in some battery backed RAM).
Re: Micropython data stored in Heap
Correct me if I am wrong in this, I understood that I have to store the local and global dictionary that has the local and global variables information into the file system and later retrieve it from the same.dhylands wrote:Objects stored in flash never change (i.e. are read-only).
locals lose their value as soon as the function they're defined in exit.
If you want to persist values, then you'll need to write the code to do that yourself, and store the data someplace (like on the filesystem, or in some battery backed RAM).
I think in micropython it stores the local and global variable information in Hash map data structure so if I could access this data structure than I can save the variable information into file system.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Micropython data stored in Heap
It might help us understand what you're trying to achieve if you could clarify a few points. What platform are you considering, Unix or a microcontroller? Are you considering normal program termination or a program interrupted by an error condition or ctrl-C? Resuming the latter in a "new session" would require significant application-specific coding. I doubt a "generic" solution would be feasible.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Micropython data stored in Heap
I am right now considering Unix platform for this implementation and a program interrupted by ctrl-c or external factor like power interruption. I know that it requires application specific coding and generic solution would not work.pythoncoder wrote:It might help us understand what you're trying to achieve if you could clarify a few points. What platform are you considering, Unix or a microcontroller? Are you considering normal program termination or a program interrupted by an error condition or ctrl-C? Resuming the latter in a "new session" would require significant application-specific coding. I doubt a "generic" solution would be feasible.
I wants to start with a simple implementation and later will modify it for specific application.
Python has inbuilt function called Locals() and Globals() that displays all the locals and global variable and value. I am not sure If I could access it or modify it externally from the program.
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: Micropython data stored in Heap
The ctrl-C interrupt can occur at arbitrary points in your code, so you'll need to trap the exception in the outermost scope of the program. As @dhylands has pointed out, there is little point in saving local variables: the ones that were current when the exception occurred will be out of scope in the exception handler. Further, if your exception handler saves data and returns you to the REPL there is no way to resume execution at the point where the exception occurred; this would be the only way you could use the locals if there were a way of storing them. If you can follow that rather convoluted sentence
The conventional way to do persistent programming in Python is along the following lines. I would store the data which needs to be persistent in some kind of Python object, let's assume a dictionary. The objects in the dictionary could be arbitrary Python objects - integers, floats, strings, lists, sets, dicts or whatever. The application uses and maintains the data in this dict. The exception handler would use the pickle module to save the dict to a file. On startup the application would load the dict from file using pickle. (An implementation detail is that you need a way to handle the initial case where the file does not yet exist.)
You can then interrupt the program which will store the partial results. When you re-start it, it will start at the beginning using the data which was current when it was interrupted. In practice this simplistic approach usually has a drawback.
This is because you need to ensure that the dataset is consistent in such circumstances: an update could have been partially complete at the time of the interrupt. The solution to this can vary from trivial to difficult depending on the application. Often the best approach is to have the interrupt handler set a "stop" flag before allowing execution to re-commence at the point where the interrupt occurred. The application would test the flag at a suitable point, save the data and exit in an orderly fashion.
An alternative to pickle is ujson. This trades generality for cross-platform compatibility: the JSON standard does not support all possible Python data types.
The conventional way to do persistent programming in Python is along the following lines. I would store the data which needs to be persistent in some kind of Python object, let's assume a dictionary. The objects in the dictionary could be arbitrary Python objects - integers, floats, strings, lists, sets, dicts or whatever. The application uses and maintains the data in this dict. The exception handler would use the pickle module to save the dict to a file. On startup the application would load the dict from file using pickle. (An implementation detail is that you need a way to handle the initial case where the file does not yet exist.)
You can then interrupt the program which will store the partial results. When you re-start it, it will start at the beginning using the data which was current when it was interrupted. In practice this simplistic approach usually has a drawback.
This is because you need to ensure that the dataset is consistent in such circumstances: an update could have been partially complete at the time of the interrupt. The solution to this can vary from trivial to difficult depending on the application. Often the best approach is to have the interrupt handler set a "stop" flag before allowing execution to re-commence at the point where the interrupt occurred. The application would test the flag at a suitable point, save the data and exit in an orderly fashion.
An alternative to pickle is ujson. This trades generality for cross-platform compatibility: the JSON standard does not support all possible Python data types.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: Micropython data stored in Heap
Thanks for the information. I really appreciates the detailed explanation. I thinks it cleared my all doubts as before I was trying to implement this with low level c implementation but using python it will be easy to implement.pythoncoder wrote:The ctrl-C interrupt can occur at arbitrary points in your code, so you'll need to trap the exception in the outermost scope of the program. As @dhylands has pointed out, there is little point in saving local variables: the ones that were current when the exception occurred will be out of scope in the exception handler. Further, if your exception handler saves data and returns you to the REPL there is no way to resume execution at the point where the exception occurred; this would be the only way you could use the locals if there were a way of storing them. If you can follow that rather convoluted sentence
The conventional way to do persistent programming in Python is along the following lines. I would store the data which needs to be persistent in some kind of Python object, let's assume a dictionary. The objects in the dictionary could be arbitrary Python objects - integers, floats, strings, lists, sets, dicts or whatever. The application uses and maintains the data in this dict. The exception handler would use the pickle module to save the dict to a file. On startup the application would load the dict from file using pickle. (An implementation detail is that you need a way to handle the initial case where the file does not yet exist.)
You can then interrupt the program which will store the partial results. When you re-start it, it will start at the beginning using the data which was current when it was interrupted. In practice this simplistic approach usually has a drawback.
This is because you need to ensure that the dataset is consistent in such circumstances: an update could have been partially complete at the time of the interrupt. The solution to this can vary from trivial to difficult depending on the application. Often the best approach is to have the interrupt handler set a "stop" flag before allowing execution to re-commence at the point where the interrupt occurred. The application would test the flag at a suitable point, save the data and exit in an orderly fashion.
An alternative to pickle is ujson. This trades generality for cross-platform compatibility: the JSON standard does not support all possible Python data types.
Re: Micropython data stored in Heap
I was able to save the locals and global dict data into the file system. So my program has a function called checkpoint(), whenever called it will save the locals and global dict data and version into the file.
I also want to implement restart function so whenever the system reboots, It will check the file whether it has any previously stored data and version information. If yes then it should start executing the program after that checkpoint().
Is there any way I could implement this?
I also want to implement restart function so whenever the system reboots, It will check the file whether it has any previously stored data and version information. If yes then it should start executing the program after that checkpoint().
Is there any way I could implement this?