New to MicroPython, question about RAM.

[ZeekDev]

Hello.
I've recently bought a MicroPython Pyboard 1.1, and I noticed the board has 192k of RAM.
I was wondering if there is/there are plans to offer expansions for RAM, either by a connector or an expansion card? or if there is another way to expand memory at home.

While 192k will work wonders for my project, I may run into memory issues if I add more functions/systems on top of the board.

Cheers,
Zeek.

[jimmo]

My understanding is that while the STM32F4 in the Pyboard does support external RAM (via the FSMC), the particular package used in the Pyboard 1.1 does not have the pins (i.e. it would need to be the 100-pin or higher packages).

The other option is (Q)SPI RAM, which you could map for reading like regular SPI Flash but I'm fairly sure the STM32F4 doesn't support writing to SPI RAM.

[ZeekDev]

My understanding is that while the STM32F4 in the Pyboard does support external RAM (via the FSMC), the particular package used in the Pyboard 1.1 does not have the pins (i.e. it would need to be the 100-pin or higher packages).

The other option is (Q)SPI RAM, which you could map for reading like regular SPI Flash but I'm fairly sure the STM32F4 doesn't support writing to SPI RAM.

Got it.
Is there a difference between Heap and RAM?
It says "Estimated heap size, 100k" for my package (pyboard 1.1) but the actual RAM size is 192k?

[jimmo]

In this case, the heap is the "Python heap", i.e. it's what's available to your Python program to use for storing variables, plus anything used by the VM such as compiled bytecode.

The rest of the RAM is used by the stack and any memory used by the very low level parts of the firmware (i.e. the data and bss segments).

Note that on the STM32F405 in the PYBV11, there's 192kiB of RAM, but 64kiB of it is currently unused (the CCMRAM). So you have 128kiB RAM available, 16kiB of stack, and a few of data/bss, leaving ~100kiB for heap.

Taking advantage of the CCMRAM should be fairly straightforward... TBH I'm not entirely sure why this isn't done yet (i.e. putting the stack in CCMRAM and maybe the heap metadata). And possibly a good performance win?

[ZeekDev]

In this case, the heap is the "Python heap", i.e. it's what's available to your Python program to use for storing variables, plus anything used by the VM such as compiled bytecode.

The rest of the RAM is used by the stack and any memory used by the very low level parts of the firmware (i.e. the data and bss segments).

Note that on the STM32F405 in the PYBV11, there's 192kiB of RAM, but 64kiB of it is currently unused (the CCMRAM). So you have 128kiB RAM available, 16kiB of stack, and a few of data/bss, leaving ~100kiB for heap.

Taking advantage of the CCMRAM should be fairly straightforward... TBH I'm not entirely sure why this isn't done yet (i.e. putting the stack in CCMRAM and maybe the heap metadata). And possibly a good performance win?

Ah ok, is there any way in python to access the CCMRAM whatsoever?

[jimmo]

Ah ok, is there any way in python to access the CCMRAM whatsoever?

What do you mean by access from Python? i.e. you can always use machine.mem8 etc, but I guess you mean you want to store program data there (i.e. more heap).

However, I just remembered that MicroPython uses the (entire) CCMRAM for the filesystem block cache.

[ZeekDev]

Ah ok, is there any way in python to access the CCMRAM whatsoever?

What do you mean by access from Python? i.e. you can always use machine.mem8 etc, but I guess you mean you want to store program data there (i.e. more heap).

However, I just remembered that MicroPython uses the (entire) CCMRAM for the filesystem block cache.

I'm simply trying to figure out exactly how much ram I should expect to have for my Python program.
The program will have about ~2000 lines, I'm going to store it on a external SD card. My main issue is I want to make sure I don't get overloaded with memory errors (It will access variables/lists a lot), so i want to know exactly how much memory I should expect to have, as well as calculating how much memory each variable uses.

Basically, I want to maximize as much Heap that I have as possible.

[jimmo]

I'm simply trying to figure out exactly how much ram I should expect to have for my Python program.
The program will have about ~2000 lines, I'm going to store it on a external SD card. My main issue is I want to make sure I don't get overloaded with memory errors (It will access variables/lists a lot), so i want to know exactly how much memory I should expect to have, as well as calculating how much memory each variable uses.

Basically, I want to maximize as much Heap that I have as possible.

The simple answer is: ~100k minus whatever space is required for the program's bytecode.

So the main thing to be aware of when it comes to program size you need to think about four things
- Size of the Python text
- Size of the compiled bytecode
- Where the above two things are stored
- How much RAM (heap) and Flash (filesystem) you have (pyboard 1.1 is 100k RAM, 112k filesystem)

In the default setup (e.g. pyboard), you put the python code on the internal flash (should be enough room for 2000 lines of code), then MicroPython compiles that to bytecode, which is in RAM (heap). Generally the bytecode takes up a fair bit less space than the original code. But the more code you have, the more bytecode, and therefore less heap for variables etc.

Note that putting the .py file onto the SDCard doesn't help here -- you still have to put the bytecode in RAM.

Then the next option is to pre-compile to bytecode on your PC (i.e. to make a .mpy file), in which case you'll end up using less space on flash (or SD), but the same RAM (heap) requirement. (.mpy files cannot be executed directly from the filesystem...yet).

The final option, is to "freeze" the code into the firmware image. Now what happens is that the bytecode is in flash (in a different area to the filesystem) and MicroPython can execute it directly from flash -- so it takes up no RAM. This isn't particularly convenient for development, as it means you have to do a full firmware flash every time, but it can be quite useful to break your program up into modules and just freeze the rarely changing modules (especially third-party ones), so the actual code that needs to be in RAM is minimised.

[ZeekDev]

I'm simply trying to figure out exactly how much ram I should expect to have for my Python program.
The program will have about ~2000 lines, I'm going to store it on a external SD card. My main issue is I want to make sure I don't get overloaded with memory errors (It will access variables/lists a lot), so i want to know exactly how much memory I should expect to have, as well as calculating how much memory each variable uses.

Basically, I want to maximize as much Heap that I have as possible.

The simple answer is: ~100k minus whatever space is required for the program's bytecode.

So the main thing to be aware of when it comes to program size you need to think about four things
- Size of the Python text
- Size of the compiled bytecode
- Where the above two things are stored
- How much RAM (heap) and Flash (filesystem) you have (pyboard 1.1 is 100k RAM, 112k filesystem)

In the default setup (e.g. pyboard), you put the python code on the internal flash (should be enough room for 2000 lines of code), then MicroPython compiles that to bytecode, which is in RAM (heap). Generally the bytecode takes up a fair bit less space than the original code. But the more code you have, the more bytecode, and therefore less heap for variables etc.

Note that putting the .py file onto the SDCard doesn't help here -- you still have to put the bytecode in RAM.

Then the next option is to pre-compile to bytecode on your PC (i.e. to make a .mpy file), in which case you'll end up using less space on flash (or SD), but the same RAM (heap) requirement. (.mpy files cannot be executed directly from the filesystem...yet).

The final option, is to "freeze" the code into the firmware image. Now what happens is that the bytecode is in flash (in a different area to the filesystem) and MicroPython can execute it directly from flash -- so it takes up no RAM. This isn't particularly convenient for development, as it means you have to do a full firmware flash every time, but it can be quite useful to break your program up into modules and just freeze the rarely changing modules (especially third-party ones), so the actual code that needs to be in RAM is minimised.

Got it.
So, if I have modules that dont typically change a lot, e.g backend or "kernel-like" modules that dont change much, I should put them in the firmware.

Where can I find documentation on how to do that? Easily? I'm running windows so, its always harder

[pythoncoder]

See this doc.