STM32 Increasing Stack, Heap and/or Filesystem size

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
chrismas9
Posts: 152
Joined: Wed Jun 25, 2014 10:07 am

STM32 Increasing Stack, Heap and/or Filesystem size

Post by chrismas9 » Sun Aug 20, 2017 1:27 pm

I have found a way to increase the Stack, Heap and/or Filesystem size on most STM32 ports. All STM32 ports use an MCU linker script that allocates RAM based on physical blocks (SRAM1, SRAM2, CCM, etc) and uses all of the smaller block for the FFS cache. By reducing the cache size and moving the stack into the second RAM block I can increase the heap by 16k. Stack can be set between 16k and 48k depending on the MCU. The loss of cache size can be made up for by using part of each spare large sector.

I have tested this with STM32L476 Discovery and analysed STM32F405 Pyboard. Some MCU's have contiguous SRAM1 and SRAM2 and on these the heap can be increased by more. Here are two examples of the change, but other tradeoffs are also possible:

STM32L476. Flash 1024K, max Flash page size 2K, SRAM1 96K, SRAM2 32K (non contiguous)
Before: Stack 16K, Cache 32K, Text (uPy + Frozen) 496K, FFS 512K
After: Heap 16K increase, Stack 30K, Cache 2K, Text (uPy + Frozen) 496K, FFS 512K

STM32F405. Flash 1024K, max Flash page size 128K, SRAM1 128K, CCM 64K (non contiguous, CPU only)
Before: Stack 16K, Cache 64K, Text (uPy + Frozen) 896K, FFS 112K
After: Heap 16K increase, Stack 16K, Cache 48K, Text (uPy + Frozen) 640K, FFS 192K (or 512K / 240K)
or: Heap 16K increase, Stack 32K, Cache 32K, Text (uPy + Frozen) 640K, FFS 144K (or 512K / 176K)
or: Heap 16K increase, Stack 48K, Cache 16K, Text (uPy + Frozen) 640K, FFS 112K (or 512K / 128K)

Before proceeding I have two questions for the experts:

1. Is there any risk putting the cache in CPU only memory? Do any peripherals or DMA ever access the stack?

2. For people pushing the bounds what gain in Heap or Stack is desirable. I know it varies from application to application but is say a 16K increase in both worthwhile? I will make everything easily tuneable, but would like to set sensible defaults.

I plan to submit a PR for just the L475 and F405 for code review first, then update the whole STM32 family to make it easier to adjust memory allocation. There are basically four changes:
1. move all the MCU dependent definitions out of storage.c into the STM32xxx.ld linker scripts
2. add more flash regions in storage.c to allow use of multiple partial sectors (eg 32K of each 128K)
3. add the Cache and Stack to the memory area of the linker script so their sizes can be tuned within any memory block
4. concatenate SRAM1 and SRAM2 into a single memory pool when they are contiguous.

If you need a big Filesystem it will also be possible to sacrifice RAM and have a 128K cache allowing for a 512K or larger Filesystem on MCUs with enough RAM.

To change the memory allocation or Filesystem size you would just create a custom STM32xxx.ld linker script with no need to delve into storage.c

All feedback really welcome.

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by Damien » Tue Sep 05, 2017 6:11 am

Is there any risk putting the cache in CPU only memory? Do any peripherals or DMA ever access the stack?
The DMA on the STM32 is currently used for the following: SD card, DAC, SPI, I2C. The DAC, SPI and I2C are pretty much exclusively called by Python code with buffers allocated on the heap. SD card read/write can be called from Python with a buffer on the heap, or the USB MSC driver which has a statically allocated buffer in the BSS segment.

So it should be OK to put the stack into CPU-only RAM, ie not accessible by DMA.

I don't know of any peripherals that can access RAM directly... are there any? But anyway this shouldn't be an issue either.
For people pushing the bounds what gain in Heap or Stack is desirable.
A 16k increase in heap is certainly worthwhile. An increase in stack size is not as useful because it mainly just increases the maximum recursion depth, which usually isn't an issue.

It would certainly be useful to be able to configure the flash FS and memory layout in the linker script.

Drako
Posts: 12
Joined: Thu Oct 19, 2017 9:28 am

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by Drako » Thu Oct 19, 2017 9:42 am

Hello,

I am trying to increase the Filesystem size on a NUCLEO-144 Stm32F767ZI, but i can't really find out how to do it.

The manual of the Stm32F76xx say's it has 11 sectors in single block memory organization.
From the It seems "stm32f767.ld" it seems like only 7 sectors are used. I tried to add the missing sectors 8-11 by inserting the follwing line:

FLASH_FS2 (rx) : ORIGIN = 0x08100000, LENGTH = 1024K /* sectors 8, 9, 10, 11 (256K each) */

I also edited toe storage.c in line 94 by adding the follwing:

#define FLASH_MEM_SEG2_START_ADDR (0x08100000) // sector 8
#define FLASH_MEM_SEG2_NUM_BLOCKS (256) // sector 8,9,10,11: 4*32k=128k

Further more I edited the flash.c by adding the follwing in line 63:

static const flash_layout_t flash_layout[] = {
{ 0x08000000, 0x08000, 4 },
{ 0x08020000, 0x20000, 1 },
{ 0x08040000, 0x40000, 3 },
{ 0x08100000, 0x40000, 4 }, // Added by MD
};

But this seems not to increase the filesystem size when flashing micropython to the CPU.

Can you give me a hint on what i am doing wrong here, or tell how you managed to increase the filesystem size.
Its kind of sad to have a CPU with 2MB ram but only beeing able to access very little of it.

What would also be nice, would be to change the flash memeory organization to dual block, so we could access more of the flashs memory. As far as i understood the filesystem on µPython can only work with 64k of each flashpage size. Or did I get this wrong?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by dhylands » Thu Oct 19, 2017 4:25 pm

When you flash micropython using DFU (or deploy-stlink), it flashes around the filesystem. So the contents of the filesystem will remain untouched.

If you're trying to change the size of the filesystem, I'd recommend that you do a filesystem reset and see if that helps.
See: http://docs.micropython.org/en/latest/p ... boot-modes

Note that the use of SEG2 was intended for splitting the filesystem across non-contiguous blocks of flash. For example, the STM32F407 on the discovery board has 2 Mb of flash arranged as 2 identical 1Mb layouts, where each 1 Mb has 4x16K, 1x64K, 7x128K. SEG1 would use 3 of the 16K and the 64K and SEG2 would use the other 16K and 64K. The F407 only has 64K of CCRAM which means that the largest flash size that can be supported is 64K, which is why the 128K's couldn't be used.

Drako
Posts: 12
Joined: Thu Oct 19, 2017 9:28 am

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by Drako » Mon Oct 23, 2017 6:17 am

Thanks mate,

it worked out! Increased the filesystem size from 79 kb to 1,06 MB.

Keep up the great work.

EasyRider
Posts: 94
Joined: Wed Dec 30, 2015 8:17 am

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by EasyRider » Wed Nov 15, 2017 8:39 pm

I am looking at evaluating Micropython on STM32F767ZI Nucleo development board.

At this stage I am not set up with a tool chain to build my own firmware.

Based on "standard" daily built firmware from Micropython downloads.

1. What is the maximum size of filesystem available in flash?
2. How much free-mem (RAM) is available for micropython applications?

3. Does anyone have a more recent hex firmware version to upload using St Link instead of dfu format?

Best Regards
John

User avatar
nikol900
Posts: 4
Joined: Thu Feb 11, 2021 12:36 pm

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by nikol900 » Thu Feb 11, 2021 12:43 pm

chrismas9 wrote:
Sun Aug 20, 2017 1:27 pm
STM32F405. Flash 1024K, max Flash page size 128K, SRAM1 128K, CCM 64K (non contiguous, CPU only)
I plan to submit a PR for just the L475 and F405 for code review first, then update the whole STM32 family to make it easier to adjust memory allocation. There are basically four changes:

Hello!
I'm trying to expand the heap volume in my application on stm32f405 uPy.
can i see your configurations "405.ld"? :roll:

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by jimmo » Fri Feb 12, 2021 2:10 am

nikol900 wrote:
Thu Feb 11, 2021 12:43 pm
I'm trying to expand the heap volume in my application on stm32f405 uPy.
can i see your configurations "405.ld"?
The heap is limited by the available RAM. The default configuration uses all available RAM (left over after .bss and .data).

The only exception is that the CCM is reserved for the block cache. Making that available to the heap would require supporting multiple non-contiguous regions, but isn't currently available.

User avatar
nikol900
Posts: 4
Joined: Thu Feb 11, 2021 12:36 pm

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by nikol900 » Fri Feb 12, 2021 5:26 am

jimmo wrote:
Fri Feb 12, 2021 2:10 am

The heap is limited by the available RAM. The default configuration uses all available RAM (left over after .bss and .data).

The only exception is that the CCM is reserved for the block cache. Making that available to the heap would require supporting multiple non-contiguous regions, but isn't currently available.
but, chrismas9 increase HEAP to 16kb, moving regions describing this:
Heap 16K increase, Stack 16K, Cache 48K, Text (uPy + Frozen) 640K, FFS 192K (or 512K / 240K)
or: Heap 16K increase, Stack 32K, Cache 32K, Text (uPy + Frozen) 640K, FFS 144K (or 512K / 176K)
or: Heap 16K increase, Stack 48K, Cache 16K, Text (uPy + Frozen) 640K, FFS 112K (or 512K / 128K)
i tried to do something like this, and increased the heap by 16kb, but not sure if the markup is correct

Code: Select all

/*
    GNU linker script for STM32F405
*/

/* Specify the memory areas */
MEMORY
{
    FLASH (rx)      : ORIGIN = 0x08000000, LENGTH = 1024K /* entire flash */
    FLASH_ISR (rx)  : ORIGIN = 0x08000000, LENGTH = 16K /* sector 0 */
    FLASH_FS (rx)   : ORIGIN = 0x08004000, LENGTH = 144K /*112K sectors 1,2,3,4 are for filesystem */
    FLASH_TEXT (rx) : ORIGIN = 0x08020000, LENGTH = 640K /*896K sectors 5,6,7,8,9,10,11 */
    CCMRAM (xrw)    : ORIGIN = 0x10000000, LENGTH = 64K
    RAM (xrw)       : ORIGIN = 0x20000000, LENGTH = 128K
}

/* produce a link error if there is not this amount of RAM for these sections */
_minimum_stack_size = 2K;
_minimum_heap_size = 16K;

/* Define tho top end of the stack.  The stack is full descending so begins just
   above last byte of RAM.  Note that EABI requires the stack to be 8-byte
   aligned for a call. */
_estack = ORIGIN(RAM) + LENGTH(RAM);

/* RAM extents for the garbage collector */
_ram_start = ORIGIN(RAM);
_ram_end = ORIGIN(RAM) + LENGTH(RAM);
_heap_start = _ebss; /* heap starts just after statically allocated memory */
_heap_end = 0x20020000; /*0x2001c000 tunable 0x20020000*/
everything seems to work, but there is a suspicion that he did it incorrectly x__x

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: STM32 Increasing Stack, Heap and/or Filesystem size

Post by jimmo » Sun Feb 14, 2021 10:50 pm

nikol900 wrote:
Fri Feb 12, 2021 5:26 am
but, chrismas9 increase HEAP to 16kb, moving regions describing this:
Yep, what christmas9 described is moving the stack into CCM.
chrismas9 wrote:
Sun Aug 20, 2017 1:27 pm
All feedback really welcome.
nikol900 wrote:
Fri Feb 12, 2021 5:26 am
i tried to do something like this, and increased the heap by 16kb, but not sure if the markup is correct
I don't see where you're defining _sstack, but what it looks like is happening is that you're now making the heap use the entire SRAM1+SRAM2 (after ebss), which means it will collide with the stack. I imagine this will look like it's working until it doesn't.

0x20020000 is the same as ORIGIN(RAM) + LENGTH(RAM)

If you want to do this (make the heap use the entire remaining SRAM1+SRAM2, after ebss), then you need to move the stack somewhere else. It appears that's what Chris is doing, I imagine he's setting sstack and estack to put it in CCM. Because the stack is 16kiB, that's why he gets 16kiB of extra heap.

You need to be careful because the CCM is used for the flash cache. I don't think the filesystem uses any flash blocks larger than 16k (it uses the first four blocks, as you can see in the linker script).

Post Reply