Memory usage understanding

[codyhanks]

I am not fully clear with how the micropython parser handles memory. When frozen byte code is loaded to the STM processors as part of the DFU files where is this being stored in the processor. Is there a limited space for frozen byte code ( not while running but in rom). I am looking at moving a relatively large Python program to the STM embedded platform. I will not need the flash drive on the processor for any reason but is there a method of allowing frozen bytecode to have extra room?

I am willing to also do research on this if I can get a slight idea of where to look for the details of where frozen bytecode is stored.

thanks

[dhylands]

Frozen byte code that is flashed along with the firmware is stored in flash. There is an upper limit to the amount of byte code which can be stored, but what that limit is depends on the size of the flash memory (which varies from chip to chip) and how big the rest of the firmware is.

Looking into the build-PYBV11 directory, for example, you'll see a frozen_mpy subdirectory which contains .mpy files. These are the byte code files produced from compiling the python sources. There is also a frozen_mpy.c file which contains C data structures which reflect those .mpy files. Those C files are just compiled in with the rest of the firmware and cause the size of the firmware to increase.

When you do a build, it prints out a little summary like this:

Code: Select all

LINK build-PYBV11/firmware.elf
   text	   data	    bss	    dec	    hex	filename
 328020	     36	  28248	 356304	  56fd0	build-PYBV11/firmware.elf

The PYB11 board uses a STM32F405RG which has 1 Mb of flash. The firmware above uses 348K of that, and the filesystem uses 112K, leaving 564K for expansion (i.e. could be frozen bytecode)

[codyhanks]

So if i am looking at the ".ld" file for the pyboard it shows
"
FLASH_ISR (rx) : ORIGIN = 0x08000000, LENGTH = 16K /* sector 0 */
FLASH_FS (rx) : ORIGIN = 0x08004000, LENGTH = 112K /* sectors 1,2,3,4 are for filesystem */
FLASH_TEXT (rx) : ORIGIN = 0x08020000, LENGTH = 896K /* sectors 5,6,7,8,9,10,11 */
"
these three items are the 1024K of Flash memory
the byte code flashed with firmware is part of the 896K or Flash sectors 5-11?
is that correct?

[dhylands]

The ELF file is actually split into 2 pieces. The ISRs need to be located starting at 0x08000000 so the first 16K flash block contains the ISR vectors and some other code. The filesystem then occupies 0x08004000 thru 0x0801ffff, and the rest of the firmware goes into the last 896K after the filesystem. If you run readelf on the firmware.elf file you'll see something like this:

Code: Select all

$ readelf -l build-PYBV11/firmware.elf 

Elf file type is EXEC (Executable file)
Entry point 0x804c9c9
There are 5 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x008000 0x08000000 0x08000000 0x039f4 0x039f4 R E 0x8000
  LOAD           0x010000 0x08020000 0x08020000 0x4c760 0x4c760 R E 0x8000
  LOAD           0x060000 0x20000000 0x0806c760 0x00024 0x0267c RW  0x8000
  LOAD           0x06267c 0x2000267c 0x0806c784 0x00000 0x04000 RW  0x8000
  LOAD           0x06667c 0x2000667c 0x0806c784 0x00000 0x00800 RW  0x8000

 Section to Segment mapping:
  Segment Sections...
   00     .isr_vector 
   01     .text 
   02     .data .bss 
   03     .heap 
   04     .stack 

The .isr_vector section goes into flash starting at 0x08000000 and the size is 0x39f4 which is bit less than 16K. The rest of the firmware is in the .text section which starts at 0x08020000 with a length of 0x4c760. You can ignore .heap and .stack sections. They're just there for space, their location will come out of RAM and they're allocated dynamically. The .data section contains non-zero initialized data, and .bss contains zero initialized data. Here, they're combined into one section. The MemSiz indicates the size of both, and the FileSiz indicates how much initialized data there is. the initialized data will come from flash as well and is copied into RAM by the startup routine. You can see many more details by looking at the linker map file (firmware.map)