Filesystem size with custom STM32F7 build
Re: Filesystem size with custom STM32F7 build
Sorry work got in the way of my "play" time and I should be able to work on this today.
A key thing to know is that Micropython re-purposes the normal flash for firmware into a filesystem, that is why the flash is broken up. The filesystem needs to be managed by a buffer that can only be so big for practicality, which then limits sector size. Damien and the other developers really did have to balance pros and cons with the arrangement.
As a short and long term tip, files that rarely or never change should ideally be frozen in the firmware. This saves a lot of filesystem size.
I luckily have the same development board so if time permits I can even test this out on actual hardware. I think this should not be too much trouble just need the time to review and actually make some edits.
A key thing to know is that Micropython re-purposes the normal flash for firmware into a filesystem, that is why the flash is broken up. The filesystem needs to be managed by a buffer that can only be so big for practicality, which then limits sector size. Damien and the other developers really did have to balance pros and cons with the arrangement.
As a short and long term tip, files that rarely or never change should ideally be frozen in the firmware. This saves a lot of filesystem size.
I luckily have the same development board so if time permits I can even test this out on actual hardware. I think this should not be too much trouble just need the time to review and actually make some edits.
_______________
Travis Travelstead
Travis Travelstead
Re: Filesystem size with custom STM32F7 build
I was able to extend flash partition to 1.1 MB but MCU crashes after some data is written to it. l can write about 220 KB and after that MCU crashes. I guess it's an address issue and data gets written somewhere in a wrong location.
Here's what I did:
Added line to stm32f767.ld
Added two lines in flashbdev.c after
#elif defined(STM32F746xx) || defined(STM32F765xx) || defined(STM32F767xx) || defined(STM32F769xx)
So here's what was expected:
sector 0 -- FLASH_ISR (32K)
sectors 1-3 -- FLASH_FS (3x32K)
sectors 4-7 -- FLASH_TEXT (128K + 3x256K)
secorts 8-11 -- FLASH_FS2 (4x256K)
Things that are still not clear to me:
UPDATE:
I tried to upload ~200 KB jpg image. After downloading it, half of it was grey (damaged file), so I guess I still have 96K :/
Here's what I did:
Added line to stm32f767.ld
Code: Select all
FLASH_FS2 (r) : ORIGIN = 0x08100000, LENGTH = 1024K /* sectors 8-11 4*256=1024 */
#elif defined(STM32F746xx) || defined(STM32F765xx) || defined(STM32F767xx) || defined(STM32F769xx)
Code: Select all
#define FLASH_MEM_SEG2_START_ADDR (0x08100000) // sector 8
#define FLASH_MEM_SEG2_NUM_BLOCKS (2048) // 1024K/512 = 2048
sector 0 -- FLASH_ISR (32K)
sectors 1-3 -- FLASH_FS (3x32K)
sectors 4-7 -- FLASH_TEXT (128K + 3x256K)
secorts 8-11 -- FLASH_FS2 (4x256K)
Things that are still not clear to me:
- Why MCU is crashing?
- Does F767 has 11 or 23 sectors?
UPDATE:
I tried to upload ~200 KB jpg image. After downloading it, half of it was grey (damaged file), so I guess I still have 96K :/
Re: Filesystem size with custom STM32F7 build
Unfortunately, the flash blocks are too big. In order to use a 256k flash block you need 256k of ram reserved.
Currently, I believe only 64k of Ram is reserved which means that you can only use up to 64k blocks or with some special code the first 64k of the 256k blocks (which means that the remaining 192k of the block is wasted).
This happens because when you modify a flash block you need to read the entire contents into ram, erase the entire flash block, modify your in ram contents and write that back out to flash.
Currently, I believe only 64k of Ram is reserved which means that you can only use up to 64k blocks or with some special code the first 64k of the 256k blocks (which means that the remaining 192k of the block is wasted).
This happens because when you modify a flash block you need to read the entire contents into ram, erase the entire flash block, modify your in ram contents and write that back out to flash.
Re: Filesystem size with custom STM32F7 build
Is it possible to reserve 256k of ram? How?
Will it be enough to run Micropython with remaining ram?
Will it be enough to run Micropython with remaining ram?
Re: Filesystem size with custom STM32F7 build
It will depend on your exact device. Some of the F7s only have 256k of RAM.
It looks like the F767 has 384K of RAM and 128K of DTCM.
Whether 128k of ram is enough would depend on your programs (it’s enough for micropython).
I don’t think that anybody has ever tried to do what you want.
It looks like the F767 has 384K of RAM and 128K of DTCM.
Whether 128k of ram is enough would depend on your programs (it’s enough for micropython).
I don’t think that anybody has ever tried to do what you want.
Re: Filesystem size with custom STM32F7 build
Sorry work deadlines are keeping me distracted.
As Dave pointed out as well the sectors need to be continuous and cannot be broken up (using a segment can get around this a little) or you will cause issues, like you have seen. I went through this same thing when playing wit hit as well.
Regarding the buffer size, I am not going to pretend I fully understand how this is implemented but on the F767 that you are using there is 128K DTCM RAM so I believe the buffer can be increased from the one size fits all 32k to the 128k.
– DTCM-RAM on TCM interface (Tightly Coupled Memory interface): 128 Kbytes for critical real-time data.
The below edits (untested directly, but should work) will give you at least another 128k which will more than double your filesystem size. Once this is working without issue we can maybe get another 128k to 256k by playing with it more. But this is probably the most straight forward way to get a major increase.
In line 101 in this file
https://github.com/micropython/micropyt ... lashbdev.c
I would probably just separate out the F767 into its own definition like this. Remember to remove it from the other elif statements.
After that you would edit the lines 11 and 12 in this file to change the math to increase the FS and compensate the TEXT.
https://github.com/micropython/micropyt ... m32f767.ld
Then on line 60 of this file I would suggest making a F767 specific if statement to avoid confusion with something like this.
https://github.com/micropython/micropyt ... 32/flash.c
As Dave pointed out as well the sectors need to be continuous and cannot be broken up (using a segment can get around this a little) or you will cause issues, like you have seen. I went through this same thing when playing wit hit as well.
Regarding the buffer size, I am not going to pretend I fully understand how this is implemented but on the F767 that you are using there is 128K DTCM RAM so I believe the buffer can be increased from the one size fits all 32k to the 128k.
– DTCM-RAM on TCM interface (Tightly Coupled Memory interface): 128 Kbytes for critical real-time data.
The below edits (untested directly, but should work) will give you at least another 128k which will more than double your filesystem size. Once this is working without issue we can maybe get another 128k to 256k by playing with it more. But this is probably the most straight forward way to get a major increase.
In line 101 in this file
https://github.com/micropython/micropyt ... lashbdev.c
I would probably just separate out the F767 into its own definition like this. Remember to remove it from the other elif statements.
Code: Select all
#elif defined(STM32F767xx)
#define CACHE_MEM_START_ADDR (0x20000000) // DTCM data RAM,
#define FLASH_SECTOR_SIZE_MAX (0x08000) // 128k max
#define FLASH_MEM_SEG1_START_ADDR (0x08008000) // sector 1
#define FLASH_MEM_SEG1_NUM_BLOCKS (448) // sectors 1,2,3,4 : 32k+32k+32+128 = 224 .... (224*1024)/512 = 448
https://github.com/micropython/micropyt ... m32f767.ld
Code: Select all
FLASH_FS (r) : ORIGIN = 0x08008000, LENGTH = 224K /* sectors 1, 2, 3 (32K each) + an extra 128*/
FLASH_TEXT (rx) : ORIGIN = 0x08020000, LENGTH = 1792K /* sectors 5-11 7*256KiB = 1792K */
https://github.com/micropython/micropyt ... 32/flash.c
Code: Select all
#elif defined(STM32F7)
// FLASH_FLAG_PGSERR (Programming Sequence Error) was renamed to
// FLASH_FLAG_ERSERR (Erasing Sequence Error) in STM32F7
#define FLASH_FLAG_PGSERR FLASH_FLAG_ERSERR
#if defined(STM32F767xx)
static const flash_layout_t flash_layout[] = {
{ 0x08000000, 0x08000, 4 },
{ 0x08020000, 0x20000, 1 },
{ 0x08040000, 0x20000, 7 },
};
#endif
_______________
Travis Travelstead
Travis Travelstead
Re: Filesystem size with custom STM32F7 build
Might help with add some visualization to my last past. Finally realized that I can use code for pasting my tables correctly.
Code: Select all
| Sector | Size KB | From (Hex) | Too (Hex) | Original | Modified |
| ------ | ------- | ---------- | --------- | -------- | -------- |
| 0 | 32 | 08000000 | 08007FFF | 32 | 32 |
| 1 | 32 | 08008000 | 0800FFFF | 96 | 224 |
| 2 | 32 | 08010000 | 08017FFF | | |
| 3 | 32 | 08018000 | 0801FFFF | | |
| 4 | 128 | 08020000 | 0803FFFF | 1920 | |
| 5 | 256 | 08040000 | 0807FFFF | | 1792 |
| 6 | 256 | 08080000 | 080BFFFF | | |
| 7 | 256 | 080C0000 | 080FFFFF | | |
| 8 | 256 | 08100000 | 0813FFFF | | |
| 9 | 256 | 08140000 | 0817FFFF | | |
| 10 | 256 | 08180000 | 081BFFFF | | |
| 11 | 256 | 081C0000 | 081FFFFF | | |
_______________
Travis Travelstead
Travis Travelstead
Re: Filesystem size with custom STM32F7 build
Thanks for your help!
Double checking few numbers:
and
Double checking few numbers:
Code: Select all
{ 0x08040000, 0x20000, 7 }, <--- shouldn't it be here 0x40000 as sector size is 256k?
Code: Select all
#define FLASH_SECTOR_SIZE_MAX (0x08000) <--- maybe 0x20000 for 128k?
Re: Filesystem size with custom STM32F7 build
Yes, those corrections sound good. I have not tested these so yes please do not blindly trust me.
_______________
Travis Travelstead
Travis Travelstead
Re: Filesystem size with custom STM32F7 build
I case you get this working can you post the settings that work, i am also looking to expand the flash on the nucleo-767zi. Maybe they can even use your settings for the newer dfu image on this website.
Thank you in advance.
Thank you in advance.