Filesystem size with custom STM32F7 build

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
User avatar
TravisT
Posts: 72
Joined: Sun Feb 23, 2014 2:31 pm
Location: Portland, OR
Contact:

Re: Filesystem size with custom STM32F7 build

Post by TravisT » Wed Aug 19, 2020 4:07 pm

Sorry work got in the way of my "play" time and I should be able to work on this today.

A key thing to know is that Micropython re-purposes the normal flash for firmware into a filesystem, that is why the flash is broken up. The filesystem needs to be managed by a buffer that can only be so big for practicality, which then limits sector size. Damien and the other developers really did have to balance pros and cons with the arrangement.

As a short and long term tip, files that rarely or never change should ideally be frozen in the firmware. This saves a lot of filesystem size.

I luckily have the same development board so if time permits I can even test this out on actual hardware. I think this should not be too much trouble just need the time to review and actually make some edits.
_______________
Travis Travelstead

User avatar
untitled
Posts: 24
Joined: Sat Nov 02, 2019 1:44 pm

Re: Filesystem size with custom STM32F7 build

Post by untitled » Wed Aug 19, 2020 10:32 pm

I was able to extend flash partition to 1.1 MB but MCU crashes after some data is written to it. l can write about 220 KB and after that MCU crashes. I guess it's an address issue and data gets written somewhere in a wrong location.

Here's what I did:

Added line to stm32f767.ld

Code: Select all

FLASH_FS2 (r)   : ORIGIN = 0x08100000, LENGTH = 1024K   /* sectors 8-11 4*256=1024 */
Added two lines in flashbdev.c after
#elif defined(STM32F746xx) || defined(STM32F765xx) || defined(STM32F767xx) || defined(STM32F769xx)

Code: Select all

#define FLASH_MEM_SEG2_START_ADDR (0x08100000) // sector 8
#define FLASH_MEM_SEG2_NUM_BLOCKS (2048) // 1024K/512 = 2048
So here's what was expected:
sector 0 -- FLASH_ISR (32K)
sectors 1-3 -- FLASH_FS (3x32K)
sectors 4-7 -- FLASH_TEXT (128K + 3x256K)
secorts 8-11 -- FLASH_FS2 (4x256K)

Things that are still not clear to me:
  • Why MCU is crashing?
  • Does F767 has 11 or 23 sectors?

UPDATE:
I tried to upload ~200 KB jpg image. After downloading it, half of it was grey (damaged file), so I guess I still have 96K :/

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Filesystem size with custom STM32F7 build

Post by dhylands » Thu Aug 20, 2020 1:52 am

Unfortunately, the flash blocks are too big. In order to use a 256k flash block you need 256k of ram reserved.

Currently, I believe only 64k of Ram is reserved which means that you can only use up to 64k blocks or with some special code the first 64k of the 256k blocks (which means that the remaining 192k of the block is wasted).

This happens because when you modify a flash block you need to read the entire contents into ram, erase the entire flash block, modify your in ram contents and write that back out to flash.

User avatar
untitled
Posts: 24
Joined: Sat Nov 02, 2019 1:44 pm

Re: Filesystem size with custom STM32F7 build

Post by untitled » Thu Aug 20, 2020 2:37 pm

Is it possible to reserve 256k of ram? How?
Will it be enough to run Micropython with remaining ram?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Filesystem size with custom STM32F7 build

Post by dhylands » Thu Aug 20, 2020 3:16 pm

It will depend on your exact device. Some of the F7s only have 256k of RAM.

It looks like the F767 has 384K of RAM and 128K of DTCM.

Whether 128k of ram is enough would depend on your programs (it’s enough for micropython).

I don’t think that anybody has ever tried to do what you want.

User avatar
TravisT
Posts: 72
Joined: Sun Feb 23, 2014 2:31 pm
Location: Portland, OR
Contact:

Re: Filesystem size with custom STM32F7 build

Post by TravisT » Thu Aug 20, 2020 4:59 pm

Sorry work deadlines are keeping me distracted.

As Dave pointed out as well the sectors need to be continuous and cannot be broken up (using a segment can get around this a little) or you will cause issues, like you have seen. I went through this same thing when playing wit hit as well.

Regarding the buffer size, I am not going to pretend I fully understand how this is implemented but on the F767 that you are using there is 128K DTCM RAM so I believe the buffer can be increased from the one size fits all 32k to the 128k.
– DTCM-RAM on TCM interface (Tightly Coupled Memory interface): 128 Kbytes for critical real-time data.

The below edits (untested directly, but should work) will give you at least another 128k which will more than double your filesystem size. Once this is working without issue we can maybe get another 128k to 256k by playing with it more. But this is probably the most straight forward way to get a major increase.

In line 101 in this file
https://github.com/micropython/micropyt ... lashbdev.c

I would probably just separate out the F767 into its own definition like this. Remember to remove it from the other elif statements.

Code: Select all

#elif defined(STM32F767xx)

#define CACHE_MEM_START_ADDR (0x20000000) // DTCM data RAM, 
#define FLASH_SECTOR_SIZE_MAX (0x08000) // 128k max
#define FLASH_MEM_SEG1_START_ADDR (0x08008000) // sector 1
#define FLASH_MEM_SEG1_NUM_BLOCKS (448) // sectors 1,2,3,4 : 32k+32k+32+128 = 224 .... (224*1024)/512 = 448
After that you would edit the lines 11 and 12 in this file to change the math to increase the FS and compensate the TEXT.
https://github.com/micropython/micropyt ... m32f767.ld

Code: Select all

FLASH_FS (r)    : ORIGIN = 0x08008000, LENGTH = 224K     /* sectors 1, 2, 3 (32K each) + an extra 128*/
FLASH_TEXT (rx) : ORIGIN = 0x08020000, LENGTH = 1792K    /* sectors 5-11 7*256KiB = 1792K */
Then on line 60 of this file I would suggest making a F767 specific if statement to avoid confusion with something like this.
https://github.com/micropython/micropyt ... 32/flash.c

Code: Select all

#elif defined(STM32F7)

// FLASH_FLAG_PGSERR (Programming Sequence Error) was renamed to
// FLASH_FLAG_ERSERR (Erasing Sequence Error) in STM32F7
#define FLASH_FLAG_PGSERR FLASH_FLAG_ERSERR

#if defined(STM32F767xx)
static const flash_layout_t flash_layout[] = {
    { 0x08000000, 0x08000, 4 },
    { 0x08020000, 0x20000, 1 },
    { 0x08040000, 0x20000, 7 },


};
#endif
_______________
Travis Travelstead

User avatar
TravisT
Posts: 72
Joined: Sun Feb 23, 2014 2:31 pm
Location: Portland, OR
Contact:

Re: Filesystem size with custom STM32F7 build

Post by TravisT » Thu Aug 20, 2020 5:04 pm

Might help with add some visualization to my last past. Finally realized that I can use code for pasting my tables correctly.

Code: Select all

| Sector | Size KB | From (Hex) | Too (Hex) | Original | Modified |
| ------ | ------- | ---------- | --------- | -------- | -------- |
| 0      | 32      | 08000000   | 08007FFF  | 32       | 32       |
| 1      | 32      | 08008000   | 0800FFFF  | 96       | 224      |
| 2      | 32      | 08010000   | 08017FFF  |          |          |
| 3      | 32      | 08018000   | 0801FFFF  |          |          |
| 4      | 128     | 08020000   | 0803FFFF  | 1920     |          |
| 5      | 256     | 08040000   | 0807FFFF  |          | 1792     |
| 6      | 256     | 08080000   | 080BFFFF  |          |          |
| 7      | 256     | 080C0000   | 080FFFFF  |          |          |
| 8      | 256     | 08100000   | 0813FFFF  |          |          |
| 9      | 256     | 08140000   | 0817FFFF  |          |          |
| 10     | 256     | 08180000   | 081BFFFF  |          |          |
| 11     | 256     | 081C0000   | 081FFFFF  |          |          |
_______________
Travis Travelstead

User avatar
untitled
Posts: 24
Joined: Sat Nov 02, 2019 1:44 pm

Re: Filesystem size with custom STM32F7 build

Post by untitled » Thu Aug 20, 2020 8:01 pm

Thanks for your help!

Double checking few numbers:

Code: Select all

{ 0x08040000, 0x20000, 7 },  <--- shouldn't it be here 0x40000 as sector size is 256k? 
and

Code: Select all

#define FLASH_SECTOR_SIZE_MAX (0x08000) <--- maybe 0x20000 for 128k?

User avatar
TravisT
Posts: 72
Joined: Sun Feb 23, 2014 2:31 pm
Location: Portland, OR
Contact:

Re: Filesystem size with custom STM32F7 build

Post by TravisT » Thu Aug 20, 2020 11:59 pm

Yes, those corrections sound good. I have not tested these so yes please do not blindly trust me.
_______________
Travis Travelstead

Florin22
Posts: 1
Joined: Sun Aug 23, 2020 3:19 pm

Re: Filesystem size with custom STM32F7 build

Post by Florin22 » Mon Aug 31, 2020 6:12 am

I case you get this working can you post the settings that work, i am also looking to expand the flash on the nucleo-767zi. Maybe they can even use your settings for the newer dfu image on this website.
Thank you in advance.

Post Reply