STM32F769I-DISC1 heap issues

[gen_spm_lth]

I developped a quite complex program in micropython (about 2000 lines) with frozen modules in the firmware and the board I'm using is the STM32F769I-DISC1. I flashed the firmware to the board using the STM32-link utility under windows plus the external loader (MX25L512G_STM32F769I-DISCO (Device Type: Nor Flash, start at 0x90000000). Everything seems to work fine but I have lots of issues with the heap (fragmentation?). Even if the total heap size on the board is 16MiB and the total heap used by the program rarely exceeds 3MiB, after a while the program stops working. Sometimes it simply resets, sometimes I get "mem manage" error. I placed at the start of the main the instruction gc.threshold(threshold) and I made several tests using threshold=2500000, threshold=2000000, threshold=1500000 but inevitably after sometime (3-8 hours) the program stops. At times it seems to me that something got corrupted in the SDRAM.
Why I can't use all the available heap and it seems that only about 3.5MiB is available? What are the options available to have the program run indefinetely without problems?
Many thanks.

[jimmo]

Hi,

Just to clarify you've manually enabled the SDRAM controller in mpconfigboard.h and adjusted sysclk?

Also, because you mentioned using stlink -- you've disabled the mboot config? Mboot is quite handy for these boards with external qspiflash because it handles writing to both internal and external flash.

[Damien]

From what you describe it doesn't sound like a heap fragmentation issue. Heap fragmentation would eventually lead to a MemoryError exception being raised, and then MicroPython will drop to the REPL. You can then run micropython.mem_info(1) to get a view of the heap to see how fragmented it is.

If you get a hard reset then there's a bug somewhere in the system (hard to say where without more details).

And what exactly do you mean by "mem manage" error?

[gen_spm_lth]

[quote=jimmo post_id=39214 time=1567859410 user_id=3071]
Hi,

Just to clarify you've manually enabled the SDRAM controller in mpconfigboard.h and adjusted sysclk?

Also, because you mentioned using stlink -- you've disabled the mboot config? Mboot is quite handy for these boards with external qspiflash because it handles writing to both internal and external flash.
[/quote]

@ jimmo
thank you for your reply.
yes I enabled the SDRAM controller in mpconfigboard.h and adjusted sysclk (I made it equal to 192MHz).
But I forgot to disable mboot. Shall I disable mboot in mpconfigboard.mk (USE_MBOOT ?= 0) and leave everything else the same?
BR

[jimmo]

But I forgot to disable mboot. Shall I disable mboot in mpconfigboard.mk (USE_MBOOT ?= 0) and leave everything else the same?

To be honest I don't think leaving it enabled would cause any problems. Sorry I probably could have phrased my question better, I was more suggesting that you could use mboot instead of using st-link and separately loading the spiflash using the loader. DFU+mboot can just flash both from the same .dfu file (this is a big part of why mboot exists, to simplify working with boards with code on external flash).

I have this exact board, so if there's any way you can make a small piece of code that reproduces the issue I might be able to investigate further. I realise that the 3-8 hour time-to-crash makes this difficult.

As Damien said, it would be good to know the exact text of the "mem manage" error.

[gen_spm_lth]

Hi jimmo,

I did this test on my board:
open REPL
check heap with micropython-mem_info(1)
there are more than 16MiB free
allocate 10MiB with a=bytearray(10000000)
I verified that heap got corrupted
See copy from REPL below


>>> import micropython
>>> micropython.mem_info(1)
stack: 556 out of 31744
GC: total: 16392960, used: 2288, free: 16390672
No. of 1-blocks: 36, 2-blocks: 15, max blk sz: 32, max free sz: 1024407
GC memory layout; from c005dd00:
00000: h=hhhhhBh==Sh=hhh==Bhhh==h=BShhh==h=Thhhh=h=h=====h=h=======ShTh
00400: h=Bh=hh=======h=====h===============================BShhThh=h=Bh
00800: =hh=h====B..h....h=....h=.......................................
(16005 lines all free)
a2000: ................................................
>>> a=bytearray(10000000)
>>>
>>> micropython.mem_info(1)
Traceback (most recent call last):
File "n▒▒▒*/▒(`▒}▒▒}waC▒▒▒c\▒▒▒#i▒▒v▒VP▒Z▒')▒▒{޺έ▒b▒▒c▒aw▒▒▒N▒-w2ޣ▒G p▒^▒bRWg▒▒g},[s۸▒▒}▒▒▒-dBY▒▒▒▒▒s~▒▒]▒I(▒▒▒▒▒B▒+▒
Υ(d▒V▒▒w
▒y▒S▒g▒▒ %*▒P▒Z▒▒pEG▒", line 1, in n▒▒▒*/▒(`▒}▒▒}waC▒▒▒c\▒▒▒#i▒▒v▒VP▒Z▒')▒▒{޺έ▒b▒g},[s۸▒▒N▒-w2ޣ▒G p▒^▒bRWg▒6Y▒▒▒^▒$▒}▒▒▒-dBY▒▒▒▒▒s~▒▒]▒I(▒▒▒▒▒B▒+▒
Υ(d▒V▒▒w
▒y▒S▒g▒▒ %*▒P▒Z▒▒pEG▒
n▒▒▒*/▒(`▒}▒▒}waC▒▒▒c\▒▒▒#i▒▒v▒VP▒Z▒')▒▒{޺έ▒b▒▒c▒aw▒▒▒N▒-w2ޣ▒G p▒^▒bRWg▒6Y▒▒▒^▒$▒}▒▒▒-dBY▒▒▒▒▒s~▒▒]▒I(▒▒▒▒▒B▒: name 'micropython' isn't defined
>>> PuTTYPuTTYPuTTY
>>>

Could you please replicate this same test on your exact same board?
Many Thanks

[jimmo]

Thank you for the detailed repro instructions, fantastic. I was able to repro on my board and I'm fairly sure I have the right fix. See https://github.com/micropython/micropython/pull/5088

[Damien]

The above fix was merged into master.

[gen_spm_lth]

@ jimmo

Excellent!! Very well done!!
I implemented the fix and my program has been running now for more than 15 hours without any error.
Thank you a lot for your support.