Question about ESP32 heap size?

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
tangerino
Posts: 17
Joined: Sun Jul 25, 2021 8:34 am

Re: Question about ESP32 heap size?

Post by tangerino » Mon Oct 25, 2021 2:00 pm

I did few tests and I agree with comments.
This is my recipe for a new installation where I can pull everything, compile and run MP.
Other versions I always get errors.
Not to mention that using OSX on M1 also add some complexity
And I get the 'right' heap space.

Thank you all

Code: Select all

cd
rm -rf micropython
rm -rf esp
git clone https://github.com/micropython/micropython.git
mkdir esp
cd esp
git clone -b v4.2.2 https://github.com/espressif/esp-idf.git
cd esp-idf
./install.sh
. ./export.sh
cd
cd micropython
make -C mpy-cross
cd ports/esp32
make clean
make submodules
make

User avatar
glenn20
Posts: 132
Joined: Mon Jun 04, 2018 10:09 am

Re: Question about ESP32 heap size?

Post by glenn20 » Sat Oct 30, 2021 12:33 am

Thanks for the headsup on this. I was chasing all sorts of dark corners trying to find this.

Unfortunately I also ran into this bug (https://github.com/espressif/esp-idf/issues/5322) when I tried to re-install IDF v4.2.2 (which I had previously deleted).

I fixed that by adding:

Code: Select all

CONFIG_MBEDTLS_CERTIFICATE_BUNDLE_DEFAULT_CMN=y
to ports/esp32/boards/sdkconfig.base.

(Offered here in case anyone else is bitten by the same bug).

bulletmark
Posts: 59
Joined: Mon Mar 29, 2021 1:36 am
Location: Brisbane Australia

Re: Question about ESP32 heap size?

Post by bulletmark » Sat Oct 30, 2021 1:29 am

@glenn20, sorry I did not mention that bug. Yes, I get that bug but I get that with every IDF version I tried so I think it is just a generic problem which everybody hits atm.

User avatar
glenn20
Posts: 132
Joined: Mon Jun 04, 2018 10:09 am

Re: Question about ESP32 heap size?

Post by glenn20 » Mon Nov 01, 2021 4:22 pm

I have raised an issue against the esp-idf for this bug: https://github.com/espressif/esp-idf/issues/7808.

Since Espressif implemented the TLSF allocator (commit bd9b921713d80) on October 2020 the semantics of heap_cap_get_largest_free_block() has changed. It now returns the largest allocatable block rounded down to the nearest power of 2. This is also in disagreement with the documentation https://docs.espressif.com/projects/esp ... k8uint32_t.

Micropython uses this function to calculate how much memory to allocate for the micropython heap: ports/esp32/main.c:main_task(), which gets rounded down from 110592 bytes to 65536 bytes on the esp32. This only affects non SPIRAM modules.

The following quick hack patch works around this issue by using the returned value as a starting point and searching for the largest allocatable block by iteration.

I'll also post an issue against micropython for this upstream bug.

EDIT: Also note that on the ESP32S2 this fix will increase the amount of RAM allocated to the gc heap from 128k to 157k (which may starve the IDF chip functions of allocatable memory). The available memory on the ESP32S2 is less fragmented than the ESP32.

Code: Select all

diff --git a/ports/esp32/main.c b/ports/esp32/main.c
index 2ba613668..dce82b7fb 100644
--- a/ports/esp32/main.c
+++ b/ports/esp32/main.c
@@ -131,7 +131,20 @@ void mp_task(void *pvParameter) {
     #else
     // Allocate the uPy heap using malloc and get the largest available region
     size_t mp_task_heap_size = heap_caps_get_largest_free_block(MALLOC_CAP_8BIT);
-    void *mp_task_heap = malloc(mp_task_heap_size);
+    void *mp_task_heap;
+    for (size_t incr = mp_task_heap_size >> 1;
+         incr > 4;
+         incr >>= 1)
+    {
+        mp_task_heap_size += incr;
+        mp_task_heap = heap_caps_malloc(mp_task_heap_size, MALLOC_CAP_8BIT);
+        if (mp_task_heap == NULL) {
+            mp_task_heap_size -= incr;
+        }
+        heap_caps_free(mp_task_heap);
+        printf("HeapSize=0x%x  %d\n", mp_task_heap_size, mp_task_heap_size);
+    }
+    mp_task_heap = malloc(mp_task_heap_size);
     #endif
 
 soft_reset:

Post Reply