Page 1 of 2
upip not working with IDF4 builds
Posted: Wed Nov 06, 2019 6:22 pm
by chibill
I am using builds esp32-idf3-20191106-v1.11-563-g4be316fb0 and esp32-idf4-20191106-v1.11-563-g4be316fb0 for this.
On both builds I first connect to my wifi then try to use upip.
I was trying to upip install urequests using
Code: Select all
upip.install("micropython-urequests")
On the IDF3 based builds the install works perfectly.
On the IDF4 based builds it throws an error.
>>> upip.install("micropython-urequests")
Installing to: /lib/
Error installing 'micropython-urequests': [Errno 12] ENOMEM, packages may be partially installed
I already asked in IRC but I figured it would be best to also post here for anyone else running into this problem.
Re: upip not working with IDF4 builds
Posted: Thu Nov 07, 2019 4:00 am
by chibill
More details on the IDF4 build running
before upip.install gives 103408 then after upip fails it returns 100480.
I am not going to bother running it on the IDF3 build as it works fine.
Running gc.collect() as suggested on IRC doesn't seem to help at all either.
Re: upip not working with IDF4 builds
Posted: Thu Nov 07, 2019 4:47 am
by jimmo
Thanks for the report.
I can repro the issue here, but only on non-psram devices (i.e. the generic build).
The ENOMEM is actually not related to the MicroPython heap running out of memory. The failure is coming out of mbedtls.
I'll look into it. I wonder if the allocator changes in IDF4 somehow such that heap_caps_get_largest_free_block ends up being too greedy and not leaving enough RAM left over for other functionality.
Re: upip not working with IDF4 builds
Posted: Thu Nov 07, 2019 5:34 am
by jimmo
Re: upip not working with IDF4 builds
Posted: Thu Nov 07, 2019 3:24 pm
by chibill
Interesting. I wonder why in IDF4 they decided to change how memory was segmented. Surprised this wasn't listed in the logs on Espressif's for the IDF4 vs IDF3 changes. (I looked once you mentioned something in IDF was likely causing it.
Re: upip not working with IDF4 builds
Posted: Thu Nov 07, 2019 6:47 pm
by chibill
I am also going to try your edit locally by building micropython. As that is how I found this bug originally and at first thought I had did something wrong while building it. Then found it to also affect the pre-built firmware.
Re: upip not working with IDF4 builds
Posted: Fri Nov 08, 2019 12:18 am
by chibill
Seems adding that patch to my local build doesn't fix the upip error. Is there a way to compile it to get extra debug info?
Re: upip not working with IDF4 builds
Posted: Fri Nov 08, 2019 1:54 am
by jimmo
Have a look at build-GENERIC/sdkconfig.h, with the patch you should see
Code: Select all
#define CONFIG_MBEDTLS_ASYMMETRIC_CONTENT_LEN 1
#define CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN 16384
#define CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN 4096
whereas before that change, it would just be
Code: Select all
#define CONFIG_MBEDTLS_SSL_MAX_CONTENT_LEN 16384
If you want to debug this further, put
Code: Select all
heap_caps_print_heap_info(MALLOC_CAP_INTERNAL | MALLOC_CAP_DEFAULT);
immediately before the call to
Code: Select all
ret = mbedtls_ssl_setup(&o->ssl, &o->conf);
in socket_new() in modussl_mbedtls.c
Re: upip not working with IDF4 builds
Posted: Sat Nov 09, 2019 10:11 pm
by chibill
Seems to work although it does output
Code: Select all
>>> upip.install("micropython-urequests")
Installing to: /lib/
Warning: micropython.org SSL certificate is not validated
Installing micropython-urequests 0.6 from https://micropython.org/pi/urequests/urequests-0.6.tar.gz
I don't remember the warning on the IDF3 version
Re: upip not working with IDF4 builds
Posted: Sun Nov 10, 2019 11:44 pm
by jimmo
I'm fairly sure the "Warning: micropython.org SSL certificate is not validated" was there before (or if it wasn't, it definitely wasn't validated before either)