Cross compiling Micropython from GITHUb source -errors

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Tue Nov 17, 2020 12:13 am

Oh, and it behaves exactly the same way without the PYHTON=python3 prefix, so previous attempts further back in the screen log were just as you advises, python 2

User avatar
jimmo
Posts: 2013
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Cross compiling Micropython from GITHUb source -errors

Post by jimmo » Tue Nov 17, 2020 1:31 am

RobinMosedale wrote:
Tue Nov 17, 2020 12:10 am
Do the build yourself from scratch, ab initio as it were. Scrape your screens and send them to me.

Happy head banging
I build the ESP32 several times per week using exactly the instructions in ports/esp32/README.md

I use Arch Linux not Ubuntu / Debian, but I don't think there should be a substantial difference.

Here are the steps that I run, they should match what's described in README.md.

1. in micropython/ports/esp32, run "make ESPIDF=" to print out the supported hashes. I use the 4.x one mostly, as it lets me use the pre-built toolchain. This output is the "configuration of compatible hashs" as you mention.

2. clone https://github.com/espressif/esp-idf.git and then checkout the relevant commit. Set the ESPIDF env variable to the location of this clone.

3. in micropython/ports/esp32, create a python 3 virtual environment, using the instructions (i.e. "python3 -m venv build-venv"), and enter it, and pip install the $ESPIDF/requirements.txt packages.

4. run ./install.sh in the ESP-IDF directory (i.e. $ESPIDF) to download the toolchain.

5. "source $ESPIDF/export.sh" from micropython/ports/esp32 (note: you might need to deal with the --no-site-packages bug in the idf scripts, the fix is described in README.md). This will set up the PATH, etc to point to the downloaded toolchain.

6. create a "makefile" in ports/esp32 containing at least "ESPIDF", "BOARD", and "include Makefile". This avoids you having to set these variables on the environment/commandline very time.

7. cd micropython/mpy-cross, run "make". This builds the micropython bytecode compiler.

8. cd micropython/ports/esp32, run "make". This will build the firmware for the selected board.


My day-to-day work flow is much simpler:

1. in micropython/ports/esp32, activate the existing virtual environment
2. in micropython/ports/esp32, source $ESPIDF/export.sh
3. make
RobinMosedale wrote:
Tue Nov 17, 2020 12:10 am
If this isn't possible, I'd suggest pulling all the time wasting misleading build stuff off GitHub, or perhaps I should.
I would love to remove the instructions that are for IDF 3.x and for the non-prebuilt-binaries steps, but unfortunately these steps are required for some people. PRs are definitely welcome if you can make it clearer, this documentation surely has suffered from small incremental fixes.

RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Tue Nov 17, 2020 1:50 am

Absolutely fabulous Jimmo.
I am completely indebted.
I'll try tomorrow

RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Wed Nov 18, 2020 2:12 pm

Jimmo,

I'm stunned.
It looks as though I've done a standard SPIRAM build.
Not sure how, since I was stumbling all over the place.
In the end I pinched the source of idf-tools.py from the 4.1 github repository. but along the route it was still reporting the wrong version of pyparsing.
However, after a few retries:-
printf "build-GENERIC_SPIRAM/esp-idf/bootloader_support/libbootloader_support.a\n build-GENERIC_SPIRAM/esp-idf/driver/libdriver.a\n build-GENERIC_SPIRAM/esp-idf/efuse/libefuse.a\n build-GENERIC_SPIRAM/esp-idf/esp32/libesp32.a\n build-GENERIC_SPIRAM/esp-idf/esp_ringbuf/libesp_ringbuf.a\n build-GENERIC_SPIRAM/esp-idf/heap/libheap.a\n build-GENERIC_SPIRAM/esp-idf/soc/libsoc.a\n build-GENERIC_SPIRAM/esp-idf/cxx/libcxx.a\n build-GENERIC_SPIRAM/esp-idf/pthread/libpthread.a\n build-GENERIC_SPIRAM/esp-idf/freertos/libfreertos.a\n build-GENERIC_SPIRAM/esp-idf/vfs/libvfs.a\n build-GENERIC_SPIRAM/esp-idf/json/libjson.a\n build-GENERIC_SPIRAM/esp-idf/log/liblog.a\n build-GENERIC_SPIRAM/esp-idf/xtensa-debug-module/libxtensa-debug-module.a\n build-GENERIC_SPIRAM/esp-idf/tcpip_adapter/libtcpip_adapter.a\n build-GENERIC_SPIRAM/esp-idf/app_trace/libapp_trace.a\n build-GENERIC_SPIRAM/esp-idf/app_update/libapp_update.a\n build-GENERIC_SPIRAM/esp-idf/newlib/libnewlib.a\n build-GENERIC_SPIRAM/esp-idf/nvs_flash/libnvs_flash.a\n build-GENERIC_SPIRAM/esp-idf/smartconfig_ack/libsmartconfig_ack.a\n build-GENERIC_SPIRAM/esp-idf/spi_flash/libspi_flash.a\n build-GENERIC_SPIRAM/esp-idf/ulp/libulp.a\n build-GENERIC_SPIRAM/esp-idf/lwip/liblwip.a\n build-GENERIC_SPIRAM/esp-idf/mbedtls/libmbedtls.a\n build-GENERIC_SPIRAM/esp-idf/mdns/libmdns.a\n build-GENERIC_SPIRAM/esp-idf/wpa_supplicant/libwpa_supplicant.a\n build-GENERIC_SPIRAM/esp-idf/sdmmc/libsdmmc.a\n build-GENERIC_SPIRAM/esp-idf/bt_nimble/libbt_nimble.a\n build-GENERIC_SPIRAM/esp-idf/esp_common/libesp_common.a\n build-GENERIC_SPIRAM/esp-idf/esp_event/libesp_event.a\n build-GENERIC_SPIRAM/esp-idf/esp_wifi/libesp_wifi.a\n build-GENERIC_SPIRAM/esp-idf/esp_eth/libesp_eth.a\n build-GENERIC_SPIRAM/esp-idf/xtensa/libxtensa.a\n" > build-GENERIC_SPIRAM/esp-idf/ldgen_libraries
GEN build-GENERIC_SPIRAM/esp32.project.ld
LINK build-GENERIC_SPIRAM/application.elf
text data bss dec hex filename
1210705 278612 37508 1526825 174c29 build-GENERIC_SPIRAM/application.elf
Create build-GENERIC_SPIRAM/application.bin
esptool.py v2.8
Create build-GENERIC_SPIRAM/firmware.bin
bootloader 23072
partitions 3072
application 1489456
total 1554992

Whether I can recreate this we'll see. I was just on the point of admitting defeat.

Sooooooo.

Very many thanks.
When I've figured out the exact sequence I may draft a guide that others might benefit. I do know that many have had difficulties

Next, blow the chip

Robin

RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Wed Nov 18, 2020 6:13 pm

Only to discover that WSL Linux doesn't support USB access. Now to find a?way to get it off there onto another machine to blow.

Nearly there.....

User avatar
dhylands
Posts: 3480
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Cross compiling Micropython from GITHUb source -errors

Post by dhylands » Wed Nov 18, 2020 6:55 pm

What I do is to use WSL to build, but I put the micropython source on a regular windows directory.

It turns out that it isn't safe for windows to directly access files from WSL, but the other way around works fine.

By keeping the source in a normal windows location it also allows regular windows editors (I use VSCode) to edit the files, build using WSL and then flash using regular windows.

RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Wed Nov 18, 2020 7:47 pm

Absolutely excellent Dave.

However, I've only just discovered how to access WSL files from Windows.

I'll remember that.

For the moment I'll copy the WSL build files onto Win10 and use the IDF Tools from there.

Tortuous isn't it.

However, feel a major hurdle overcome

Thanks to you both

Robin

RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Thu Nov 19, 2020 1:26 pm

Well, it seems that a new set of make and configuration files need to be set up otherwise menuconfig won't work. Insists on a build and flash which fails. Read the template files, further obfuscation.

Easier if I set aside a partition on a machine, install a proper Linux and start again rather than spending further hours discovering the intricacies of setting up IDF

User avatar
jimmo
Posts: 2013
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Cross compiling Micropython from GITHUb source -errors

Post by jimmo » Thu Nov 19, 2020 10:46 pm

RobinMosedale wrote:
Thu Nov 19, 2020 1:26 pm
otherwise menuconfig won't work
FWIW, menuconfig shouldn't be required at all to do anything with MicroPython.

RobinMosedale
Posts: 29
Joined: Fri Jul 26, 2019 9:40 pm

Re: Cross compiling Micropython from GITHUb source -errors

Post by RobinMosedale » Fri Nov 20, 2020 1:26 am

Well idf.py -p COM7 flash fails no cmake file

Post Reply