Trying to use the toolchain via docker

[pythoncoder]

Has anyone any thoughts on this? I feel sure it's almost working and I'm doing something very dumb. It seems like it can't find my Makefile.

[pythoncoder]

Thanks to @mattyt I now have some grasp of docker I can build the cross compiler, but whenever I try to build a native module it complains it can't find elftools.elf.

In the following test I changed the Makefile of examples/natmod/features0 to use xtensawin. Trying to build inside the container I get:

Code: Select all

path/to/micropython/examples/natmod/features0$ make
CC features0.c
LINK build/features0.o
Traceback (most recent call last):
  File "/mnt/qnap2/data/Projects/MicroPython/micropython/examples/natmod/features0/../../../tools/mpy_ld.py", line 32, in <module>
    from elftools.elf import elffile
ModuleNotFoundError: No module named 'elftools'

Line 32 of tools/mpy_ld.py reads

Code: Select all

from elftools.elf import elffile

but this file doesn't seem to exist in the source tree. Where can I find it?

[mattyt]

Elftools can be installed with:

Code: Select all

pip install pyelftools

[pythoncoder]

This is already installed on my real machine. If I fire up CPython3 it imports fine. But it's unavailable inside the docker container. I thought the point of the container is that dependencies are satisfied with the correct versions?

If I try to install it from the bash prompt within the container I get this result:

Code: Select all

 pip install pyelftools
WARNING: The directory '/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Defaulting to user installation because normal site-packages is not writeable
Collecting pyelftools
  Downloading pyelftools-0.27-py2.py3-none-any.whl (151 kB)
     |████████████████████████████████| 151 kB 2.7 MB/s 
Installing collected packages: pyelftools
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/.local'
Check the permissions.

In a normal bash shell I'd try sudo or su, but sudo is unavailable within the container and su produces an authentication failure.
[EDIT]
I guess I need to edit the Dockerfile to replace

Code: Select all

RUN pip3 install pyserial pyparsing==2.3.1

with

Code: Select all

RUN pip3 install pyserial pyparsing==2.3.1 pyelftools

and create a new docker image. I'm puzzled why I'm hitting this problem.

[pythoncoder]

I think I've fixed the interface between my brain and docker: I can now compile native modules with apparent success. I edited the Makefile in the examples/natmod/features0 demo to specify xtensawin, deleted the .mpy file and issued make from within the docker container. This appeared successful, generating features0.mpy without error.

But it wasn't. I copied features0.mpy to an ESP32 and tried to import it and got a crash.

A possible indication of what may be wrong is the outcome when I try to compile the ESP32 firmware:

Code: Select all

I have no name!@c91727514aad:/mnt/qnap2/data/Projects/MicroPython/micropython/ports/esp32$ make -j 8
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
Building with ESP IDF v4
CC /esp/components/bootloader_support/src/bootloader_clock.c
CC /esp/components/bootloader_support/src/bootloader_common.c
CC /esp/components/bootloader_support/src/bootloader_flash.c
CC /esp/components/bootloader_support/src/bootloader_flash_config.c
CC /esp/components/bootloader_support/src/bootloader_init.c
CC /esp/components/bootloader_support/src/bootloader_random.c
CC /esp/components/bootloader_support/src/bootloader_utility.c
CC /esp/components/bootloader_support/src/flash_qio_mode.c
/esp/components/bootloader_support/src/bootloader_common.c: In function 'bootloader_common_check_chip_validity':
/esp/components/bootloader_support/src/bootloader_common.c:309:29: error: 'CONFIG_IDF_FIRMWARE_CHIP_ID' undeclared (first use in this function)
     esp_chip_id_t chip_id = CONFIG_IDF_FIRMWARE_CHIP_ID;

[ttmetro]

I think I've fixed the interface between my brain and docker: I can now compile ...
[/code]

Here's another take on using docker to compile for the esp32:

Make sure you have docker installed. Then create a file with the text below somewhere on your path. Call it "idf.py" and make it executable.

Go to "ports/esp32" and run make as usual. The first time the docker image is pulled. Change the tag (see Note 1 below) to get a different ide version.

Code: Select all

#!/bin/bash
# idf.py

# we assume we are in ports/esp32
mp_root="../.."
mpy_cross="mpy-cross-ubuntu"

# build mpy-cross for Ubuntu
if [ ! -f "$mp_root/mpy-cross/$mpy_cross" ]; then
    echo building "$mp_root/mpy-cross/$mpy_cross"
    cd "$mp_root/mpy-cross"
    make PROG="$mpy_cross"
    cd -
fi

# and use it
export MICROPY_MPYCROSS="$mp_root/mpy-cross/$mpy_cross"

# delegate idf.py to docker
# Note 1: check https://hub.docker.com/r/espressif/idf/tags for available versions (Tags)
# Note 2: also set --device and/or --pivileged to enable flashing
docker run --rm -v $(realpath "$mp_root"):/mp -w /mp/ports/esp32 "espressif/idf:release-v4.2" idf.py "$@"

I have not tried this with other use cases. The $mp_root path may need adjusting.