Micropython Build Environment for ESP32 in a Docker Container

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Tue Nov 26, 2019 3:16 pm

Following up on a conversation in another thread I created an example demonstrating how to install the ESP32 tool chain and use it inside of a Docker container.

https://github.com/wieck/micropython-esp32-devel

The same can certainly be done for other chip types, but so far I only have ESP32s running Micropython and could not test the result.

My goal is to present an easy to follow example how to build the Micropython firmware binaries independent of the used Linux distribution. Following the README.md the end result is that one can run commands like

Code: Select all

cd .../micropython/ports/esp32
esp32 make -j4
esp32 make erase
esp32 make deploy
As this is the initial commit, nothing is set in stone and suggestions/comments are welcome.


Regards, Jan

PM-TPI
Posts: 75
Joined: Fri Jun 28, 2019 3:09 pm

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by PM-TPI » Tue Dec 03, 2019 10:15 pm

I have made many attempts trying to build littlevgl with micropython.
Downloading/installing all tools to no avail.
It is sensory overload trying to understand all that is needed to build these firmware's .

Could you make a Docker Container for these two....

Firmware built with ESP-IDF v4.x, with support for bluetooth but no LAN or PPP:
GENERIC : esp32-idf4-20191203-v1.11-614-g82a19cb39.bin
GENERIC-SPIRAM : esp32spiram-idf4-20191203-v1.11-614-g82a19cb39.bin

but with LittlevGL added.
Gereric is for an Adafruit huzzah32 and
GENERIC-SPIRAM is for a wemos Lion D2 Pro

Is this doable?
I am currently using Windows 10 Subsystem for Linux (WSL) with Ubuntu.
Simple instructions for this environment would be also appreciated if possible.

Or better yet if someone can make completed binarys, this would be great!
I think LittlevGL should also have a download page with firmware's like Micropython does.
Having pre-built binaries gives new developers the opportunity to get right to coding, testing, and evaluating LittlevGL without all the "building" hassles.

If someone wants a real challenge, try porting this to a Pycom GPy board.
Thanks

User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Wed Dec 04, 2019 1:05 am

PM-TPI wrote:
Tue Dec 03, 2019 10:15 pm
Could you make a Docker Container for these two....

Firmware built with ESP-IDF v4.x, with support for bluetooth but no LAN or PPP:
GENERIC : esp32-idf4-20191203-v1.11-614-g82a19cb39.bin
GENERIC-SPIRAM : esp32spiram-idf4-20191203-v1.11-614-g82a19cb39.bin
My docker example is meant as generic build environment, not a single shot for a specific version of firmware. But yes, those binaries could be built using such a container. Just have to use the v4 IDF.
PM-TPI wrote:
Tue Dec 03, 2019 10:15 pm
but with LittlevGL added.
Gereric is for an Adafruit huzzah32 and
GENERIC-SPIRAM is for a wemos Lion D2 Pro
There is an important piece missing here. The LittlevGL seems to be a C library. Micropython would need a substantial amount of glue code to produce a usable Python API. Is there any such code available?


Regards, Jan

User avatar
mattyt
Posts: 410
Joined: Mon Jan 23, 2017 6:39 am

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by mattyt » Wed Dec 04, 2019 2:12 am

MostlyHarmless wrote:
Wed Dec 04, 2019 1:05 am
There is an important piece missing here. The LittlevGL seems to be a C library. Micropython would need a substantial amount of glue code to produce a usable Python API. Is there any such code available?
There is a fork of MicroPython that contains LittlevGL bindings - and describes how to build it. I followed it a few months ago and it worked fine; let me know if there are issues and I'll try to help.

User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Wed Dec 04, 2019 4:46 pm

mattyt wrote:
Wed Dec 04, 2019 2:12 am
There is a fork of MicroPython that contains LittlevGL bindings - and describes how to build it. I followed it a few months ago and it worked fine; let me know if there are issues and I'll try to help.
Thanks, that repo looks promising.

Compiled out of the box with the ESP-IDF v3 setup. However, I don't have such fancy touch-display in my spare parts box. All that I have right now are ssd1306 OLEDs. Does anyone have some example uPy code (simple Hello World will do) that can be used to test all this?


Regards, Jan

User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Wed Dec 04, 2019 7:07 pm

So far lvgl is at least twitching, but not really coming alive.

From the examples and the lvgl documentation I was assuming that lvgl doesn't need any real display, because the hardware should be driven by whatever the "callbacks" do. So I created the following test script:

Code: Select all

import lvgl as lv

def my_flush_cb(*args):
    print("my_flush_cb():", args)

def my_set_px_cb(*args):
    print("my_set_px_cb():", args)

def my_rounder_cb(*args):
    print("my_rounder_cb():", args)

disp_buf1 = lv.disp_buf_t()
buf1_1 = bytearray(480*10)
lv.disp_buf_init(disp_buf1,buf1_1, None, len(buf1_1)//4)
disp_drv = lv.disp_drv_t()
lv.disp_drv_init(disp_drv)
disp_drv.buffer = disp_buf1
disp_drv.flush_cb = my_flush_cb
disp_drv.set_px_cb = my_set_px_cb
disp_drv.rounder_cb = my_rounder_cb
disp_drv.hor_res = 480 
disp_drv.ver_res = 320 
lv.disp_drv_register(disp_drv)
Note that except for the additional set_px and rounder callbacks, this is pretty much exactly the example from the lv_micropython README.

It all works up to the last line (call to disp_drv_register). At that point I get:

Code: Select all

Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x401054fa  PS      : 0x00060b30  A0      : 0x8010554c  A1      : 0x3ffc8f70  
A2      : 0x3ffe7ca0  A3      : 0x00000080  A4      : 0x3ffe7c80  A5      : 0x3ffe7c78  
A6      : 0x3ffe7d75  A7      : 0x00000000  A8      : 0x00000000  A9      : 0x3ffc8f60  
A10     : 0x00000048  A11     : 0x3ffe7c80  A12     : 0x00000004  A13     : 0x3ffe7c84  
A14     : 0x000002ec  A15     : 0x00000000  SAR     : 0x0000001e  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x00000048  LBEG    : 0x4000c46c  LEND    : 0x4000c477  LCOUNT  : 0xffffffff  

ELF file SHA256: 0000000000000000000000000000000000000000000000000000000000000000

Backtrace: 0x401054fa:0x3ffc8f70 0x40105549:0x3ffc8fa0 0x400fb59c:0x3ffc8fc0 0x40103d5d:0x3ffc8fe0 0x401334a5:0x3ffc9000 0x4012edf6:0x3ffc9020 0x400df02d:0x3ffc9040 0x400df095:0x3ffc9060 0x400ec8af:0x3ffc9080 0x400e31bc:0x3ffc9120 0x400df02d:0x3ffc9190 0x400df05a:0x3ffc91b0 0x4013e2dc:0x3ffc91d0 0x4013e544:0x3ffc9270 0x400f3f98:0x3ffc92b0 0x4008e26d:0x3ffc92e0

Rebooting...
While playing with all this I also realized that building uPy firmware binaries with LittlevGL preinstalled is impractical. The physical display size and color depth are compile time options in the C code. We would essentially have to build separate binaries for every display type.


Regards, Jan

User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Thu Dec 05, 2019 10:47 pm

PM-TPI wrote:
Tue Dec 03, 2019 10:15 pm
I am currently using Windows 10 Subsystem for Linux (WSL) with Ubuntu.
Simple instructions for this environment would be also appreciated if possible.
While getting Ubuntu (16.04 and 18.04) WSL running is quite easy, running docker inside of them apparently is not. I tried this for a few hours on the side now and I give up. The docker service just doesn't want to start. There is a wonderful post here, but that procedure just doesn't seem to work with Windows 1909 any more.

If you can get docker to work on your WSL install, let me know. The usual docker "Hello World" will do, we can take it from there. And don't forget to follow up with a nice how-to, so others will be able to repeat the steps.


Regards, Jan

User avatar
elliotwoods
Posts: 23
Joined: Wed Dec 04, 2019 8:11 am

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by elliotwoods » Fri Dec 06, 2019 2:18 am

This is great!

Docker on Windows is best installed using their install tools + virtualisation, rather than via the Linux subsystem

Looking forward to trying this out!

User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Fri Dec 06, 2019 1:23 pm

Yes, I have experimented with Docker on Windows before. I stopped using it there because it is in conflict with VMware Workstation. I don't know if a container there can access USB ports and how well building images works, but will find out later today.

User avatar
MostlyHarmless
Posts: 166
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: Micropython Build Environment for ESP32 in a Docker Container

Post by MostlyHarmless » Fri Dec 06, 2019 7:46 pm

I followed this tutorial and it works well to the point of "docker run hello-world". The problems start there.

TL;DR is that sharing files between Docker and WSL becomes a nightmare.

Long version:
  • Accessing files stored on the EXT4 filesystem inside of WSL from somewhere else might be possible, but just doesn't feel right.
  • The Windows host drives are available from WSL as /mnt/c, /mnt/d, ..., however, one cannot use "git clone" there unless the drive is first remounted with "-o metadata". Cloning fails on a chmod attempt otherwise.
  • Inside of the Docker Container the Host drives can be made accessible as /c, /d, ..., however, now the git-clone'd repositories don't show up because they have non-default permissions.
With those problems for someone like me from the very start, I really don't want to "suggest" this setup to someone else.

The reason for me to use Docker is that my normal habitat consists of RedHat based VMs, either in VMware (laptop) or in QEMU-KVM (servers in the basement), or EC2 instances in the cloud. Getting this sort of toolchain working on a native CentOS 7 or one inside a VM? Good luck. But getting it working with the toolchain executed by a Ubuntu based container is rather simple, as my setup demonstrates.

WSL and Docker promise to provide Unix flavor on Windows without the resource overhead of a full Virtual Machine. While that is true in many cases, the whole "integration" required to do this type of stuff just isn't there yet.

My recommendation is to stay away from the Docker approach on Windows for now. Disk and RAM are cheap these days and it will be a lot less pain to just setup a full Ubuntu VM, then maybe use Docker containers inside of that (like I do).


Best Regards, Jan

Post Reply