How to build uPy going forwards?

[elliotwoods]

Hi all!

As mentioned in the uPy readme, there are more reasons to write a C module for uPy than there are for normal Py. This means a lot of users will inevitably need to build uPy from source, as this is the only way to add a module.

Currently the method for building is a little complicated, fragile and confounding (I spent ~10 hours hitting walls on that before finally getting something stable). When we introduce variables (e.g. different board SDK's, ESP IDF v3.3 vs v4, etc / different OS's) we end up with a large diverging set of methodologies to document.

I can see the following possible options for building;
1. Per-platform instructions (current approach)
2. Docker / Vagrant containers for stable version builds (requires install of system services, large space requirements)
3. PlatformIO
4. Cloud containers / cloud build services (not sure which ones are free / suitable)

I know that Docker is discussed elsewhere, so I'll focus on PlatformIO here:

The idea would be to have uPy itself as a PlatformIO project. Ideally you could build for different boards simply by changing the platformio.ini file

Disadvantages of PlatformIO:
- It's a 3rd party platform
- Very different approach compared to current Makefile
- Might even require changes to the source (unlikely - but undesirable if true)

Advantages of PlatformIO:
- It already provides toolchains for ESP-IDF and a large range of other board platforms
- Works on Linux/OSX/Windows without installing any system services
- Some nice workflows (e.g. test in C using standard PlatformIO, then copy into a MicroPython project)
- User-friendly
- Many 'free features', e.g. debugging MicroPython C code on-chip

I'm wondering if this has been considered before, and whether it might be a potential direction in the future.

Thank you
Elliot

[jimmo]

To your general comment -- yes I agree the "onboarding" experience for a new developer can be quite confusing, and there's lots of work we can do to improve it. Especially if someone is not already familiar with the sorts of tools and styles used, there's a lot to learn. I think that part of this is just that the documentation makes a lot of assumptions, but also isn't always perfectly up to date. And partly to avoid it becoming a novel, it has to skip over a lot of details.

I would add that I personally quite like the way the MicroPython build system works... especially on STM32 and NRF5x where all you need is arm-none-eabi-gcc (which is available as a distro package). But yes, ESP32 and ESP8266 are a different story. Although docker does a lot to improve the situation... I would really like to update the esp8266 and esp32 README.md to make this the "preferred" option, but only so many hours in the day.

Taking the IDF 3.3 vs 4.0 situation... that is super unfortunate (and definitely something I didn't want to have to introduce, and hopefully temporary), but just documenting that requires explaining so many things.

That said -- yes! PlatformIO. It's awesome. Would love to see MicroPython be a part of it! Especially if it can be done in a way that works alongside the current system without a lot of parallel maintenance. It would make me extremely happy to be able to point people at Platform IO instead of a "here's README.md... uhh start with installing a toolchain...uhhh all the best!".