How to freeze module for ESP32

[Peugot206]

Hello forum users,

For some libraries I want to use, my ESP32 does not have enough heap memory available, so I need to freeze the module in order to store it in the ROM and thus save RAM.
However, I see most tutorials point to this https://learn.adafruit.com/micropython- ... en-modules tutorial. This tutorial is for the ESP8266 and seems outdated. I can't really find any video's on this topic as well. I'm very rookie with all the stuff I see in the Adafruit tutorial and I'm not sure where to start.. is there a more beginner friendly tutorial somewhere?

I also found this 'tutorial' for the ESP32, but it's not really a tutorial: https://it-qa.com/how-to-freeze-esp32-m ... cropython/. It is more or less telling me that the freezing module for ESP8266 is mostly the same as ESP32. So that should be good news for the Adafruit tutorial, but I still can't figure out where to start..

[davef]

Look for a tutorial for building Micropython from the source files.
https://github.com/micropython/micropyt ... g-Started
Check that your result actually works on the ESP32 then add the code you want to freeze to:

Code: Select all

micropython/ports/esp32/modules

and re-build. There is probably a <clean> in there somewhere.

I was changing the code so often that I bought the SPIRAM variants, so I didn't have to do this.

[Peugot206]

Look for a tutorial for building Micropython from the source files.
https://github.com/micropython/micropyt ... g-Started
Check that your result actually works on the ESP32 then add the code you want to freeze to:

Code: Select all

micropython/ports/esp32/modules

and re-build. There is probably a <clean> in there somewhere.

I was changing the code so often that I bought the SPIRAM variants, so I didn't have to do this.

Thanks davef. After reading I have some questions:
- I know how to setup a venv using PyCharm on my Windows machine, but building is not supported on Windows it seems like? Do I need to setup a virtual machine on my PC with like Linux or something? I'm pretty new to all that kind of stuff.
- The library I want to freeze is multiple files, do I then add the whole folder to 'modules'? If I remember correctly there is a difference between adding files to the 'modules' and the 'scripts' folder, where one is compiled into Python bytecode already, to optimize (ROM?) usage even more.

[davef]

1) I only run Linux, no Windows experience for Micropython.
2) I added about 6-10 individual files to 'modules'

Hope someone chips-in and helps you out.

[Peugot206]

Ok so I've been spending hours now trying to get it working, but I'm having a hard time.

I've setup a VM with Ubuntu 16.04 (https://www.youtube.com/watch?v=jG7WBY_vmpE) following this guide it was recommended.
I ran in many installation errors while trying to install the esp-open-sdk toolchain, so I used this fork: https://github.com/esp-open-sdk/esp-open-sdk that fixes those installation errors. However whatever I tried I couldn't get any 'make' command working, (for the /ports/unix it worked, I got micropython running, but the make command for /ports/esp32 didn't). I assume it's because this toolchain is specifically for the ESP8266 (I thought I read somewhere that the process would be the same for the ESP8266 & ESP32).

So on to the next tutorial https://github.com/micropython/micropyt ... /README.md, trying to install this I ran into errors again that my Python version is not up-to-date. I'm unable to get it higher than Python 3.5 for some reason.. however I lowered the ESP-IDF version now and it seems to have installed the toolchain. Now once again I'm in the folder /micropython/ports/esp32 and put the command 'make' and get the error:

Code: Select all

idf.py -D MICROPY_BOARD=GENERIC -B build-GENERIC  build
make: idf.py: Command not found
Makefile:34: recipe for target 'all' failed
make: *** [all] Error 127

I just don't understand that so many people are able to 'just' freeze a module, while I've been spending hours now trying to figure this all out lol.. where do people learn this kind of stuff? I've no clue what I'm doing sometimes..

Any help (and explanation) on this whole process would be appreciated!


Edit: seems I had to add the idf.py to my path following: https://docs.espressif.com/projects/esp ... ofile.html I've done this now, which results in the following error:

Code: Select all

idf.py -D MICROPY_BOARD=GENERIC -B build-GENERIC  build
Traceback (most recent call last):
  File "/home/peugot206/esp/esp-idf/tools/idf.py", line 45, in <module>
    from idf_py_actions.tools import (executable_exists, idf_version, merge_action_lists, realpath)  # noqa: E402
  File "/home/peugot206/esp/esp-idf/tools/idf_py_actions/tools.py", line 1, in <module>
    import click
ImportError: No module named 'click'
Makefile:34: recipe for target 'all' failed
make: *** [all] Error 1

Edit 2: Seems this is because the ESP-IDF tools are not setup correctly due to the Python version not being >3.7.. I've been trying to follow all kinds of tutorial to update python from 3.5 to 3.7: https://hackthestuff.com/article/how-to ... -in-ubuntu, however I keep getting

Code: Select all

Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package python3.7
E: Couldn't find any package by glob 'python3.7'
E: Couldn't find any package by regex 'python3.7'

Once I have python 3.7 I need to re-run this setup ESP tools part again, I think: `cd ~/esp/esp-idf` and `./install.sh esp32`(https://docs.espressif.com/projects/esp ... -the-tools)

[davef]

esp-open-sdk toolchain is just for the ESP8266. I tried to do that for ESP8266 and failed. The ESP8266 and the ESP32 use different processors and therefore different toolchains.

For the ESP32 I thought the link I copied would show how to do it or most of it.

Have you got folder called <esp> in /home? And in that esp-idf? Did you run install.sh?

Now, running Linux as a VM I seem recall causes further complications, could be wrong.

Another thing with Linux is that you often need <sudo> in front of the command.

For the ESP8266 build environment you do have to re-run every time you upgrade Linux. Don't know about the ESP32.

Not being able to upgrade Python is a show-stopper. Can you upgrade anything? What happens if on the command line you put:

Code: Select all

sudo apt update

[davef]

Standard Setup of Toolchain for Linux - ... ation.html

[davef]

Looks like I used this the last time I built for the ESP32, probably need to change the 4.2.2

cd
rm -rf micropython
rm -rf esp
git clone https://github.com/micropython/micropython.git
mkdir esp
cd esp
git clone -b v4.2.2 https://github.com/espressif/esp-idf.git
cd esp-idf
./install.sh
. ./export.sh
cd
cd micropython
make -C mpy-cross
cd ports/esp32
make clean
make submodules
make

The . ./export.sh maybe should be .. ./export.sh, ie go back one directory

[Peugot206]

Have you got folder called <esp> in /home? And in that esp-idf? Did you run install.sh?

Yes I have, this should be the correct toolchain for the ESP32.

Not being able to upgrade Python is a show-stopper. Can you upgrade anything? What happens if on the command line you put:

Yes, updating works fine

Code: Select all

Reading package lists... Done
Building dependency tree       
Reading state information... Done
138 packages can be upgraded. Run 'apt list --upgradable' to see them.

It's just Python 3.7 that won't install with the error messages in my previous message.

Standard Setup of Toolchain for Linux - ... ation.html

This URL returns 'not found 404'.


Thanks for the detailed steps you provide! I'm following them and at the step where I need to run

Code: Select all

./install.sh

I get the following error (the message I've been stuck on all the time too):

Code: Select all

Creating a new Python environment in /home/peugot206/.espressif/python_env/idf4.2_py3.5_env
Traceback (most recent call last):
  File "/home/peugot206/esp/esp-idf/tools/idf_tools.py", line 1524, in <module>
    main(sys.argv[1:])
  File "/home/peugot206/esp/esp-idf/tools/idf_tools.py", line 1520, in main
    action_func(args)
  File "/home/peugot206/esp/esp-idf/tools/idf_tools.py", line 1233, in action_install_python_env
    import virtualenv   # noqa: F401
  File "/home/peugot206/.local/lib/python3.5/site-packages/virtualenv/__init__.py", line 3, in <module>
    from .run import cli_run, session_via_cli
  File "/home/peugot206/.local/lib/python3.5/site-packages/virtualenv/run/__init__.py", line 7, in <module>
    from ..app_data import make_app_data
  File "/home/peugot206/.local/lib/python3.5/site-packages/virtualenv/app_data/__init__.py", line 9, in <module>
    from platformdirs import user_data_dir
  File "/home/peugot206/.local/lib/python3.5/site-packages/platformdirs/__init__.py", line 5
    from __future__ import annotations
SyntaxError: future feature annotations is not defined

Which most likely has to do with my Python version not being up to date.. but whatever I try, I can not update my Python version!

[davef]

Sorry it wasn't a link:
https://docs.espressif.com/projects/esp ... setup.html
but don't think it is going to help you with your problem.

Can't help you with your Python issue.