General Question

[Firas_Baccouri]

Hello Guys,
There are something that still not totally obvious in my mind , and I really want to make my mind about them

1 :When I use putty or any other software for serial communication and I send a line of python code and send it to the controller to be running there , what exactly happen in the low layer , how this code is changed to ASCII code to be sent on the serial comm and then to be running on the controller

2:When I have implemented a C module and then I have used this module from python , it worked smoothly and very well , but what I didn´t understand is :
*When I import that module from my python side what happen in the middle and low layer how could that module be imported while the IDE still not recognising this module (he still show the red line under it , to show that he don´t know that module )
*And then when I use one of the function of that module (of course the function that I configured from the C side to be running from python side I mean I configure it to take python datatype as a parameters and return python datatypes ) how this function is called ? I know that I should use this function STATIC MP_DEFINE_CONST_FUN_OBJ_X( ) (1*)and put it inside that table mp_rom_map_elem_t mp_module_subsystem_globals_table[] , but I just followed the example I don´t know what (1) do and what the table do .

3: Is micropython can be used in a real projects , I am kind of afraid about security , is there any solution to secure the mpy code running on the customer board ? , For example could I encrypt the micropython code and then decrypt it when it is on the controller to let it run . For encrypt and decrypt the micropython code I think it is possible while when we send this micropython from a standalone python software it is treated as a string so we can encrypt it , but I think the problem still with the decrypted how is that can be done , to let this micropython code run on the controller ?

I really want to make my mind about all of that to get the whole picture .
Thank you in advance .

[karfas]

1) What happens...
Much to broad question. Putty sends each typed character to the UART on the controller. There, the "firmware" (e.g. micropyhon's REPL) does something with it.
Between putty and the UART may be: your computer's OS, a USB driver, a cable, a USB/serial chip on the controller and a myriad of electrical and physical details nobody doing "software" cares about.
For most practical purposes, you might call all this either "technology" or "magic" (if you prefer).
2a) Your IDE is not the python interpreter. It's only a colorful version of notepad and most likely requires special treatment to recognize your native functions.
2b) ... "how this function gets called"
In short (and simplified): The macro adds address and name of your C function to a global table. Whenever the interpreter looks for a function to execute, he looks (also) into this table and performs a call to the address associated with the function name.

[karfas]

Too much simplified, of course.
The MP_DEFINE.. macro creates the C representation of a micropython function object (name, C function address).
Then you add this object to one of the tables searched by the interpreter.

[jimmo]

I think karfas has covered the first two questions. Please reply with more specific questions if you'd like to know more.

3: Is micropython can be used in a real projects , I am kind of afraid about security , is there any solution to secure the mpy code running on the customer board ? , For example could I encrypt the micropython code and then decrypt it when it is on the controller to let it run . For encrypt and decrypt the micropython code I think it is possible while when we send this micropython from a standalone python software it is treated as a string so we can encrypt it , but I think the problem still with the decrypted how is that can be done , to let this micropython code run on the controller ?

This has been discussed many times on this forum. MicroPython is used for a wide range of real projects and in commercial products.

If you plan to make a shipping product then you should definitely be freezing the bytecode into the firmware (rather than just putting .py files on the filesystem). This not only makes it harder to obtain the code, but is more RAM efficient.

The basic answer is that it totally depends on how advanced your 'attacker' is, and how much of an issue it is if they have access to your compiled bytecode. There are ways to disassemble the bytecode, but not back into Python code. (It's not much different to being able to disassemble the thumb/ARM native code for a compiled application).

It also depends very much which microcontroller you're on. On something like the Pico/rp2040, where all the code is in external flash, then your decryption algorithm is also able to be reverse engineered. To support this use case you need hardware support for encrypted flash. (The ESP32 has this for example but we do not support it from MicroPython).

On stm32, if you're using internal flash then you can lock it (which prevents the flash being accessed externally over SWD or DFU). But then you also need to ensure there's no way someone can access the REPL (which is fine, this can be disabled).

[martincho]

If you plan to make a shipping product then you should definitely be freezing the bytecode into the firmware (rather than just putting .py files on the filesystem). This not only makes it harder to obtain the code, but is more RAM efficient.

I haven't thought this through or done any research at this stage. My first thought when it comes to freezing to bytecode is: How do you implement firmware updates?

Context is important: Your device is in a black box at the top of a radio tower. No access other than a serial port cable that you can connect to on the ground. No access to the REPL. And it isn't you installing the update. It's your customer, who is only good for downloading a file and double-clicking it.

With Python files in the MP filesystem there are a number of options. Not sure what one might do with bytecode. I think it would require writing code in MicroPython to replace the UF2 or point the bootloader to an alternative UFS. Not sure that's possible with stock MP.

[jimmo]

I haven't thought this through or done any research at this stage. My first thought when it comes to freezing to bytecode is: How do you implement firmware updates?

Yep, absolutely. It's very difficult to design a one-size-fits-all approach here, but (on my giant TODO list) is a guide to the considerations for each of the ports we support.

For example on STM32, with the MicroPython bootloader (mboot) we support loading a new firmware image from a compressed .gz file on the filesystem. That works well with frozen code (I have worked on MicroPython-based systems that use this exact mechanism and load new firmware OTA via BLE).

On ESP32, they have an OTA feature. I don't know much about this but it does work.

The RP2 is relatively more limited in this regard.

Also the other option is to just use .mpy files rather than freezing. This provides the same security benefit (to the OP's question) but without the RAM benefit, and is much easier to update (and even do A/B updates). We are very close to supporting loading .mpy directly from the filesystem without any RAM cost too, and this will make .mpy files almost equivalent to freezing. See https://github.com/micropython/micropython/pull/8381

[Firas_Baccouri]

Too much simplified, of course.
The MP_DEFINE.. macro creates the C representation of a micropython function object (name, C function address).
Then you add this object to one of the tables searched by the interpreter.

Thank you Karfas for both of your answers , it looks quite obvious now .

[Firas_Baccouri]

If you plan to make a shipping product then you should definitely be freezing the bytecode into the firmware (rather than just putting .py files on the filesystem). This not only makes it harder to obtain the code, but is more RAM efficient.

On stm32, if you're using internal flash then you can lock it (which prevents the flash being accessed externally over SWD or DFU). But then you also need to ensure there's no way someone can access the REPL (which is fine, this can be disabled).

What I have understood from that are , to have satisfy the security of the code written on the controller there are two ways :
1 : Freezing the bytecode into the firmware
Or
2 : Locking the flash memory (I am working on stm32f4 controller and using the internal flash ) and disable the REPL .

For the first solution , is there a documentation about how could I achieve that .
For the second point I will check how can I lock the flash memory , and for disabling the REPL I think it is possible to do it whether from the python code or disabling it from the C side from the main function .
Am I true ?? Or there are a lot of mistakes on what I have said ?

[jimmo]

What I have understood from that are , to have satisfy the security of the code written on the controller there are two ways :

It's not really either/or. You need to understand what you're protecting against.

1. Freezing (or compiling to .mpy) will prevent your raw Python code from being on the device. An attacker can only get the raw bytecode, which will either be in the filesystem (.mpy) or in the frozen code section.

2. Locking the flash will prevent someone using hardware tools (i.e. SWD, JTAG, DFU) to access the contents of flash.

3. Disabling the REPL will prevent someone using MicroPython itself to access the flash. i.e. if I can execute code on the device at the REPL, then hardware flash protection can be trivially bypassed by just using machine.mem8 to read the flash memory.

For the second point I will check how can I lock the flash memory

Consult the ST docs.

and for disabling the REPL I think it is possible to do it whether from the python code or disabling it from the C side from the main function .

You can kind of do this from Python, but safer to just remove it altogether in the C code.

Another options is to remove any functionality that allows arbritrary memory access (e.g. machine.mem*).

But again... I would seriously question how valuable this is and whether it's worth the additional cost and complexity to your development. What are you really protecting against?

[Firas_Baccouri]

But again... I would seriously question how valuable this is and whether it's worth the additional cost and complexity to your development. What are you really protecting against?

I want to prevent the micropython code running on the controller from any attacker . This is my main goal .