Example for working native mpy module with SPI communication?

[Frans]

I'm currently trying to provide an it8951 driver as a native MicroPython module but it looks like I've underestimated the effort. So I wonder whether I'm on the right track anyway. Can you have a look and give me some feedback about existing projects or bad approaches?

[*] Goal: have a IT9851 e-Paper driver for the M5Paper (communicating via SPI) (M5Paper has 8MB of SPI_FLASH)
[*] My approach: provide a native module (as in https://docs.micropython.org/en/latest/ ... atmod.html)
[*] Alternative1: provide a built-in module similar to https://github.com/russhughes/st7789_mpy
(that seems to be not very flexible and I have to build/flash the firmware for each try)
[*] Alternative2: Provide a pure Python module, but that would be slow, and quite memory inefficient

Currently I'm taking examples from micropython/examples/natmod and try to make them SPI capable but that somehow feels like the wrong way. It looks like in order to be able to import "extmod/machine_spi.h" I have to fiddle with dozens of include directories in the Makefile, and currently I'm facing a compiler error that doesn't make sense to me:

Code: Select all

 _Static_assert(portGET_ARGUMENT_COUNT() == 0, "portGET_ARGUMENT_COUNT() result does not match for 0 arguments");

Do I have to build MicroPython for that first? I've read that those modules have to be SPI_FLASH aware, is that true?

Can someone provide an example for a working native module that deals with SPI already?

Anyway, my next step would be to take code from one of these projects:
* https://github.com/lovyan03/LovyanGFX/t ... x/v0/panel
* https://github.com/GregDMeyer/IT8951

.. and bring the code to the new module.

How would you do that? Is there a Micropython driver for IT9851 already?

Here is my current Makefile / source file:

Code: Select all

MPY_DIR = ../micropython
ISPIDF_DIR = ../esp-idf
MOD = it8951
SRC = it8951.c
ARCH = xtensawin

CFLAGS += -I$(MPY_DIR)/ports/esp32
CFLAGS += -I$(MPY_DIR)/ports/esp32/build-GENERIC/config
CFLAGS += -I$(ISPIDF_DIR)/components/xtensa/include
CFLAGS += -I$(ISPIDF_DIR)/components/xtensa/esp32/include
CFLAGS += -I$(ISPIDF_DIR)/components/freertos/include
CFLAGS += -I$(ISPIDF_DIR)/components/freertos/port/xtensa/include
CFLAGS += -I$(ISPIDF_DIR)/components/freertos/include/esp_additions/freertos
CFLAGS += -I$(ISPIDF_DIR)/components/esp_common/include
CFLAGS += -I$(ISPIDF_DIR)/components/esp_rom/include
CFLAGS += -I$(ISPIDF_DIR)/components/esp_system/include
CFLAGS += -I$(ISPIDF_DIR)/components/esp_hw_support/include
CFLAGS += -I$(ISPIDF_DIR)/components/soc/esp32/include
CFLAGS += -I$(ISPIDF_DIR)/components/soc/include
CFLAGS += -I$(ISPIDF_DIR)/components/hal/include
CFLAGS += -I$(ISPIDF_DIR)/components/hal/esp32/include
CFLAGS += -I$(ISPIDF_DIR)/components/esp_timer/include
CFLAGS += -I$(ISPIDF_DIR)/components/newlib/platform_include
CFLAGS += -I$(ISPIDF_DIR)/components/heap/include
CFLAGS += -I$(ISPIDF_DIR)/components/driver/include

include $(MPY_DIR)/py/dynruntime.mk

Code: Select all

#include "py/dynruntime.h"

#include <stdlib.h>
#include <math.h>
#include <stdio.h>
#include <string.h>
#include "py/obj.h"
#include "py/objstr.h"
#include "py/objmodule.h"
#include "py/runtime.h"
#include "py/builtin.h"
#include "py/mphal.h"
#include "extmod/machine_spi.h"


// THE CODE BELOW IS JUST THE factorial EXAMPLE

// Helper function to compute factorial
STATIC mp_int_t factorial_helper(mp_int_t x) {
    if (x == 0) {
        return 1;
    }
    return x * factorial_helper(x - 1);
}

// This is the function which will be called from Python, as factorial(x)
STATIC mp_obj_t factorial(mp_obj_t x_obj) {
    // Extract the integer from the MicroPython input object
    mp_int_t x = mp_obj_get_int(x_obj);
    // Calculate the factorial
    mp_int_t result = factorial_helper(x);
    // Convert the result to a MicroPython integer object and return it
    return mp_obj_new_int(result);
}
// Define a Python reference to the function above
STATIC MP_DEFINE_CONST_FUN_OBJ_1(factorial_obj, factorial);

// This is the entry point and is called when the module is imported
mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) {
    // This must be first, it sets up the globals dict and other things
    MP_DYNRUNTIME_INIT_ENTRY

    // Make the function available in the module's namespace
    mp_store_global(MP_QSTR_factorial, MP_OBJ_FROM_PTR(&factorial_obj));

    // This must be last, it restores the globals dict
    MP_DYNRUNTIME_INIT_EXIT
}

[jimmo]

Can you have a look and give me some feedback about existing projects or bad approaches?

The high-level answer is that dynamic native modules have a big limitation which is that they have a defined API to the main firmware that is hosting them. On a real OS, the linker/loader does all this for you via the symbol table etc.

This API just includes a minimal set of runtime functionality (e.g. making lists or invoking Python methods etc), see dynruntime.h. It does not include any port-specific or extmod functionality (e.g. the machine API).

The compromise (of sorts) is that you can use the dynruntime API to do almost everything you can do from Python -- e.g. look up a method on an object and invoke a method etc. So if you have a `mp_obj_t` for the machine.SPI instance, you can get its `write` method and invoke it, etc. It's obviously not as fast as just being able to call the underling C method directly, but still a lot faster than doing this in Python as the VM will not be involved.

[Frans]

Do you have a minimal example of how to provide the correct include paths and actual includes to do basic SPI communication? I tried to use the code @ https://github.com/russhughes/st7789_mpy as a source for example snippets but already failed to include "extmod/machine_spi.h"..

[jimmo]

Do you have a minimal example of how to provide the correct include paths and actual includes to do basic SPI communication? I tried to use the code @ https://github.com/russhughes/st7789_mpy as a source for example snippets but already failed to include "extmod/machine_spi.h"..

This repo is a user c module. I thought you were talking about a dynamic native module.

The approach I was describing would not involve including machine_spi.h -- you are treating the passed-in machine.SPI object as a generic Python method, querying for its write method, etc. What you get back is a "function object" (that you can then invoke with mp_call_function_n_kw).

[Frans]

Yes, https://github.com/russhughes/st7789_mpy indeed is a C module - I was hoping the code is not too different from a native module as in examples/natmod and it deals with SPI communication as I need it.
So my plan was to take one of the natmod examples and just see how include paths are being managed there and try to do what's needed to make

Code: Select all

#include "extmod/machine_spi.h"

work.

But maybe even that is a step in the wrong direction so I was hoping to find another "natmod" example which demonstrates the use of SPI.

Another approach would be to just turn the st7789 project into a it8951 driver, but this way every time I make changes to the driver I would have to re-flash the whole firmware and I'm stuck to a given set of modules AFAIK...

[jimmo]

I was hoping the code is not too different from a native module as in examples/natmod and it deals with SPI communication as I need it.

Unfortunately they're not really the same. A user c module is just a part of the build -- it has access to anything it likes. A native module must go via the runtime.

Here is an example of accessing Python objects from a native module. https://github.com/jimmo/micropython/bl ... uf_utils.c (via viewtopic.php?f=3&t=8998&p=51068)

The idea here is that we're accessing methods such as `.pixel()` from a FrameBuffer object.

[Frans]

So as far as I understand my current options would be to
* either turn the code from e.g. https://github.com/lovyan03/LovyanGFX/t ... x/v1/panel into a MicroPython C module as done in https://github.com/russhughes/st7789_mpy
* or to write a Python native driver for the it9851 and later (maybe even optionally) provide the number cruncher as dynamically loadable native module.

Do you see a problem with one of these? Apart from the obvious (like performance, effort to transform / translate the code) is there a preferable way?

[jimmo]

Do you see a problem with one of these? Apart from the obvious (like performance, effort to transform / translate the code) is there a preferable way?

In general the intent for the dynamic module feature is to write small pieces of code that need to be compiled because they're doing number crunching or data processing. i.e. it's kind of the next step up from using @native or @viper. The dynamic loader provides a standard set of functions that let you interact with the runtime enough to process arguments and generate return values, but the idea is that most of your code is just processing.

Generating a QR code was one of the original examples used to demo this -- all it does is take bytes in and send bytes out, but otherwise needs very little support from the runtime.

The obvious benefit of the dynamic modules is that you can deploy them without deploying the whole firmware.

On the other hand user c modules are much more flexible.

I don't really know enough about the display driver to know why this needs to be done like this? Can you use this display with framebuf.FrameBuffer? Or is the idea that the driver provides a lot more rendering functionality?

[Frans]

the it8951 is an e-paper driver for some waveshare displays as used in the M5Paper - I was hoping to be able to provide a memory and CPU efficient driver using those native modules without the need to compile the whole firmware every time.

But I think I got the idea now - I'll just do it in pure Python for now and later consider to provide an optional replacement for time and memory consuming operations.

Thanks for the insights!