Native C module: accessing objects. [SOLVED]

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
User avatar
pythoncoder
Posts: 4662
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Native C module: accessing objects. [SOLVED]

Post by pythoncoder » Fri Sep 04, 2020 3:23 pm

Some seriously n00b questions about writing native C modules.

My planned native .mpy module exports a single function accessible from Python. The function args include two Framebuf instances.

I've studied the docs and built and tested the examples in natmod, but I'm unclear about accessing Framebuf methods from C. Is there any documentation I've missed or example code which might get me on the right track?

In particular some of the methods I need to access are declared as static inline:

Code: Select all

static inline void setpixel(const mp_obj_framebuf_t *fb, int x, int y, uint32_t col) {
    formats[fb->format].setpixel(fb, x, y, col);
}
My C is decidedly rusty, but I don't think the linker can cope with those. The functions pointed to by the LUT are declared as STATIC - what are the implications here?

How to proceed?
Peter Hinch

User avatar
deshipu
Posts: 1380
Joined: Thu May 28, 2015 5:54 pm

Re: Native C module: accessing objects

Post by deshipu » Sat Sep 05, 2020 9:47 am

I'm afraid you can't do that. The only MicroPython C functions you can call from the inside of native modules are the ones that are explicitly listed in the mp_fun_table.

Christian Walther
Posts: 121
Joined: Fri Aug 19, 2016 11:55 am

Re: Native C module: accessing objects

Post by Christian Walther » Sat Sep 05, 2020 2:00 pm

I have never done any of this, so I’m guessing here, but what I gather from looking at the code is:

The setpixel function you quote is an implementation detail of the framebuf module, you can’t call it from outside (not even in C code that is part of the core). It is not part of the API of the FrameBuffer class. However, it is essentially the implementation of the FrameBuffer.pixel() Python method (framebuf_pixel() in modframebuf.c), and you should be able to call that the same way as you would call any method of a Python object, whether implemented in C or Python. I don’t know offhand how to do that, but there are some suspiciously-looking functions like mp_load_method() and mp_call_method_n_kw() that are available in mp_fun_table.

In other words, you use the C-to-Python-call machinery to do a call that then immediately goes back into C through the Python-to-C-call machinery. I don’t think there is a more direct way than that in this case, without making assumptions about the internal structure of a framebuffer object and replicating the needed parts of modframebuf.c in your module. (For other cases like file-like objects (IOBase) I believe something of that sort exists, but it needs to be explicitly provided in the base class.) The advantage is that your code can then also deal with objects that are not instances of the FrameBuffer class, but just implement the same interface.

(Hopefully the experts will correct me if my guesses are way off.)

User avatar
pythoncoder
Posts: 4662
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Native C module: accessing objects

Post by pythoncoder » Sun Sep 06, 2020 9:28 am

deshipu wrote:
Sat Sep 05, 2020 9:47 am
I'm afraid you can't do that. The only MicroPython C functions you can call from the inside of native modules are the ones that are explicitly listed in the mp_fun_table.
I feared as much :(
Peter Hinch

stijn
Posts: 507
Joined: Thu Apr 24, 2014 9:13 am

Re: Native C module: accessing objects

Post by stijn » Sun Sep 06, 2020 2:37 pm

deshipu wrote:
Sat Sep 05, 2020 9:47 am
I'm afraid you can't do that. The only MicroPython C functions you can call from the inside of native modules are the ones that are explicitly listed in the mp_fun_table.
Aren't there workarounds which allow to access 'MicroPython C' functions as long as they are discoverable in Python? That doesn't mean you can access the entire public API directly, but might give access to some functions of interest indirectly; and by going around via Python land so not ideal performance-wise. Didin't check this, but I'm thinking along the lines of using load_global/load_attr/load_method and the likes to get hold of a Pyton method/function then use mp_call_function_n_kw. So if that setpixel() method happens to be accessible (possibly indirectly) as a member of framebuf, it should be possible to call it once you have a framebuf object? Or perhaps I'm completely missing the point here.

User avatar
deshipu
Posts: 1380
Joined: Thu May 28, 2015 5:54 pm

Re: Native C module: accessing objects

Post by deshipu » Sun Sep 06, 2020 2:44 pm

You can do that, but you get a performance penalty, and I imagine that for a function like set_pixel that really matters.

In this particular case there is another possible solution: since framebuf is itself a native module, you can simply link against it — this will create a copy of that function in your binaries, but it should be working the same on the framebuf objects, at least as long as there isn't some fundamental refactoring of that module.

Christian Walther
Posts: 121
Joined: Fri Aug 19, 2016 11:55 am

Re: Native C module: accessing objects

Post by Christian Walther » Sun Sep 06, 2020 6:53 pm

stijn, I think that’s what I was suggesting, or am I misunderstanding you?
deshipu wrote:
Sun Sep 06, 2020 2:44 pm
since framebuf is itself a native module, you can simply link against it
That doesn’t work because the required functions are all static and not accessible to the linker. You’d need to use a modified version of the module with the “static” modifiers removed, which is essentially the same thing as what I meant by “replicating the needed parts of modframebuf.c in your module”.

User avatar
deshipu
Posts: 1380
Joined: Thu May 28, 2015 5:54 pm

Re: Native C module: accessing objects

Post by deshipu » Sun Sep 06, 2020 10:30 pm

You are right, at this point it's probably easiest to just copy the relevant functions over.

User avatar
pythoncoder
Posts: 4662
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Native C module: accessing objects

Post by pythoncoder » Mon Sep 07, 2020 8:07 am

I suspect this is all above my pay grade ;)

Perhaps it's worth my explaining what I was hoping to achieve. My font-to-py repo includes Writer and CWriter classes; these render fonts to a framebuf or to a display driver subclassed from Framebuf. Writer is for the case where the destination framebuf is monochrome and uses .blit: it is therefore fast. CWriter has to render pixel by pixel in Python: this is because .blit does not behave in a reasonable way when the source framebuf is monochrome and the destination is color.

This has been discussed on GitHub without discernible progress. A fully generic solution of blitting between arbitrary color spaces is difficult and certainly beyond me. I offered to produce a PR for a partial solution where the source is mono but this was not taken up.

I therefore wanted to write a loadable module which exported a function with this call signature:

Code: Select all

blit(source, dest, fgcolor, bgcolor)
I don't want users to have to compile MicroPython but providing an option to create a loadable module - and offering pre-built ones for popular archs - would be worthwhile.

These are the implied questions:
  • Is this fundamentally achievable?
  • If so is it achievable by me?
  • If so, how would I go about it?
I think I know sufficient C to write it, but I'll probably need help on the MicroPython interface and I'm rusty on the gory details of linking...
Peter Hinch

stijn
Posts: 507
Joined: Thu Apr 24, 2014 9:13 am

Re: Native C module: accessing objects

Post by stijn » Mon Sep 07, 2020 10:23 am

Christian Walther wrote:
Sun Sep 06, 2020 6:53 pm
stijn, I think that’s what I was suggesting, or am I misunderstanding you?
No you're right: I'm sorry but I completely missed your post and basically wrote the exact same thing as you did :?

Post Reply