How can I manually add a C module without micropython.mk?

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
cduran
Posts: 54
Joined: Thu Mar 17, 2016 4:52 pm

How can I manually add a C module without micropython.mk?

Post by cduran » Tue Jan 21, 2020 4:09 pm

I currently have a very mature code base that I have added Micropython to. Now I want to add a C module but I don't want to tinker with the existing makefile. The C module will be very complex and interconnected with the existing firmware so building it as a separate file and building mp on its own to import it is no feasible (at least I don't see it as such).

Any ideas on how I can do this?

User avatar
jimmo
Posts: 1189
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How can I manually add a C module without micropython.mk?

Post by jimmo » Wed Jan 22, 2020 12:25 am

This sounds like it might require a lot more knowledge of your codebase to answer.

But when you say "I don't want to tinker with the existing makefile" -- is that the existing makefile for your codebase, or for MicroPython's makefile. If it's the former, surely you had to modify your makefile to add MicroPython?

Anyway, adding an additional module requires four things to happen:
- The file (modfoo.c) needs to be found by the QSTR extraction process (technically optional if you want to manage the QSTRs manually).
- The file just needs to be part of the build (i.e. modfoo.o) needs to be linked into the final firmware.
- Your module needs to be added to the list of built-in modules. This is probably best achieved with MP_REGISTER_MODULE combined with py/makemoduledefs.py (i.e. makemoduledefs.py needs to know where to find your modfoo.c).
- You need the right include paths.

cduran
Posts: 54
Joined: Thu Mar 17, 2016 4:52 pm

Re: How can I manually add a C module without micropython.mk?

Post by cduran » Wed Jan 22, 2020 4:01 pm

jimmo wrote:
Wed Jan 22, 2020 12:25 am
This sounds like it might require a lot more knowledge of your codebase to answer.

But when you say "I don't want to tinker with the existing makefile" -- is that the existing makefile for your codebase, or for MicroPython's makefile. If it's the former, surely you had to modify your makefile to add MicroPython?
I don't want to tinker too much with my codes makefile. At least no more than just adding the files I need to build.

User avatar
jimmo
Posts: 1189
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How can I manually add a C module without micropython.mk?

Post by jimmo » Thu Jan 23, 2020 1:42 am

cduran wrote:
Wed Jan 22, 2020 4:01 pm
I don't want to tinker too much with my codes makefile. At least no more than just adding the files I need to build.
How did you integrate MicroPython core then? Do you separately build libmicropython (i.e. like the examples/embeddeding example) and link against that?

cduran
Posts: 54
Joined: Thu Mar 17, 2016 4:52 pm

Re: How can I manually add a C module without micropython.mk?

Post by cduran » Tue Jan 28, 2020 3:31 pm

jimmo wrote:
Thu Jan 23, 2020 1:42 am
cduran wrote:
Wed Jan 22, 2020 4:01 pm
I don't want to tinker too much with my codes makefile. At least no more than just adding the files I need to build.
How did you integrate MicroPython core then? Do you separately build libmicropython (i.e. like the examples/embeddeding example) and link against that?
I include all of MP's files into our project and build. MP isn't built separately, which is important in our case. This firmware runs on various platforms (ranging from bare metal MCU to Linux based) and new ones added in the future, so having to build MP separately just add complexity to porting efforts.

User avatar
jimmo
Posts: 1189
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How can I manually add a C module without micropython.mk?

Post by jimmo » Tue Jan 28, 2020 11:36 pm

cduran wrote:
Tue Jan 28, 2020 3:31 pm
I include all of MP's files into our project and build. MP isn't built separately, which is important in our case. This firmware runs on various platforms (ranging from bare metal MCU to Linux based) and new ones added in the future, so having to build MP separately just add complexity to porting efforts.
Oh, cool. In that case then what you want to do should be relatively straightforward -- you can treat your custom module just like any of the built-in ones, and register it in mpconfigport.h in MICROPY_PORT_BUILTIN_MODULES and included it in the list of C sources. (That way you don't need to worry about any of the magic that micropython.mk does -- as long as it's in the C sources it'll be found for QSTR extraction, and the only file you need to modify is mpconfigport.h, which you had to provide anyway).

cduran
Posts: 54
Joined: Thu Mar 17, 2016 4:52 pm

Re: How can I manually add a C module without micropython.mk?

Post by cduran » Wed Jan 29, 2020 9:33 pm

jimmo wrote:
Tue Jan 28, 2020 11:36 pm

Oh, cool. In that case then what you want to do should be relatively straightforward -- you can treat your custom module just like any of the built-in ones, and register it in mpconfigport.h in MICROPY_PORT_BUILTIN_MODULES and included it in the list of C sources. (That way you don't need to worry about any of the magic that micropython.mk does -- as long as it's in the C sources it'll be found for QSTR extraction, and the only file you need to modify is mpconfigport.h, which you had to provide anyway).
Is there documentation somewhere that shows this? I'm using the minimal build, where can I find examples of how this looks.

User avatar
jimmo
Posts: 1189
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How can I manually add a C module without micropython.mk?

Post by jimmo » Sun Feb 02, 2020 10:24 pm

cduran wrote:
Wed Jan 29, 2020 9:33 pm
Is there documentation somewhere that shows this? I'm using the minimal build, where can I find examples of how this looks.
Not that I'm aware of, but take a look at ports/stm32/mpconfigport.h MICROPY_PORT_BUILTIN_MODULES

cduran
Posts: 54
Joined: Thu Mar 17, 2016 4:52 pm

Re: How can I manually add a C module without micropython.mk?

Post by cduran » Mon Feb 24, 2020 6:43 pm

cduran wrote:
Wed Jan 29, 2020 9:33 pm
jimmo wrote:
Tue Jan 28, 2020 11:36 pm

Oh, cool. In that case then what you want to do should be relatively straightforward -- you can treat your custom module just like any of the built-in ones, and register it in mpconfigport.h in MICROPY_PORT_BUILTIN_MODULES and included it in the list of C sources. (That way you don't need to worry about any of the magic that micropython.mk does -- as long as it's in the C sources it'll be found for QSTR extraction, and the only file you need to modify is mpconfigport.h, which you had to provide anyway).
Is there documentation somewhere that shows this? I'm using the minimal build, where can I find examples of how this looks.
So I tried my best to follow stm32 port as an example, but it still seems that I am missing a step in the build.

#define MICROPY_PORT_BUILTIN_MODULES \
{ MP_ROM_QSTR(MP_QSTR_c_sample), MP_ROM_PTR(&c_sample_user_cmodule) },

This line is giving me a build error 'MP_QSTR_c_sample' undeclared here, when I compare to the untouched stm32 port this line:

{ MP_ROM_QSTR(MP_QSTR_pyb), MP_ROM_PTR(&pyb_module) },

'MP_QSTR_pyb' is not declared anywhere in the project so why would I get an undeclared for mine, unless there is something missing in the build process.

User avatar
jimmo
Posts: 1189
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How can I manually add a C module without micropython.mk?

Post by jimmo » Tue Feb 25, 2020 3:38 am

cduran wrote:
Mon Feb 24, 2020 6:43 pm
'MP_QSTR_pyb' is not declared anywhere in the project so why would I get an undeclared for mine, unless there is something missing in the build process.
Here's a guide on how this works in the regular build -- you might need to adapt this to your build. http://docs.micropython.org/en/latest/develop/qstr.html

Post Reply