How do I look up the implementation of a MicroPython function?

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
appels
Posts: 10
Joined: Thu Dec 03, 2020 3:14 pm

How do I look up the implementation of a MicroPython function?

Post by appels » Thu Dec 31, 2020 3:37 pm

Hi,

I'm wondering how I can look up how MicroPython functions are implemented?
If I'm not mistaking MicroPython is written in C so I would assume the different MicroPython functions have som kind of C implementation.
But after browsing the MicroPython github repository I can't seem to find what I'm looking for.

Say I have a Pyboard and I run a script that performs the machine.deepsleep() function, how is this implemented in C?
What C functions are executed and how can I find this out?

Or is it a wrong asumption that MicroPython functions all have some specific C code to implement them?

Thanks in advance!

stijn
Posts: 588
Joined: Thu Apr 24, 2014 9:13 am

Re: How do I look up the implementation of a MicroPython function?

Post by stijn » Thu Dec 31, 2020 4:47 pm

The principle for figuring out is a plain text search (find ... | xargs grep ... or from a text editor supportig that) on the function name in all .c and .py (theoretically also .cpp but currently not as far as I'm aware) on the complete source code. There are shortcuts, like searching only in the directory of the port you're using but then you risk not finding anything if the function is generic. If you also have things installed with pip or so you might also have to search those directories.

By example: if you're using esp32 you'll see there's a machine_deepsleep function in a file modmachine.c, statically stored in a MicroPython function object machine_deepsleep_obj, which in turn is added to the machine module globals machine_module_globals_table. So you can be failry certain that's the one. As you can see another shortcut would be to search for MP_QSTR_deepsleep directly, assuming that the function you want is indeed implemented in C and is statically registered in a module implemented in C.
appels wrote:
Thu Dec 31, 2020 3:37 pm
Or is it a wrong asumption that MicroPython functions all have some specific C code to implement them?
Assumptions are by definition sort of wrong :) But in this case: yes the majority or maybe even all core functions are implemented in C.

Online
User avatar
pythoncoder
Posts: 5066
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: How do I look up the implementation of a MicroPython function?

Post by pythoncoder » Sat Jan 02, 2021 5:56 am

Learning your way round the source takes some time but is well worthwhile. The way I find code is

Code: Select all

$ grep --include=\*.c -rnw '.' -e 'machine_deepsleep'
./ports/esp32/modmachine.c:143:STATIC mp_obj_t machine_deepsleep(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) {
./ports/esp32/modmachine.c:146:STATIC MP_DEFINE_CONST_FUN_OBJ_KW(machine_deepsleep_obj, 0,  machine_deepsleep);
./ports/esp8266/modmachine.c:130:STATIC mp_obj_t machine_deepsleep(size_t n_args, const mp_obj_t *args) {
./ports/esp8266/modmachine.c:176:STATIC MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(machine_deepsleep_obj, 0, 1, machine_deepsleep);
./ports/nrf/modules/machine/modmachine.c:160:STATIC mp_obj_t machine_deepsleep(void) {
./ports/nrf/modules/machine/modmachine.c:164:MP_DEFINE_CONST_FUN_OBJ_0(machine_deepsleep_obj, machine_deepsleep);
./ports/stm32/modmachine.c:369:STATIC mp_obj_t machine_deepsleep(size_t n_args, const mp_obj_t *args) {
./ports/stm32/modmachine.c:377:MP_DEFINE_CONST_FUN_OBJ_VAR_BETWEEN(machine_deepsleep_obj, 0, 1, machine_deepsleep);
./ports/cc3200/mods/modmachine.c:147:STATIC mp_obj_t machine_deepsleep (void) {
./ports/cc3200/mods/modmachine.c:151:STATIC MP_DEFINE_CONST_FUN_OBJ_0(machine_deepsleep_obj, machine_deepsleep);
[adminpete@capybara]: /mnt/qnap2/data/Projects/MicroPython/micropython
which throws up the implementation for each port.
Peter Hinch

User avatar
jimmo
Posts: 2262
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How do I look up the implementation of a MicroPython function?

Post by jimmo » Mon Jan 11, 2021 12:25 am

One handy "trick" is that any identifier accessible from Python (i.e. a method name, class name, global variable, kwarg, etc) must have a QSTR assigned for it. You don't need to know the details of how QSTRs work, but in general if there's a "deepsleep" then somewhere in the code there _must_ be a MP_QSTR_deepsleep.

So I typically just do a global search for QSTR_foo whenever I'm trying to find anything. In this case, you get one result for each port that implements machine.deepsleep, adding it to the machine module. And then the "machine_deepsleep_obj" will take you to the relevant C implementation, almost always in the same file.

Code: Select all

$ ag MP_QSTR_deepsleep
ports/esp32/modmachine.c
236:    { MP_ROM_QSTR(MP_QSTR_deepsleep), MP_ROM_PTR(&machine_deepsleep_obj) },

ports/stm32/modmachine.c
400:    { MP_ROM_QSTR(MP_QSTR_deepsleep),           MP_ROM_PTR(&machine_deepsleep_obj) },

ports/cc3200/mods/modmachine.c
177:    { MP_ROM_QSTR(MP_QSTR_deepsleep),           MP_ROM_PTR(&machine_deepsleep_obj) },

ports/esp8266/modmachine.c
409:    { MP_ROM_QSTR(MP_QSTR_deepsleep), MP_ROM_PTR(&machine_deepsleep_obj) },

ports/esp8266/modesp.c
359:    { MP_ROM_QSTR(MP_QSTR_deepsleep), MP_ROM_PTR(&esp_deepsleep_obj) },

ports/nrf/modules/machine/modmachine.c
201:    { MP_ROM_QSTR(MP_QSTR_deepsleep),          MP_ROM_PTR(&machine_deepsleep_obj) },

stijn
Posts: 588
Joined: Thu Apr 24, 2014 9:13 am

Re: How do I look up the implementation of a MicroPython function?

Post by stijn » Mon Jan 11, 2021 7:22 am

in general if there's a "deepsleep" then somewhere in the code there _must_ be a MP_QSTR_deepsleep
As far as I'm aware this is currently indeed the case for anything in the MicroPython core, likely the complete repository (?), but strictly speaking what must be there is a qstr representation of "deepsleep", but that is not necessarily MP_QSTR_deepsleep: it can aso be the result of qstr_from_str("deepspleep") for instance.
The core code uses static objects for registering methods and functions so in those case it has to use a 'static' definition i.e. prefix with MP_QSTR_, and in all other cases like attr() function implementations, it also prefers using MP_QSTR_ macros (also because that is faster than calling qsrt_from_str a runtime). But other codebases might not do that, hence the note here: people can definitely use MP_QSTR_ as a shortcut, but should be aware what's going on if they don't find anything with it.

User avatar
jimmo
Posts: 2262
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: How do I look up the implementation of a MicroPython function?

Post by jimmo » Mon Jan 11, 2021 11:34 pm

stijn wrote:
Mon Jan 11, 2021 7:22 am
it can aso be the result of qstr_from_str("deepspleep") for instance.
Yep, definitely true, and good to point out the limitiations/surprises.

It's not just faster, it's more space efficient, and in any place where it's a compile-time definition of a module or a locals dict, then it's required. In general, I'd consider any place a literal quoted string is used that isn't an error message (i.e. wrapped with MP_ERROR_TEXT) would be a sign that something weird is happening.

For example... utils/pyexec.c does qstr_from_str("LCD") which I'm pretty sure should be MP_QSTR_LCD (and other things in this file use MP_QSTR_).

stijn
Posts: 588
Joined: Thu Apr 24, 2014 9:13 am

Re: How do I look up the implementation of a MicroPython function?

Post by stijn » Tue Jan 12, 2021 7:26 am

Yeah I also noticed. And one other place as well, will PR that.

Post Reply