MP_OBJ_NEW_QSTR

[pohmelie]

Trying to create my own module, but have problems with declaring(?) my module and it's functions
moddos.c

Code: Select all

#include "dos.h"

#include "py/runtime.h"


static mp_obj_t mod_dos_inportb(mp_obj_t address) {

    return mp_obj_new_int(inportb(mp_obj_get_int(address)));

}
static MP_DEFINE_CONST_FUN_OBJ_1(mod_dos_inportb_obj, mod_dos_inportb);

static const mp_map_elem_t mp_module_dos_globals_table[] = {
    {MP_OBJ_NEW_QSTR(MP_QSTR___name__), MP_OBJ_NEW_QSTR(MP_QSTR_dos) },
    {MP_OBJ_NEW_QSTR(MP_QSTR_inportb), (mp_obj_t)&mod_dos_inportb_obj },
};

STATIC MP_DEFINE_CONST_DICT(mp_module_dos_globals, mp_module_dos_globals_table);

const mp_obj_module_t mp_module_dos = {
    .base = { &mp_type_module },
    .name = MP_QSTR_dos,
    .globals = (mp_obj_dict_t*)&mp_module_dos_globals,
};

I took the pattern from https://github.com/micropython/micropyt ... /modtime.c
Also, I update mpconfigport.h with

Code: Select all

...
#endif

#define MICROPY_PY_DOS_DEF { MP_OBJ_NEW_QSTR(MP_QSTR_dos), (mp_obj_t)&mp_module_dos },

#define MICROPY_PORT_BUILTIN_MODULES \
    MICROPY_PY_TIME_DEF \
    MICROPY_PY_DOS_DEF \
    { MP_OBJ_NEW_QSTR(MP_QSTR__os), (mp_obj_t)&mp_module_os } \

...

And update SRC_MODE in Makefile with

Code: Select all

SRC_MOD += moddos.c

But I get compile error:

Code: Select all

In file included from ../py/mpstate.h:34:0,
                 from ../py/objmodule.c:30:
./mpconfigport.h:166:46: error: 'MP_QSTR_dos' undeclared here (not in a function)
 #define MICROPY_PY_DOS_DEF { MP_OBJ_NEW_QSTR(MP_QSTR_dos), (mp_obj_t)&mp_module_dos },
                                              ^
../py/obj.h:84:56: note: in definition of macro 'MP_OBJ_NEW_QSTR'
 #define MP_OBJ_NEW_QSTR(qst) ((mp_obj_t)((((mp_uint_t)(qst)) << 2) | 2))
                                                        ^
./mpconfigport.h:170:5: note: in expansion of macro 'MICROPY_PY_DOS_DEF'
     MICROPY_PY_DOS_DEF \
     ^
../py/objmodule.c:197:5: note: in expansion of macro 'MICROPY_PORT_BUILTIN_MODULES'
     MICROPY_PORT_BUILTIN_MODULES
     ^
make: *** [build/py/objmodule.o] Error 1

So, I do not get MP_OBJ_NEW_QSTR macros and why this works fine with modtime.c and does not work for me?

[pfalcon]

Grep codebase for where e.g. "_os" is defined, and you'll know what to do next.

[pohmelie]

pfalcon
I don't get... "_os" is in unix/qstrdefsport.h and other ports folder, but not in py/. In qstrdefsport.h there is some Q(...) lines. There is no "#define Q(" in project. What is that Q(...)?

[pohmelie]

Oh, got it. I must add names(?)
Q(dos)
Q(inportb)
...

This looks weird a bit.

[pfalcon]

The main point is that it took you 9 minutes to get it. And that's indeed one of the most not immediately obvious parts of MicroPython. But general, universal solution of grepping the codebase works here too (it works on any codebase!). Except that you need to figure that you need to grep not for complete identifier like MP_QSTR___name__, but just for part of it. Not a big problem for human mind .

As for "weird", you're welcome to understand completely how it works, and with fresh mind, you're welcome to propose how to improve it. For example, when I started with uPy hacking, what I wanted to do is effectively making an automated script which would scan for new symbols like MP_QSTR_foo and automatically add Q(foo) to qstrdefs.h . I never came to actually do that - because what we have already works pretty will. And I'm also a big fan of automation. That means I know its limitations and drawbacks. For example, the more you automate, the more havoc it causes when it breaks. And automation always breaks in real world. So, there's no need to automate trivial things unless you want to spend noticeable deal of your life maintaining that automation of trivial things.

Can't end this rant with anything else but https://github.com/NARKOZ/hacker-scripts

[pohmelie]

I always search in codebase but I can't find "MP_QSTR_sleep_ms" definition (for example) anywhere, so I go there.
As I can see all this preprocessing purpose is to generate some header with names, modules etc. In this case I like Lua style, where everything is explicit. I define a structure in my module (like in micropython), but it's already with real strings (names) of module objects. Then just "extern my_module_structure" and add it name to the Lua modules decription. As I realized, here target is "not to change any files in /py" so maybe this macros magic is good for building for different platforms, but definetly more complicated for developing.

[pfalcon]

It's needed to calculate efficient symbol lookup structures at compile time. Lua is nowhere near as efficient as MicroPython. You of course can do it in less efficient manner in MicroPython (with explicit strings), but inefficiency is not MicroPython's forte .

[pohmelie]

Lua is nowhere near as efficient as MicroPython

Not sure this right, I've just some test with unix build of micropython and lua 5.3.1. I took http://benchmarksgame.alioth.debian.org ... test=fasta and correct source to Lua 5.3 (cause it failed). Also I remove almost all output for both.

Code: Select all

[poh ~/pro/wrk/bench]$ time ./lua fasta.lua 250000
>ONE Homo sapiens alu
>TWO IUB ambiguity codes
>THREE Homo sapiens frequency

real    0m0.392s
user    0m0.387s
sys     0m0.004s
[poh ~/pro/wrk/bench]$ time ./micropython fasta.py 250000
b'>ONE Homo sapiens alu\n'
b'>TWO IUB ambiguity codes\n'
b'>THREE Homo sapiens frequency\n'

real    0m2.384s
user    0m2.374s
sys     0m0.008s

[dhylands]

Not sure if that benchmark is heap intensive or not. The unix version of micropython only runs with 2Mb heap.

You can use micropython -X heapsize=1000M to run with a larger heap.

[pfalcon]

Lua is nowhere near as efficient as MicroPython

Not sure this right, I've just some test with unix build of micropython and lua 5.3.1.

You mix up efficiency and speed. Efficiency is being able to do as much as possible with as little as possible resources. So, try running the code with 2K (kilobytes!) heap and report results. (Lua won't be able run with such heap at all.)

As for speed, in the nearby topic I showed a testcase where CPython is about 10000 times slower than MicroPython. Such testcases usually can be constructed for any language, and microbenchmark testing usually doesn't do much good. And again, MicroPython is not designed for raw speed, it's designed for as low as possible memory usage, and with such usage, to be as efficient as possible (don't spend more CPU cycles than needed). And it's well-known that memory efficiency and raw speed are on the opposite sides of spectrum. So we do what we can - we can't use speed-efficient, memory-bloated data structures, but we e.g. precompute hashes for constant data at runtime, unlike most other folks who waste runtime cycle and memory to compute and store them.