Micro Python development wish list

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: Micro Python development wish list

Post by Damien » Sat Dec 27, 2014 12:02 pm

One other thing that would be nice (directly related to embeddability) is to provide a "Micro Python amalgamation" ala sqlite. This is simply a concatenation of all uPy source files into one file (in the right order), plus a single header (and perhaps a single config file). This gives lowest barrier to entry using uPy, as users can very easily download and compile a single .C file, and it would provide them with a complete Python system.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Micro Python development wish list

Post by pfalcon » Sat Dec 27, 2014 1:45 pm

Yes, I thought about amalgamation some time ago too. I would then be satisfied by just ability to create one file which would #include *.c, but that's not possible without guarded includes. So, doing them first step makes sense. Then, just producing a static lib makes sense. Self-contained amalgamation would be the last step.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Micro Python development wish list

Post by pfalcon » Sat Dec 27, 2014 2:18 pm

Damien wrote:Okay, let's add loadable C modules as a 5th item.

There is actually already something we can use for this: the native emitter has a function table (see nativeglue.c) to "link" native code into the rest of the system.
Do you mean that as an example of approach, or to actually reuse emitter dispatch table as loadable module API? If you meant approach, that's what I meant either. The table should also contain API version number (and for a long time we'd need to check for "not equal", not "at least"), and good point that in uPy case, it'll need to contain a lot of data pointers in addition to functions. Then there would be defines like:

Code: Select all

#define mp_obj_is_true(o) APITABLE->_obj_is_true(o)
so a module can be both builtin and loadable.

I can point to https://github.com/pfalcon/squirrel-mod ... /binmodule as a complete example.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: Micro Python development wish list

Post by Damien » Sat Dec 27, 2014 4:55 pm

pfalcon wrote: Do you mean that as an example of approach, or to actually reuse emitter dispatch table as loadable module API?
I meant to actually reuse the native glue table, to save on code bytes. Of course we would need to add stuff (like you say the version number), but the majority of the calls already exist in this table, so why not use it?
I can point to https://github.com/pfalcon/squirrel-mod ... /binmodule as a complete example.
Yes, something like that, but in static/const table form.

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Micro Python development wish list

Post by stijn » Mon Jan 05, 2015 8:53 am

Damien wrote:Okay, let's add loadable C modules as a 5th item.

For me they would idealy be: 1) portable/system independent; 2) completely static and not require any linking.
I'm not sure how to interpret the second point, especially the 'not require any linking' part, care to explain that some more?
Module source is built into a static library, then linked into the main uPy executable, some entry is added to a global dict with modules so they can be discovered?

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Micro Python development wish list

Post by pfalcon » Mon Jan 05, 2015 9:57 am

No, "linking" happens at runtime using vtable, as described above.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

Damien
Site Admin
Posts: 647
Joined: Mon Dec 09, 2013 5:02 pm

Re: Micro Python development wish list

Post by Damien » Mon Jan 05, 2015 10:00 am

stijn wrote:I'm not sure how to interpret the second point, especially the 'not require any linking' part, care to explain that some more?
By "no linking" I mean that I don't want to have to modify in any way the loaded C code. For efficiency and simplicity, but mainly so that such modules can be written to flash once, and then don't need any modification, and so can execute directly from flash (saving RAM). This way you could have a few 100k of C modules and not use any RAM.

An example of how it would work, consider the C function that add's 1 to its argument:

Code: Select all

mp_obj_t add_one(mp_func_table_t *table, mp_obj_t arg) {
    return table->binary_op(MP_BINARY_OP_ADD, arg, table->new_int(1));
}
This would compile to pseudo-assembler:

Code: Select all

# r0, r1 are input args, r0 is return arg
add_one:
    mov r3, r0 # save table pointer to r3
    ldr r0, 1 # load first arg with 1
    ldr r2, [r3 + offset(new_int)] # load pointer to new_int
    call r2 # call table->new_int(1)
    mov r2, r0 # store result in r2 (third arg)
    ldr r0, MP_BINARY_OP_ADD # load first arg
    ldr r3, [r3 + offset(binary_op)] # load pointer to binary_op
    call r2 # call table->binary_op(ADD, arg, 1)
    ret # return added value
The above assembler is position independent and requires no dynamic linking.

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Micro Python development wish list

Post by stijn » Tue Jan 06, 2015 9:42 am

Yes I understand the part about using the function table - what I'm not grasping is how you load the code into the process?
Say there's a sourcefile containing the 'add_one' function which has to go in a module 'mymodule'. How do you go from that to

Code: Select all

import mymodule
a = mymodule.add_one(1)
For unix dynamic libraries for example that is rather simple (which is probably why I don't see the other ways): add a function like init_mymodule which calls mp_obj_new_module and populates the returned modules' global dict with an entry for the add_one function. Build all that into a shared lib. When uPy sees 'import mymodule' it loads the lib, calls the init_mymodule function and that's it.
That can be altered to make use of mp_func_table_t in a way that makes no linking required (I think): either pass a pointer to init_mymodule and store it globally in the module and make all functions like add_one access the global, or add a new function type like mp_obj_fun_module_t which always gets the mp_func_table_t pointer as first argument.
But that is for a platform with unportable functions for loading executable code into a process - how do you do that without it?

eduardo
Posts: 31
Joined: Mon Jan 05, 2015 6:48 pm

Re: Micro Python development wish list

Post by eduardo » Tue Jan 06, 2015 11:03 am

Verifiable environment.

I think Micropython does have a great future as a minimalistic "operatingsystemless device" for
absolut secure devices.

(think about it as a signing or encrypting device, or digital key, or Bitcoin storage)

To be a secure device, the vectors of attac in a device should be minimal.

Micropython having no OS has very few attac vectors.

Microython is small enough to be codereviewed by one person. Linux 20 million lines of code can never
be code reviewed.

But developers must be able to receive the trust of the code reviewers of micropyhon. And they can if the whole system is
digitaly signed and versionverified.

The signing could be communicated by the board over LED to a mobile phone camera (sw could be compromised), or an oscilloscope,
or a simple film camera and read out and veryfied by a human.

So I think, making micropython image on the board verifiable would be a great feature.

just my 0.02 c

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Micro Python development wish list

Post by stijn » Wed Jan 07, 2015 9:24 am

Not sure if I'm buying that: if you give me a board, and I can verify with a scope that it's ok, that doesn't tell me anything at all about the security of the code running it? For all I know you modified the source and made it do all kinds of evil, but just made sure the digital signing is still intact? The only way to trust it would be if I acquired, verified and built the source myself and uploaded it to the board. Just as everyone does now.

Post Reply