Reading global Python variable from C code

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
Maksym Galemin
Posts: 11
Joined: Mon May 28, 2018 11:48 pm

Reading global Python variable from C code

Post by Maksym Galemin » Tue May 29, 2018 1:02 am

Hi all,

I'm in the process of replacing the existing ladder logic/function block diagram embedded scripting application with MicroPython on our STM32 product and it's progressing really well. It's such a great relieve to switch to normal Python programming after a few years of GUI mouse programming and I'm really impressed with MicroPython project. :)

In my case I'm running MicroPython in a separate Keil RTX thread in a kind of a "virtual HW board" environment with all the callbacks called from another thread. Also, I've replaced FatFS with YAFFS and simplified a few pyb modules. Unfortunately, there is a few GCC dependencies for autogenerated things (I'm using Keil MDK-ARM), but surprisingly the whole porting process wasn't too painful. As far as I can tell after digging into MicroPython's internals it's a very, very good and highly customisable embedded project.

However, at the moment I'm struggling to understand how I can synchronise an internal small "database" of bool/int/float values between the MicroPython script and the main C++ code. Ideally, I would like to read/write global Python variables from my C++ code in a separate thread via a mechanism similar to PyEval_GetGlobals or PyObject_GetAttrString. Obviously, MicroPython doesn't support this functionality at the moment (and I'm not expecting it to do this) but I'm wondering what are my options apart from exposing variables through an additional module? The internal database can be changed from 3 different threads asynchronously and all other observers (including MicroPython) should update their values and also on a change to any of the monitored Python global variables the internal database should be updated accordingly. Thanks.

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: Reading global Python variable from C code

Post by jickster » Tue May 29, 2018 3:48 pm

Maksym Galemin wrote:
Tue May 29, 2018 1:02 am
Ideally, I would like to read/write global Python variables from my C++ code in a separate thread via a mechanism similar to PyEval_GetGlobals
uPy has mp_globals_get()

jickster
Posts: 629
Joined: Thu Sep 07, 2017 8:57 pm

Re: Reading global Python variable from C code

Post by jickster » Tue May 29, 2018 3:52 pm

Maksym Galemin wrote:
Tue May 29, 2018 1:02 am
should update their values and also on a change to any of the monitored Python global variables
You can make the database be a uPy object - like a dictionary - and map it into the global uPy context using mp_store_global() at initialization.

This way, you won't need any kind of "callbacks" to perform synchronization because everyone will be referencing the same thing in memory.

The only downside is that when your 3 C++ threads modify it, it'll be slower because the database AND the entries - int, bool, floats - will be uPy OBJECTS so you'll have to use the uPy C-API to modify/access the underlying values.

bool mp_obj_is_true(mp_obj_t arg) <-> mp_obj_t mp_obj_new_bool(mp_int_t x)
mp_int_t mp_obj_get_int(mp_const_obj_t arg) <-> mp_obj_t mp_obj_new_int*(mp_int_t value) (* = see obj.h; more variants depending on size of value)
mp_float_t mp_obj_float_get(mp_obj_t self_in) <-> mp_obj_t mp_obj_new_float(mp_float_t value)

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Reading global Python variable from C code

Post by stijn » Tue May 29, 2018 6:48 pm

Instead of mp_load/store_global you could also use mp_load/store_name: it takes current scope into account (local/global/builtins) should you need that. Some nested/recursive functions which do that and take care of uPy <-> C++ object conversion that can be found here: https://github.com/stinos/micropython-w ... variable.h

It's not completely clear to me what you want to do but be aware that you'll probably need some kind of synchronisation: depending on the float implementation used, calls like mp_obj_new_float have to allocate from the uPy heap which you cannot do from multiple threads. Moreover allocating might raise an exception (a Python one, not a C++ one) so if you are not doing that allocation on the thread which is running the uPy interpreter you should wrap them in nlr_push/nlr_pop to avoid all hell breaking loose :]

All in all, writing a module might still be the most convenient (again, not 100% about the usecase): you can expose a simple API to uPy like mymodule.getValue(key) and mymodule.setValue(key, value) while taking care of all synchronisation/notifications in C++. Not exactly the same as writing global variables from C++ but doing that while providing synchronisation might be painful, as far as I know at least.

Maksym Galemin
Posts: 11
Joined: Mon May 28, 2018 11:48 pm

Re: Reading global Python variable from C code

Post by Maksym Galemin » Wed May 30, 2018 1:34 am

Thanks a lot for your responses jickster and stijn! I'll try to use mp_load_global/mp_store_global and mp_load_name/mp_store_name with gc_lock/gc_unlock guards and extra synchronisation first.

Post Reply