Page 1 of 1

Writing C(++) code that is easy to add (as a module) to micropython

Posted: Mon Dec 31, 2018 8:02 pm
by christoph
I'd like to write a library in C (or C++) that is easy to use in C++ and also reasonably easy to port to micropython. There are some topics that I'd like to sort out before I start designing my interface.

- As I understand it, micropython works on structs that are initialized and modified using normal functions (as described here: https://micropython-dev-docs.readthedoc ... odule.html). Does that rule out any C++ class and template trickery?

- passing objects to functions or methods: can I use references or is a pointer easier to port? Within the C code it's a no-brainer, but there will also be methods exposed to micropython that need to pass around references (or pointers) to objects. If I'm limited to plain C, then I'll have to use pointers anyway. However, since modules can throw exceptions (can't they?), C++ is apparently ok.

- abstraction: some kind of abstract interface will be needed. It's easy enough to do this with function pointers in a struct (I had to do this for AVRs and it worked just fine), but is it necessary?

If I just had to write it in C++, my abstract class would have the following virtual methods (How can I format as code? I couldn't find an option for that):

class MyAbstractClass
- bool at_end()
- SomeStruct& next()
- void reset()
- MyAbstractClass& applyTo(MyAbstractClass& other)

Used primarily like this: somethingEntirelyDifferent.digest(myDerivedClassA()).applyTo(myDerivedClassB()). ... (I want to chain different implementations of MyClass)

The classes inheriting from myclass would just have custom constructors and maybe some setters, so nothing out of the ordinary.

So to sum it up: What should I look out for when designing an interface in C/C++

Re: Writing C(++) code that is easy to add (as a module) to micropython

Posted: Tue Jan 01, 2019 10:24 am
by stijn
christoph wrote:
Mon Dec 31, 2018 8:02 pm
Does that rule out any C++ class and template trickery?
Not really: you can use templates just fine, as long as in the final 'bridge' between the C++ code and uPy, namely the C struct/functions which expose your code to uPy, you only use an instantiation.
can I use references or is a pointer easier to port?
You need to write wrapper functions anyway so doesn't really matter in my opinion, it's just a matter of writing one extra *
However, since modules can throw exceptions (can't they?), C++ is apparently ok.
That is not how it works: the exceptions thrown by the Python code in uPy use the nlr/jmp mechanism which is not at all the same as C++ exceptions. In fact if your C++ code can throw, and it gets called by uPy, all of it has to be wrapped in try/catch in order to make sure the C++ exceptions don't escape to C. It's up to you how you handle the catch block, I usually build a uPy exception there and raise it with nlr_raise which basically comes down to translating C++ exceptions to uPy exceptions. Likewise you might have to deal with the other way around: suppose your C++ code calls a uPy function, you must wrap that so it doesn't jump out of the C++ function (since it wouldn't call destructors of whatever was constructed in the C++ function)
some kind of abstract interface will be needed
Such a thing doesn't exist in Python but you also don't really need it. This works rather transparently: you write your wrapper struct/functions to work on MyAbstractClass* but don't provide make_new so in Python it's not possible to call MyAbstractClass(), instead to create an instance you provide functions which return the struct with the MyAbstractClass* pointing to a concrete C++ implementation.

One other thing you might have to take care of is lifetime of C++ objects vs lifetime of uPy objects. The former is deterministic but the latter isn't. Say X and Y are custom classes whos uPy structs store pointers to C++ objects and implement __del__ to destruct the C++ objects (depends on the situation, you don't have to do that, but that also means your C++ objects won't ever get destructed so if you create a lot of them, or operate on external resources like files, you might get into problems a.k.a. memory/resource leaks). Now if you have code like

Code: Select all

y = Y()
y.Store( X() )
then you could have a problem because gc.collect() might call __del__ on the X object created, so the pointer Y stores to it now points to an already destructed X object hence accessing it is undefined behaviour. To solve this I ended up with just using std::shared_ptr everywhere to store my C++ objects which also seems a better match for Python's 'object is allocated once but can have different names'. Also this basically matches with what the C++ code shoyld do anyway: if Y wants to store X, it shoud normally do so by taking a shared_ptr<X> since that is the only guranteed way to make sure X outlives Y.

In the end it doesn't matter how you design your C++ API since pretty much anything can be converted back and forth, but some situations require some more thought in writing the uPy wrappers than others. Also there are some type conversion things to take care of: suppose you have enums in C++ you'll need to take care of a conversion somehow. And uPy has integers of arbitrary size which you obviously can't store in an std::int32_t or so. I ended up with solutions for most of the mentioned poblems in because we have like a hundred C++ classes/functions which need to be called from uPy and didn't want to repeat argument extraction/conversion/lifetime stuff/numeric conversion with range checking/exception translation all over the place. Might be complete overkill for you and might 'obfuscate' things too much (i.e. if you write everything manually it's easier to see what's going on) but should give some ideas at least.