extmod generator

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
User avatar
stick
Posts: 13
Joined: Fri Jan 08, 2016 9:17 pm

extmod generator

Post by stick » Thu Mar 03, 2016 4:20 pm

Recently I was trying to learn how to write my own external modules and found out it was not completely straightforward.

To help me out, I wrote a generator that will analyze a module written in Python3 syntax and generate a module to use with Micropython (to be placed in extmod directory).

Source is available here:

https://github.com/prusnak/micropython-extmod-generator

It has some rough edges, but I think it is already quite usable and helpful.

What do you think? As always, pull requests are welcome!

pohmelie
Posts: 55
Joined: Mon Nov 23, 2015 6:31 pm

Re: extmod generator

Post by pohmelie » Thu Mar 03, 2016 6:39 pm

Nice idea. I think example.py and example.c should be in readme.

User avatar
stick
Posts: 13
Joined: Fri Jan 08, 2016 9:17 pm

Re: extmod generator

Post by stick » Thu Mar 03, 2016 10:36 pm

Added links to examples to README.md

SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: extmod generator

Post by SpotlightKid » Thu Mar 03, 2016 11:19 pm

Cool, very useful. Two suggestions:

- I think the convention in Python is generally to use __author__ instead of _author (and __version__, etc.) for module meta data.
- Would be nice to generate the qstr definitions as well (maybe put them in a comment in the .c file).

User avatar
stick
Posts: 13
Joined: Fri Jan 08, 2016 9:17 pm

Re: extmod generator

Post by stick » Thu Mar 03, 2016 11:36 pm

SpotlightKid wrote: - I think the convention in Python is generally to use __author__ instead of _author (and __version__, etc.) for module meta data.
OK, changed in 409337515c0db6789a8344164102955b7ce0ce96.
SpotlightKid wrote: - Would be nice to generate the qstr definitions as well (maybe put them in a comment in the .c file).
Aren't they supposed to be generated automagically via makeqstrdata.py during the build?

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: extmod generator

Post by dhylands » Thu Mar 03, 2016 11:43 pm

stick wrote:Aren't they supposed to be generated automagically via makeqstrdata.py during the build?
No makeqstrdata takes the qstrs found in one of the qstrdefs.h files and generates the c structures associated with a qstr.

The file you'd need to create would be a file that contained the:

Code: Select all

Q(someString)
Q(someOtherString)
in it. The developer would then need to merge this with https://github.com/micropython/micropyt ... #L665-L671 (I highlighted the ubinascii qstr's).

I'd love to see a tool which created the qstrdefs.h file automagically, but it hasn't been written yet. Getting it right is definitely non-trivial.

User avatar
stick
Posts: 13
Joined: Fri Jan 08, 2016 9:17 pm

Re: extmod generator

Post by stick » Fri Mar 04, 2016 12:09 am

dhylands wrote:No makeqstrdata takes the qstrs found in one of the qstrdefs.h files and generates the c structures associated with a qstr.
Ah I see, implemented in 53ff05cd419ea37b128807effe833b480845fbaf
dhylands wrote: I'd love to see a tool which created the qstrdefs.h file automagically, but it hasn't been written yet. Getting it right is definitely non-trivial.
Hm, I think that similar magic to what I used in the commit above could be used to generate all required qstrdefs.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: extmod generator

Post by dhylands » Fri Mar 04, 2016 12:28 am

The issue I see with generating qstrdefs automtically is figuring out what #ifdefs they belong in. We don't want to include unnecessary qstrs in the image.

User avatar
braiins
Posts: 19
Joined: Mon Feb 23, 2015 1:09 pm
Contact:

Re: extmod generator

Post by braiins » Mon Mar 07, 2016 7:25 am

qstr generation is something that should definitely be looked into. I believe the point is to not bloat sources with #ifdefs or even better to ban the use of ifdefs except for header file guards ;-)

Stick is in much better position while generating qstr definitions since with python reflection the generator know exactly what comes into the output. However, this is not the case with hand written uPy module in C
braiins - Your partner in embedded world - http://www.braiins.cz

User avatar
stick
Posts: 13
Joined: Fri Jan 08, 2016 9:17 pm

Re: extmod generator

Post by stick » Mon Mar 07, 2016 12:36 pm

dhylands wrote:The issue I see with generating qstrdefs automtically is figuring out what #ifdefs they belong in. We don't want to include unnecessary qstrs in the image.
I created a spin-off thread for this topic here: qstrdef-generator

Post Reply