MicroPython with MS compiler

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

MicroPython with MS compiler

Post by stijn » Thu Apr 24, 2014 9:26 am

I'm looking for a Python distribution to run on Tenasys' INTime realtime system and by the looks of it uPy seems like it could work out: I mainly need interpreting, the basic builtins/math/collections etc. The rest I'd be providing myself (most of it ic C++) but it doesn't seem that hard to wrap it in C and create a mp_obj_type_t for it.

One requirement however is that everything has to be compiled with a Microsoft toolset and the main problem is it's not C99 compliant, meaning compilation as it is know equals thousands of errors. I tried to compile some files and the main problem is for MS's C dialect basically all variables need to be declared at the top of the function. If I just compile the whole thing as C++ I don't have that problem, but then I'm going to have to cast a whole lot of pointers because C-style implicit pointer conversions are invalid in C++.

Anyway I'm sure I can get everything to compile with some work. My main question is: will it run? Are there any dirty hacks/special tricks being used that depend a whole lot on the compiler? Anything I sould look into specifically?
Second question: did anybody look into this before?

edit

I spent a couple of hours getting everything to compile (excluded a bunch of modules though), and it runs simple scripts, both on Windows and INTime :]
Mainly what had to change was inserting pointer casts all over the place (because of cimpiling as C++), and fixing the C-style struct initialization (or whatever that is called, the one with { .<member name> = ... }).
I still have to build and run all tests though, but it looks promising nonetheless.

Btw I took a generated qstr header from a linux build - could this cause problems?

If this works out there's a good chance we're going to use uPy. I can't really talk about the way it is going to be used atm due to IP, but it would mean it's going to be a key part in scientific experiments for at least the next decade, and the results produced will be in numerous papers, some of them published in premier journals (think Nature/Journal of Neuroscience). And also that I can spent quite some time on it as part of the daytime job. So it would be a shame if I can not contribute the efforts back, however due to the nature of the changes to the code this can become messy. So we should discuss the easiest way to do this. Atm I only see these possibilities:
1. I used some scripts to make the transition quicker. Basically automatically inserting the casts etc. I could supply these scripts, hence others could compile using Visual Studio version whatever by first running the scripts, then building. However that has the serious disadvantage that if some piece of code is changed it's pretty hard to get back into the main branch since the compete diff is huge. On the other hand I guess it's not that hard to either create partial patches, or write an 'undo' script.
2. I maintain a windows only fork which I regularly update using the original main branch. Don't really feel like this though for various reasons.
3. I try compiling as C anyway and use #ifdefs for the parts that need to change. Less change is required, easier to get in the main branch, but it still requires lots of maintainance unless the other developpers become aware of it and write MS-compiler compatible code from the start.
Please share your views on this.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: MicroPython with MS compiler

Post by pfalcon » Sat Apr 26, 2014 1:24 pm

Did you give *real serious* consideration to getting a real compiler? That doesn't even have to be GCC if your boss has problems with it - LLVM conquers the world nowadays. Microsoft will switch to it too eventually, they will be just the last to get it, just the same as with implementing 15-year old C standard.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: MicroPython with MS compiler

Post by stijn » Sat Apr 26, 2014 4:32 pm

I did briefly consider it a couple of years ago, but didn't look much further into it: IIRC the whole InTime SDK is only compatible with the MS toolsets. I'll check again too make sure though.
But even if that works out I'm not sure if e.g. debugging in Visual Studio would work the same way as it does know (and that is pretty much a hard requirement) nor if the produced executables have the same performance.
All in all switching to a new toolset, figuring out which one etc seems a lot more work than just getting uPi to compile. Though it might pay off on a longer term.

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: MicroPython with MS compiler

Post by stijn » Mon Apr 28, 2014 11:14 am

Checked with the InTime people and getting things working with another compiler seems doubtful - maybe ICC might work.

However I kinda forgot MS did finally do quite some work towards C99 compatibility with the VS2013 release, and indeed: uPy compiles with a minimum of changes and about two thirds of the tests pass. Yay!
The rest are all crashes, assertion errors and endless loops, but probably all casued in the same pieces of code. I'll check on that further.

Post Reply