Does anybody know what is going on with micropython?

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Does anybody know what is going on with micropython?

Post by stijn » Fri Sep 20, 2019 8:15 am

The sad truth is there's no way to build something realistic with micropython on *any* platform
You make some valid points, but here I think you're extrapolating your particular experience to a more general case which isn't correct. For example I've seen many neat projects here using microcontrollers, and we're on PC ourselves and ones system we built with MicroPython consists of a large C++ codebase running A/D conversion, various signal processing and talking to a myriad of external hardware with a Labview user interface on top of that. The other system we built uses MicroPython to describe sequences of visualizations shown using OpenGL etc, so again a C++ codebase wrapped in Python for ease of use (and talking to the other aforementioned system). So all driven by MicroPython and in use for about 4 years now, daily, for running scientific experiments in the field of neurophysiology, so no room for mistakes. Just to illustrate that even despite some limitations, the only truth here is that it might not work for you but it definitely works well for others.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Does anybody know what is going on with micropython?

Post by pythoncoder » Fri Sep 20, 2019 8:48 am

As a retired engineer my Pyboard projects are "amateur". But, had MicroPython and Pyboards been available at the time, there are several professional projects which I would have implemented using them. I'm not qualified to comment on compiler internals, but in terms of reliability the Pyboard variants score top marks. Aside from some instrumentation projects I have a project comprising four Pyboards which have been operating 24/7 for nearly four years.

I've not seen anything in over five years of use which suggests that the Pyboard is unsuitable for professional applications. Moreover, there is ample evidence that they are actually being used as such by various companies.
Peter Hinch
Index to my micropython libraries.

jhonpacker
Posts: 1
Joined: Fri Sep 20, 2019 10:46 am

Re: Does anybody know what is going on with micropython?

Post by jhonpacker » Fri Sep 20, 2019 10:49 am

Good to see this post. Almost anything you can imagine! Just like an Arduino board MicroPython can control hardware and connected devices. You can control GPIO pins to blink lights, read switches, and more. You can drive PWM outputs for servos, LEDs, etc. or read analog sensors with an analog to digital converter. Thanks for sharing..
You may check out this website: https://liteblue.run

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Does anybody know what is going on with micropython?

Post by jimmo » Fri Sep 20, 2019 12:05 pm

pmp-p wrote:
Fri Sep 20, 2019 2:19 am
jimmo wrote:
Fri Sep 20, 2019 12:38 am
You've chosen a particularly difficult part of MicroPython -- the emscripten port is complicated
First the fact that running the VM can run without any "referenced" board - pricey or not - in multiple ways should open eyes to some about the fact they are missing a train.
I agree - like I said, this whole area is exciting for MicroPython. But like anything else, it's all tradeoffs. Whether it be developer time, how features are implemented, etc. Sadly, you just can't build a thing that does everything perfectly.

MicroPython was built to run on microcontrollers. It does an amazingly good job of that. It makes difficult design decisions that will fundamentally always revolve around saving RAM and ROM. Where possible, flexibility exists to enable/disable things, but microcontrollers will always come first.

Many people want MicroPython to be other things too. It's great when that works, but sadly sometimes it doesn't.
pmp-p wrote:
Fri Sep 20, 2019 2:19 am

But i did not choose anything, i don't even use javascript port or android ports though i honestly tried to promote them.
i only use my own wasm build to pop out I/O flaws because it's a tough platform but very fast build in order to backport solutions to cpython. The sad truth is there's no way to build something realistic with micropython on *any* platform for several reasons.
stjin and pythoncoder have already covered this well, but I feel strongly about this too. Have a search on youtube for presentations from people who have built real products. Read about the hundreds of thousands of children who've learnt robotics for the first time using MicroPython (and CircuitPython). The scientists who use MicroPython every day in their experiments and spent their valuable funding money and time on other more useful things than writing C code or buying very expensive test equipment. You can even watch the creator of MicroPython talk about the ESA using it. Or vendors like Digi and ST who sell products based on MicroPython.
pmp-p wrote:
Fri Sep 20, 2019 2:19 am
#VM:

- string interning is actually bad.
I'm genuinely curious as to what you mean. As a _user_ of MicroPython, I'm not sure I've ever written any code that could notice that string interning exists. As a developer, other than one time I had to figure out how to make a MP_QSTR_ value that had a special character in it, it's never been an issue. It... just works? Seriously, considering how much goes on behind the scenes to make interning work, I always thought of this as quite an achievement.

And more importantly, string interning saves bytes. Precious bytes! And with interning it's both RAM _and_ ROM. Do you have an alternative approach in mind?
pmp-p wrote:
Fri Sep 20, 2019 2:19 am

- lack of finalizers that's even worse.
I think there's some confusion about this feature.

Finalisers in MicroPython work. They do exactly what you need (call __del__ at some point in the future when all references are gone), for any type (user defined or written in C), but only if you enable it when allocating the instance. Support for it is enabled on most ports, and most ports actually rely on it for a few features.

The only thing that is missing is a way for a user-defined type to indicate that it should be created with finaliser support by default. But how to do this?

stinos's suggestion in the PR is correct -- you just enable it always. Done! Send PR!

But this perfectly highlights the "micro" of MicroPython. A vanishingly small number of users of MicroPython want this feature. So should they pay the cost on every object creation (and I believe there's also the memory overhead required to remember that the block has a finaliser). This seems completely obvious to me, the answer is definitely no.

OK, so maybe you adapt stinos' suggestion slightly and somehow figure out whether the object being created has a __del__ method. But now you've just moved the performance cost from the GC to object creation (at least you've saved the memory cost though).

MicroPython has a solution to this -- feature flags. So, if finalisers are important to you, the way to progress this issue is to add an additional flag, dependent on MICROPY_ENABLE_FINALISER, perhaps named MICROPY_ENABLE_FINALISER_ALL_OBJECTS, that conditionally does stinos' suggestion. That way a port (or a custom build) can enable it. And then perhaps there can be a discussion about enabling it by default in the Emscripten (and maybe even Unix) ports?

This would be much easier than demanding a roadmap from the maintainer for a feature that likely isn't on their radar.
pmp-p wrote:
Fri Sep 20, 2019 2:19 am

- random disregard of some basic Python functions, ok it is called "The MicroPython Language" it's not "a" Python flavour but who need another crippled spinoff with no stdlib ?
I've seen four really good reasons why MicroPython is not 100% CPython compatible:
- Saving bytes. (This is the big one. microcontrollers.)
- It just isn't done yet. (Some features are hard to implement!)
- It isn't a feature that makes sense for microcontrollers (i.e. very difficult to implement)
- A suitable compromise can be reached (i.e. partial compatibility). Most people don't notice, many people don't need it anyway, very few need actual workarounds, and extremely few complain.

f-strings are worth discussing here. They're clearly a great feature. Once we can move past the bytes cost (compatibility with a growing amount of existing code is a great reason, notwithstanding the two existing ways of doing string formatting), the first part is someone needs to actually do it. #4998 is a good PR, it got quick attention, it'll hopefully get merged once the (very reasonable and quite helpful) review comments are addressed. But the implementation is subtle, and the resulting CPython incompatibility is quite interesting.

It would be fantastic if f-strings could be implemented exactly like CPython, but then literal concatenation would have to move to the parser. See why it was moved to the lexer in the first place. This PR is making a compromise.

Back to your question. The answer to "who needs (this)"? Embedded developers who would otherwise be writing C.

To put it another way... I would still prefer to write C-style MicroPython than actual C (and sometimes I do occasionally need do this).

As for the stdlib... as MicroPython grows, to bigger ports with more functionality, I agree some of the stdlib functionality will become more relevant. asyncio is one that is personally relevant to me. The current implementation goes a long way (and there are excellent community-supported extensions and improvements), but it would be great to improve it. (I think the general point there though is... "PRs welcome!").
pmp-p wrote:
Fri Sep 20, 2019 2:19 am
- the lack of documented code for building user modules ( objects AND classes )
Yep, absolutely, the developer documentation could use some work. But other than the fact that whomever writes it first has to understand it, there's very few barriers here. Documentation PRs are always welcome. I'm excited to see where viewtopic.php?f=3&t=6874&start=10#p39242 goes.
pmp-p wrote:
Fri Sep 20, 2019 2:19 am

- no proper options for embedding as a library ( not even an option to create a single C file for any given port as a workaround).
This has not been my experience, and I do not understand what you mean by the second part. I have worked on (professional) projects that embedded MicroPython, it's worked really well.
pmp-p wrote:
Fri Sep 20, 2019 2:19 am
- MicroPython dependency on C stack.
I agree, and clearly so does Damien -- see MICROPY_ENABLE_PYSTACK which resolves exactly this. It's not enabled by default yet for a couple of reasons (again, everything is tradeoffs, and there's still some more work to be done), but I can see it possibly becoming the default in the future.

And related, see MICROPY_STACKLESS. These two features go well together. (And they're _good_ features).
pmp-p wrote:
Fri Sep 20, 2019 2:19 am
- lack of an universally qualified reviewer, since obviously absolutely everything must pass by one way narrow passage approval on any subject ( that is intentionally sarcastic as a reaction to #5111 ).
Couldn't agree more. It would be wonderful if there was a whole team of reviewers. One for every time zone, etc! But who? There are very few people contributing to the core on a regular basis, and it takes a lot for the owner of a project to (figuratively) hand over the keys.

Maybe if they weren't "universal" (in the sense that maybe you could have port-specific maintainers, etc). That would be great too, and probably easier to find such people.

I personally don't have a huge amount of experience with open source, but this is a fairly common situation in corporate software development too. There's always going to be some parts of the code base that have fewer "owners" than is ideal.

I'd be curious to hear any suggestions you have?

Re #5111 (and #4875), happy to send some private feedback as a neutral observer, but I don't think anyone was actually saying anything negative about #4875? #5111 is objectively a simple PR, and the current state of #4875 is a little hard to understand (this is true of any PR that goes through a lot of discussion).
pmp-p wrote:
Fri Sep 20, 2019 2:19 am
PS: What has the removal of nlr has to do with emscripten ? could you elaborate here https://github.com/micropython/micropython/issues/4993?
It seems like you've already explained this there? Removing NLR means not having to use setjmp/longjmp. (NLR is a bit of a confusing term because it both refers to the overall feature (which may or not be implemented with setjmp), and also the "native" implementation of NLR (the alternative to setjmp)).

pmp-p
Posts: 13
Joined: Mon Apr 24, 2017 12:45 am

Re: Does anybody know what is going on with micropython?

Post by pmp-p » Fri Sep 20, 2019 12:42 pm

stijn wrote:
Fri Sep 20, 2019 8:15 am
the only truth here is that it might not work for you but it definitely works well for others.
That's not how you build a community : such a reply is the best springboard for "not staying".

Because if you don't care for my use case or my friend's ones . why should we care about those of yours and give free time to help on them.
pythoncoder wrote:
Fri Sep 20, 2019 8:48 am
I've not seen anything in over five years of use which suggests that the Pyboard is unsuitable for professional applications.
Well be surprised : i don't own a pyboard, did not get one for free, won't buy one any time soon nor start to (re)sell them.
Why anyone in that case should stay "on board" ?
jimmo wrote:
Fri Sep 20, 2019 12:05 pm
(...)
i'm sorry mate but you look like a fanboy who stall forum and issues and has no real technical clue about the points i've listed.
also you should learn that it takes time to test software before sending any PR do you think the list popped up one morning in my clipboard ?

*and* anyway i still think pfalcon - despite his terrible manners - is the guy that can fix them for breakfast and far better than me.

Note this, i'm only interested in Python "code correctness" to quote someone famous from the other side of MicroPython ( the C side nobody sane would wants to deal with when using *any* Python flavour).

The vm problem list is there. it's C and if nobody is around to fix them, then there lies the problem.

i have no time for nitpicking / post burial / personnal attacks / education / walled gardens and commercial considerations of any sort so please try to start answering the OP or at least move toward fixing any point of the previous list to show some good faith

for chit chat there are place like https://www.reddit.com/r/Python/comment ... g_on_with/

stijn
Posts: 735
Joined: Thu Apr 24, 2014 9:13 am

Re: Does anybody know what is going on with micropython?

Post by stijn » Fri Sep 20, 2019 1:07 pm

pmp-p wrote:
Fri Sep 20, 2019 12:42 pm
stijn wrote:
Fri Sep 20, 2019 8:15 am
the only truth here is that it might not work for you but it definitely works well for others.
That's not how you build a community : such a reply is the best springboard for "not staying".

Because if you don't care for my use case or my friend's ones . why should we care about those of yours and give free time to help on them.
Apologies for the misunderstanding, but my point was simply: there are issues making MicroPython not work for you, but that doesn't mean it's doesn't work for others. Never did I mean to say "I'm ok with MicroPython so it's feature-complete". On the contrary: I only want it to move forward further, which implies fixing things which I personally don't have use for. I do care for that. Which is exactly what I did for instance when presenting a solution on github for finalizer support on user-defined classes. (my githb nick is stijn, here I'm stinos for reasons I don't recall)
i'm sorry mate but you look like a fanboy who stall forum and issues and has no real technical clue about the points i've listed.
Hmm, after reading Jimmo's comment I was about to post a reply just to thank him for writing such an elaborate reply, which I didn't find clueless at all.

nekomatic
Posts: 37
Joined: Thu May 08, 2014 9:31 pm

Re: Does anybody know what is going on with micropython?

Post by nekomatic » Mon Sep 23, 2019 3:21 pm

stijn wrote:
Fri Sep 20, 2019 1:07 pm
Hmm, after reading Jimmo's comment I was about to post a reply just to thank him for writing such an elaborate reply, which I didn't find clueless at all.
Same here!

pmp-p
Posts: 13
Joined: Mon Apr 24, 2017 12:45 am

Re: Does anybody know what is going on with micropython?

Post by pmp-p » Mon Sep 23, 2019 5:11 pm

nekomatic wrote:
Mon Sep 23, 2019 3:21 pm
Same here!
Maybe, but I think @stinos a.k.a. stijn knows the differences beetween the C stack and the Python stack and what a recursive C call imply for pre-emption in bytecode loop, do you too ?

"technical clue" was a bad wording i meant "expertise technique" wich has nothing to do with a "clue" that could lead to a more technical analysis pursuit - but after the fact - like reading https://github.com/micropython/micropython/issues/1036 of what is a stackless design , why it is _better_ than _good_ and also discover why it is not actually available https://github.com/micropython/micropyt ... -366246656 * and guess what ? cpython has the same problem so no need to stay on that road.

* and for those interested by the subject "generators" are not python ones : but only call path recording in a static record in heap linked to the sys max recursion level and a few push/gosub/ret/pop C macros, and of course not using same identifiers for imbricated scopes ( see the vm loop exception handler). Anyone interested in poc is welcomed to send me a beer.


jimmo thanks *a lot* for https://micropython-usermod.readthedocs ... ea=default it is indeed very instructive and is the kind of document that give hope anew since https://github.com/deshipu/micropython-dev-docs though maybe C finalizers lack a chapter as shown in the search link above.

User avatar
jimmo
Posts: 2754
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: Does anybody know what is going on with micropython?

Post by jimmo » Tue Sep 24, 2019 5:39 am

pmp-p wrote:
Mon Sep 23, 2019 5:11 pm
Maybe, but I think @stinos a.k.a. stijn knows the differences beetween the C stack and the Python stack and what a recursive C call imply for pre-emption in bytecode loop, do you too ?
This is completely unnecessary. I don't know if the "do you too" is addressed at nekomatic or me, but it doesn't matter.
pmp-p wrote:
Mon Sep 23, 2019 5:11 pm
"technical clue" was a bad wording i meant "expertise technique" wich has nothing to do with a "clue" that could lead to a more technical analysis pursuit - but after the fact - like reading https://github.com/micropython/micropython/issues/1036 of what is a stackless design , why it is _better_ than _good_ and also discover why it is not actually available https://github.com/micropython/micropyt ... -366246656 * and guess what ? cpython has the same problem so no need to stay on that road.
If you have a an issue with my (or anyone's) technical analysis, then please feel free to provide an alternative analysis or explanation.
pmp-p wrote:
Mon Sep 23, 2019 5:11 pm
jimmo thanks *a lot* for https://micropython-usermod.readthedocs ... ea=default it is indeed very instructive and is the kind of document that give hope anew since https://github.com/deshipu/micropython-dev-docs though maybe C finalizers lack a chapter as shown in the search link above.
Even a minimal amount of effort can clearly show that I am not the author of this (very excellent) site so this looks like sarcasm. I was prepared to give you the benefit of the doubt here but I now see that the search query is for "__del__" so you're clearly trolling.

Anyway, I've had enough, as have the number of people who have messaged me privately. So consider this your first and only warning. Either modify your conduct in this forum and on GitHub or you will be banned from both. No more personal attacks and no more unfair or derogatory comments about CPython or MicroPython or their respective communities.

Please read https://www.python.org/psf/codeofconduct/ if you'd like some more background.

pmp-p
Posts: 13
Joined: Mon Apr 24, 2017 12:45 am

Re: Does anybody know what is going on with micropython?

Post by pmp-p » Tue Sep 24, 2019 8:46 am

jimmo wrote:
Tue Sep 24, 2019 5:39 am
If you have a an issue with my (or anyone's) technical analysis, then please feel free to provide an alternative analysis or explanation.
i think i just did, you had mistaken C-stack and Python stack and yes in embedded world it's an issue especially when dealing with wasm computing units where setjmp or sleep is even more costly than other targets ( 10x slow down on overall computing not just a switching context problem) .

If you judge the materials i pointed you to are insufficient and that an more detailed explanation would benefit MP then please do open a topic in devel forum and i'll try to do my best. I can understand that if you work only with ARM or ESP32 my saying can be quite fuzzy.

------------------------------------------------------------------------------------------------------------------------------------------------
For the technical part i already answered and i'll stick to that list redacted from your remarks because that list is technically valid in the context of evolution.
------------------------------------------------------------------------------------------------------------------------------------------------
now for the more personnal stuff that has no place here, for that aspect of things, sadly no : i don't know what is going on with micropython.
------------------------------------------------------------------------------------------------------------------------------------------------

1), i'll remind again. English is not the native of tongue of everyone on this earth. So please avoid "feelings" when reading non-natives people who try to clumsy mix-in like me. This is The Internet, thank you.

I'll try to do the same when i misread you but only if what you are saying is relevant to *SOLVING* issue, there's no need to leave GH issues wide open for status-quo or musing around.

If you were hurt *anywhere* i'm deeply sorry and yes, you could have private messaged me here to help correct me : BUT YOU DID NOT.

i'm also on https://gitter.im/micropython/Lobby and freenode #micropython and #micropython-fr since years where you can speak your mind to me freely : BUT YOU DID NOT.


> Even a minimal amount of effort can clearly show that I am not the author of this (very excellent) site so this looks like sarcasm.

I have absolutely *no idea* of what you mean so maybe 1)


> I now see that the search query is for "__del__" so you're clearly trolling.

I have absolutely *no idea* of what you mean *again*

yes i was searching documentation for implementing group finalizer and C __del__ is a attribute introduced by @dpgeorge for fixing specific needs in external libs but with no tests and samples : it may be MISSING in both documentation available.

but thankfully @stinos gave me required detailed explanation here https://github.com/micropython/micropython/issues/1878 so consider that matter closed, i'll do a PR to doc if understand how to use that special function.


> as have the number of people who have messaged me privately

Well i don't read your private message so again i have no idea what you mean by that, and i did not receive complaints from anyone yet except maybe 1 person that should keep 1) in mind and probably misread me because i was trying to add a value to his work ( which wasn't primarily made for MicroPython and was not discussed much) via a open layer.

> So consider this your first and only warning.

Is it unilateral ? And where are moderators of this forum ? Because i'm the one who feel constantly stalked on GH issues with noise irrelevant to issues solving. So it's maybe time that threatening gets arbitrated.

> unfair or derogatory comments about CPython or MicroPython or their respective communities.

i came to MicroPython because i needed experiencing on platform porting/maintenance over existing codebase to evaluate cpython situation regarding Android / WASM / WASI platform which are currently not supported, because Victor Stinner warned me personnaly that handling such tasks need great effort and most of the time gets a greater *burden* than expected and maybe solitude ( maybe an example is the VxWorks cpython port which afaik was is 1 man effort ).

i don't feel you represent both communities either officially / more or less than me or any contributor around. But i don't know you at all, so i could be wrong. In that case maybe display your current position/work/special connections in your forum/gh signature to prevent disrespect of that particular situation ?.

Please do take the case to PSF before i meet their members at PYCON so they can tell me by themselves ( and in my native tongue) what's wrong with me. ( Apart from my very bad attempt at joking in a wide audience english international forum which i now regret, but a moderator saved me and lesson learned )

But if you do that please don't forget to tell people nobody mentored me in this community since 2017 except some assistance from deshipu (nice and helpfull) and pfalcon ( less nice but very helpfull for code that matters).

Also i don't think a technical view of things can be "fair or unfair", but it can be right/wrong relevant/irrelevant to a context or badly worded.


> you will be banned from both.

FYI i already asked multiple times an owner of the GH repo if was bothering and should go away from myself. And i did not receive any formal answer yet.

FYI i'm the only person actually attempting to maintain the javascript port layed out by matthewelse / flowergrass / andrewleech / kkimdev / ... and numerous other people that can be found in the numerous wasm/javascript threads. Even a bounty was opened by alandjs ( not fullfilled because time.sleep() and input() just crash browser).

Are you trying to leverage my efforts on that port or undermine my general influence on talks ? if you have reasons please state them here i made a space dedicated to that:

port-javascript: check list #4993
________________________
https://github.com/micropython/micropython/issues/4993



For people new to wasm: wasm/asm.js is the micropython port with the less sdk tools, most unfriendly processor ever, with no signals / interrupts or registers and with a 15 seconds hardcoded watchdog that prevent any blocking IO.

For short : if it can work there it will work everywhere even better.

https://github.com/micropython/micropyt ... javascript

That platform is also incidently the most spreaded in the world either on desktop or mobile because it can run in browser and webviews.

Post Reply