State of MicroPython universe (was: of micropython-lib?)

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
User avatar
tuupola
Posts: 54
Joined: Sun Sep 17, 2017 12:10 am
Contact:

Re: State of micropython-lib?

Post by tuupola » Mon Feb 05, 2018 10:56 am

stijn wrote:
Mon Feb 05, 2018 9:17 am
make forks instead of sending PRs
That is not very accurate, it's not one or the other, they go hand in hand. You create a fork with the intent of being able to submit PRs, that's how github works best. If that PR is merged you sync your fork with the upstream again.
I do understand how Git works. By forking in here I do not mean a git fork. Forking here refers to taking old codebase and releasing a new project and continuing in different development paths. See for example Pycom MicroPython vs original MicroPython.

User avatar
pythoncoder
Posts: 3544
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: State of micropython-lib?

Post by pythoncoder » Mon Feb 05, 2018 11:39 am

@tuupola Agreed. If things can't be patched up I think it's the only workable solution. :(
Peter Hinch

User avatar
tuupola
Posts: 54
Joined: Sun Sep 17, 2017 12:10 am
Contact:

Re: State of micropython-lib?

Post by tuupola » Mon Feb 05, 2018 1:14 pm

FWIW to me micropython/micropython-lib is the standard lib. Would be nice to get confirmation about which one is used for libraries installed from pypi.

User avatar
fos
Posts: 42
Joined: Sat Feb 18, 2017 3:05 pm
Location: Texas and Kansas
Contact:

Re: State of micropython-lib?

Post by fos » Tue Feb 06, 2018 12:28 am

I prefer micropython/pyboard due to its polish and stability compared to circuitpython and/or pypi. I hope to stay with the original lib stream provided here. Hopefully all of this will be clarified soon.

In the alternative, I can always return to Microchip and CCS C.

Jeff

EasyRider
Posts: 94
Joined: Wed Dec 30, 2015 8:17 am

Re: State of micropython-lib?

Post by EasyRider » Tue Feb 06, 2018 2:17 am

I prefer micropython/pyboard due to its polish and stability compared to circuitpython and/or pypi. I hope to stay with the original lib stream provided here. Hopefully all of this will be clarified soon.
+1.

This situation needs to be clarified sooner rather than later.
I guess there are commercial developments relaying on micropython that need certainty and clarification regarding future development of micropython.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: State of micropython-lib?

Post by pfalcon » Tue Feb 06, 2018 11:08 pm

There going to be series of posts by me, because one would get too long.

First one is:

The Github is wonderful tool to understand (open-source) project at a glance. So, let's have a look:
What we can learn here? That someone "pfalcon" contributed 2777 commits to MicroPython. Taking just the commits of this contributor and the project owner, that's 42% of all commits (and that's not too much skewed from the absolute figures, because the next contributor, dhylands, has 210 commits).
Here you can see that pfalcon did 1148 commits, while dpgeorge - 9, less than 1%. Overall (again, leaving the exact math to the interested readers), it should be fair to say that I contributed 95% of the project changes.

Those were raw numbers, which are hard to argue with, a summary would be more subjective of course, here's mine:
  • Over almost 4 years, I contributed close to a half of MicroPython content, including designing and implementing important MicroPython subsystems, writing and editing docs, processing bug tickets and pull requests, etc.
  • The micropython-lib project was started by me almost 4 years ago, and since then, largely developed and maintained by me.
(Both projects have many more great contributors, sorry for focusing on humble me this time.)
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: State of micropython-lib?

Post by pfalcon » Tue Feb 06, 2018 11:43 pm

From raw numbers I'd like to jump into the land of fairytales, and tell you one. 5-6 years ago I, for what would be called "IoT" nowadays, hacked on a small language called "Squirrel". That language wasn't actively maintained (even) then, in the sense that his author was pretty satisfied with it how it was, so my suggestions/patches were barely reviewed, and I effectively had to fork it to make the changes I wanted.

My language of choice for last 20 years was Python, and I regretted that I have to hack on something else instead of something well-known and well-designed (designing interfaces is hard!). I thought of hacking on a small Python implementation, but never was crazy enough for that. That's why when I heard about MicroPython kickstarter and read very detailed, technically well-versed updated, I was totally frenzied and went to convince its author, Damien, to let the benefits of OpenSource blossom, and release the source of MicroPython immediately after the campaign, and not half a year later, as was originally planned.

It's easy to advertise OpenSource as "people will help you with the development", it's much harder to put action into those words. Each of us finds many interesting projects (I for sure find at least one a day on average), but most of them are just passing backgrounds. But I actually started to contribute back. Largely, it's also a matter of good circumstances. We had a guy here in 1st or 2nd year, who wanted to rewrite GC. Like, completely, with looong patches which would require a lot of work yet. His last words were "I have much less time for that, I have a baby born." For me, it was the opposite - I'm a sensitive sleeper, so when it was my time to nurse my new-born kid at night, I couldn't fall asleep, having been woken up, and instead hacked on MicroPython. I remember one day within first couple of weeks or so after the code release I received email from Damien along the lines: "Hey Paul, your code is helpful, don't give up on MicroPython." I responded "No-no, I'm still interested, we with wife are just totally zombied with newborn care."

It was a lot of fun, and we didn't need to discuss a lot of things or make rules - we seemed to understand each other and the direction even without it. I again remember receiving a github comment from Damien "I also use a subject 'f <something>' for git fixup commits", I replied: "hey, you scared me, I thought I pushed that to master!"
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: State of micropython-lib?

Post by pfalcon » Wed Feb 07, 2018 12:09 am

It went on, we hacked on things, discussed ideas (a lot of ideas), and we even decided to make a kickstarter together. Damien never said that directly, but it was pretty clear that after his first one, he possibly gave himself semi-oath to "never do it again". Myself, making a kickstarter is what I wanted most of all at that time. I'm very grateful to Damien for giving me such a chance. I learned why people swear to not do kickstarters again ;-). It also never works exactly as expected. It was a great boost to MicroPython in general, more than I would have expected actually, and somewhat less than expected for ESP8266 specifically. Overall, I guess this "skew" is for the benefit of MicroPython users, as any hardware is transient, while MicroPython should carry on. One thing which didn't work as expected for me is that I expected more involvement from the community in development of the ESP8266 port, the whole idea was to bring it to solid foundation and improve MicroPython to do a lot more than before, and to show people that it's easy, so they can work on many more ESP8266 things which were outside the campaign. Instead, it went to episodes when people assumed something like there's a Santa's wishlist where they throw names of things in, and get the things out.

All in all, project was growing very fast, while issues, tasks, and disagreement started to accumulate. I tried to propose ways to formalize the process of resolving such situations to Damien, but it didn't work. Some of my technical proposals went that way too. In the end, it started to look that I'm let to work, instead of my work being appreciated. As the final point, my commit access to the MicroPython repository was removed (of course, I was able to contribute so productively because I had it, and felt as a co-maintainer).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: State of micropython-lib?

Post by pfalcon » Wed Feb 07, 2018 12:31 am

With fairytale off, here's the situation:
  1. As you imagine, I'm not exactly happy to have invested 4 years of my life (2777 commits / 4 years = 1.9 commit daily, every flaming day, or better see it as one commit at day, one at night), almost made a half of the project, for such an outcome.
  2. But who cares. I hacked on Squirrel before, MicroPython, especially half-built by me, is only much better base, and I can hack on it in the same way.
  3. I'm continuing in the same vein on MicroPython as I did before, which is documented at https://github.com/micropython/micropyt ... Guidelines (which also happens to be written by me).
  4. I'm rebasing on the upstream, I certainly trust Damien as a programmer, except he's a human too, and makes goofs (or doesn't adhere to the manifesto above from my PoV), which I'm fixing up.
  5. My specific aim is to "finish" MicroPython - implement all the functionality which can be considered "core", which enabled wide array of efficient usage.
  6. micropython-lib has always been "my" project, and I'm taking it with me. I entrusted it to the "micropython" organization on Github, and chose my project's name to promote that organization (of course, it wasn't like that, it's just there was need to divide "mine" and "yours" in the old good time). The trust was broken. I'm effectively a hostage of that name now, as the migration to another name would be rather resourceful. Anyway, I'm interested in the development, not in all this dividing-up and teh-dramas. As I was "tricked" into using that name by everyone's good intentions, the name is there. I so far have commit access to https://github.com/micropython/micropython-lib and sync to it, but may lose that access abruptly, like happened with https://github.com/micropython/micropython. The main location is https://github.com/pfalcon/micropython-lib/ .
  7. Damien is a smart guy, and surely thought about all the "inconveniences". If he chose that way, then apparently it was what had to be done, from his side. Myself, I don't know where it goes, I'm interested in building the very best embedded language core, per the existing and published requirements.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

User avatar
pythoncoder
Posts: 3544
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: State of micropython-lib?

Post by pythoncoder » Wed Feb 07, 2018 7:24 am

Paul, I'm very sorry it's come to this. I feel I can see both sides of the difficulty. But there is a basic technical issue which needs to be resolved. If your library requires your fork of MicroPython, and you push to PyPI, it will cause chaos.

A new user buys a Pyboard, installs firmware from the MicroPython site, installs a library from PyPI and it won't work. Somehow this must be fixed.

From a user perspective you have a fork which is not a fork. A user of CircuitPython or PyCom has a single source of code and a single point of contact for support. If such a user comes here for support we can politely direct them to the relevant forum. I don't see how we can support a non-fork.

I accept that you, as a (highly) technical person, probably see this as a non-problem. But as someone who attempts to support users of varying levels of experience it will become a can of worms with four combinations of firmware and library and separate maintainers of each. In the case above the user has a product that doesn't work and can reasonably expect support here. I don't relish the task of providing it.

For example I am familiar with your uasyncio code. But despite this I don't actually know if uasyncio 2.0 can be expected to run with a current build of the official MicroPython code. I plan to find out, but repeating this ad nauseam for different users, with unfamiliar libraries, doesn't strike me as fun.
Peter Hinch

Post Reply