pypi.org/pypi vs micropython.org/pi

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
User avatar
aivarannamaa
Posts: 109
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

pypi.org/pypi vs micropython.org/pi

Post by aivarannamaa » Tue Jul 28, 2020 7:21 am

Is the purpose of `https://micropython.org/pi/___/json` simply to provide leaner metadata json, compared to `https://pypi.org/pypi/___/json`?

I noticed that `micropython-os` is 0.8 in PyPI but 0.6 in micropython.org/pi. Is this on puropose?
Aivar Annamaa
https://thonny.org

User avatar
mattyt
Posts: 329
Joined: Mon Jan 23, 2017 6:39 am

Re: pypi.org/pypi vs micropython.org/pi

Post by mattyt » Tue Jul 28, 2020 2:29 pm

I believe that the idea is that micropython.org snapshots versions of libraries known to work for the latest stable release of MicroPython.

User avatar
aivarannamaa
Posts: 109
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by aivarannamaa » Tue Aug 04, 2020 11:39 am

If micropython.org/pi has a package with a version, then does it correspond to the package with the same name and version on PyPI?

Can micropython.org/pi contain a package or a version of a package which is not present on PyPI?

I'm planning to extend Thonny's pip GUI to support MicroPython. Currently it queries package metadata from PyPI, but the json at micropython.org doesn't have even summary. Would it be proper to read only the recommended version number from micropython.org and perform all other actions (including reading other metadata) against PyPI (at the same version)?
Aivar Annamaa
https://thonny.org

User avatar
pythoncoder
Posts: 4929
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by pythoncoder » Tue Aug 04, 2020 12:39 pm

micropython.org/pi is under the control of the maintainers and holds official packages guaranteed to work with official firmware. Some packages on PyPi only work with unofficial MicroPython forks. Hence upip searches the official archive first.
Peter Hinch

pidou46
Posts: 81
Joined: Sat May 28, 2016 7:01 pm

Re: pypi.org/pypi vs micropython.org/pi

Post by pidou46 » Tue Aug 04, 2020 1:24 pm

nodemcu V2 (amica)
micropython firmware Daily build 05/31/2016

User avatar
aivarannamaa
Posts: 109
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by aivarannamaa » Tue Aug 04, 2020 2:19 pm

Let's take the example I started the thread with: micropython-os.

micropython.org/pi has version 0.6 of it. Latest version on PyPI is 0.8, but there is 0.6 as well. I now compared the contents of both version 0.6 distributions and found that they have identical Python code and minor changes in metadata -- different order of requirements and different author ("MicroPython Developers" vs "micropython-lib Developers").

I assume that the difference in metadata is caused by different automatic packaging mechanism (neither author matches with https://github.com/micropython/micropyt ... tadata.txt) and the main content in the packages of equal version is equal on PyPI and micropython/pi.

Can someone confirm or refute this assumption?
Aivar Annamaa
https://thonny.org

User avatar
pythoncoder
Posts: 4929
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by pythoncoder » Wed Aug 05, 2020 7:38 am

I suggest separating the general from the specific. There doubtless will be instances where code on PyPi is current. However there are definitely instances of code requiring a specific MicroPython fork. This was causing mayhem and I pushed for a closed archive for officially supported code, with corresponding changes to upip. This was done some time ago.

As for micropython-lib at this location https://github.com/micropython/micropython-lib it is officially controlled.

My advice is to use upip (or micropip). Or use official library modules from the above link. With modules elsewhere and on PyPi caveat implementator applies, as there is no official control over what is posted.
Peter Hinch

User avatar
aivarannamaa
Posts: 109
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by aivarannamaa » Wed Aug 05, 2020 3:50 pm

pythoncoder wrote:
Wed Aug 05, 2020 7:38 am
There doubtless will be instances where code on PyPi is current. However there are definitely instances of code requiring a specific MicroPython fork. This was causing mayhem and I pushed for a closed archive for officially supported code, with corresponding changes to upip.
I read about this confusion and I think I understand the big picture, but I probably missed some important details (considering the amount of uninformative text in those threads). I wanted to know specifically about the correspondence between PyPI and micropython.org/pi at the same version number of a package. If we forget about the tools for now, then:
  • Can there be different Python code under the same version number of a package downloaded from PyPI vs. micropython.org/pi? (Forgive me if you already answered this, I just want to make sure you meant the case with the same version number)
  • When a package in https://github.com/micropython/micropython-lib gets an update,
    • how will the maintainers choose the new version number? Will they increment the latest version number on PyPI or the version number on micropython.org/pi?
    • In case they increment the latest version number on PyPI, will they upload the new version also to PyPI or only to micropython.org/pi? Do they have access to "micropython-lib" user on PyPI?
pythoncoder wrote:
Wed Aug 05, 2020 7:38 am
My advice is to use upip (or micropip).
I was planning to use micropip.py and I intended to ask your permission for bundling it with Thonny. I now assume that this is OK with you :) Thanks!
Aivar Annamaa
https://thonny.org

User avatar
pythoncoder
Posts: 4929
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by pythoncoder » Thu Aug 06, 2020 8:23 am

Your questions about version numbering on micropython-lib can only be answered by the maintainers.

Archives on PyPi are (in general) not owned by the maintainers. Clearly version numbers on such archives are under the control of their owners, along with the code itself. Anyone can create a PyPi repository. Its contents are uncontrolled and unofficial, just like independently maintained GitHub repositories such as my own. You need to form your own judgement as to whether to use and trust them; the maintainers cannot guarantee their quality or compatibility.

The confusion has arisen because micropython-lib repositories on PyPi are owned by an independent individual with his own fork of MicroPython. Consequently in some cases his libraries are incompatible with official MicroPython. In a few instances (notably uasyncio) the differences are radical.

By contrast official micropython-lib and micopython.org/pi are officially maintained. You can have a high degree of trust in their compatibility and be confident that version numbering will be consistent and rational.

Re micropip.py you are welcome to use this as you see fit. There have been some recent changes to upip to improve its ability to download certain archives. I haven't yet found time to review these and port them to micropip. So their may be some minor changes imminent.
Peter Hinch

User avatar
aivarannamaa
Posts: 109
Joined: Fri Sep 22, 2017 3:19 pm
Location: Estonia
Contact:

Re: pypi.org/pypi vs micropython.org/pi

Post by aivarannamaa » Thu Aug 06, 2020 9:16 am

pythoncoder wrote:
Thu Aug 06, 2020 8:23 am
The confusion has arisen because micropython-lib repositories on PyPi are owned by an independent individual with his own fork of MicroPython.
I see. I thought that after pfalcon started to use "pycopy-" prefix, the ownership of "micropython-" PyPI packages was transferred to https://github.com/micropython/micropython-lib maintainers.
Aivar Annamaa
https://thonny.org

Post Reply