Page 1 of 3

upip - Self-hosted package manager

Posted: Sat Jan 31, 2015 4:03 pm
by pfalcon
As the number of modules written for MicroPython grows, it would be nice to install them directly on a target running MicroPython, including smaller ones, like PyBoard - instead of current situation where package manager is a wrapper around standard CPython's "pip", so running it requires full CPython3, so packages have to be "cross-installed" on a host.

I've been working towards that direction in the background for some time, and guess it's worth to have a specific aim now. I've submitted https://github.com/micropython/micropyt ... /issues/19 to track that. Any (focused) ideas and help are welcome.

Re: Self-hosted package manager

Posted: Sat Jan 31, 2015 4:27 pm
by Turbinenreiter
Is your initial prototyping available somewhere?

Re: Self-hosted package manager

Posted: Sat Jan 31, 2015 6:24 pm
by pfalcon
As usual, writing something more or less serious in uPy leads to bunch of missing dependencies and bugs. So, currently I'm implementing/fixing/polishing initial dirty hacks I have. Here's recent example, non-dummy gzip module: https://github.com/micropython/micropyt ... 22c7e58764 . To implement it, had to add support for raw DEFLATE decoding to uzlib, and turned out the module had issues with decompressing big data blocks. All those issues are fixed now. More to do.

So, I hope to spool all the prerequisites required for current prototype in the near time, and elaborate the prototype itself too.

Re: Self-hosted package manager

Posted: Sat Jan 31, 2015 7:02 pm
by Turbinenreiter
I think it would be worth knowing which prerequisites are missing. Kinda hard to help when I don't know what to hack on ;)

Re: Self-hosted package manager

Posted: Sun Feb 01, 2015 5:42 pm
by pfalcon
As ticket quoted above mentioned, on critical path to complete self-hostedness is SSL support. There's a separate ticket for that: https://github.com/micropython/micropython/issues/952 . Other issue without solving which self-hosted package manager won't work on small-memory targets is stream support for uzlib - currently, uncompressed size is limited by the amount of continuous free memory block.

Past that, what package manager does is downloading tarballs (tar.gz) from HTTPS, and uncompessing them. The latter part is already handled (modulo memory issues above) by micropython-lib:

Re: Self-hosted package manager

Posted: Mon Feb 16, 2015 9:49 pm
by pfalcon
A basic working implementation is now available in https://github.com/pfalcon/micropython- ... aster/upip . Help with testing is welcome.

Re: Self-hosted package manager

Posted: Sun Apr 05, 2015 12:51 pm
by pfalcon
So far I wasn't able to get back to this and thoroughly test it myself, so would like to extend request for help with testing.

Re: Self-hosted package manager

Posted: Sun May 03, 2015 9:32 pm
by pfalcon
There unfortunately wasn't any feedback during this time. I finally sat down to make sure there're fruits of all this effort. It is able to install https://github.com/pfalcon/notes-pico , which is pretty good case of dependency handling. Then with some cleanups, upip 0.5 is released: https://pypi.python.org/pypi/micropython-upip/0.5 . I'll be preparing/submitting changes to make upip the default/recommended package manager for micropython.

Re: Self-hosted package manager

Posted: Mon May 04, 2015 7:39 am
by stijn
Can you lay out all the steps needed to test this?

Re: Self-hosted package manager

Posted: Mon May 04, 2015 10:37 am
by pfalcon
Yes:

1. Build unix version of micropython, make sure it's available in PATH.
2. Bootstrap upip using script from: https://github.com/micropython/micropython/pull/1219
3. Clone https://github.com/pfalcon/notes-pico
4. Use "make lib" on it to install dependencies using upip
5. Use "make run" to start the webapp.
6. Following printed link, try to add/delete some notes.