Page 1 of 1

Announcing pygitgrab

Posted: Wed Dec 04, 2019 10:58 am
by kr-g
hiya,
i have developed a small tool which grab's only certain information from a remote git repo and stores them local.

for instance if you want to bring in https://github.com/jczic/MicroWebSrv2 into your project.
create a local 'pygg.cfg' file with the following lines

[MicroWebSrv2]
url='https://github.com/jczic/MicroWebSrv2'
src="MicroWebSrv2/*.py"
license="LICENSE.md","LICENSE/MicroWebSrv2.MD"

if your project needs a special version just add (if obmitted 'tag' defaults to 'master' otherwise)
tag=v2.0.5

then execute 'python3 -m pygitgrab' to get the content downloaded
quite helpful to keep a local project small and stay flexible in case you need a different version

the tool can be found on github https://github.com/kr-g/pygitgrab
and on pypi https://pypi.org/project/pygitgrab/

any feedback welcome. (guess there are still some bugs inside)
hope you like it :-)

Re: Announcing pygitgrab

Posted: Wed Dec 04, 2019 12:30 pm
by stijn
Looks nice. So this is like "pip -e" (which also works from local directories, not clear if you have that), but with filtering of files?

Re: Announcing pygitgrab

Posted: Wed Dec 04, 2019 1:06 pm
by kr-g
its not a replacement for pip or git. its also not dealing on package level. it just for handling files.
so its doing "less" than pip. pip installs also not files into my project folder (at least as far as i know...)
in principle its possible to bring in e.g. MicroWebSrv2 also as git submodule but then my project blows up and syncing only the relevant files to my board consumes also some effort.
anyway there are plenty of possibilties to bring code on a board or organize development.
this one is just one more. ;-)

Re: Announcing pygitgrab

Posted: Wed Dec 04, 2019 1:19 pm
by jedie
Nice...

Maybe it's a way to replace https://github.com/rdehuyss/micropython-ota-updater ? This Project used github API, but the json are so big, that it won't run on a small ESP8266

Re: Announcing pygitgrab

Posted: Wed Dec 04, 2019 1:38 pm
by kr-g
you are addressing a different problem
what i do also have with esp8266 and even esp32
pygitgrab just deals how to bring 3rd party source your project folder.
i was using mpfshell (along with an own script) to deploy to my board.
sadly mpfshell hangs or fails by encountering hardreset 's during the 'put'
recently i therefore developed an extra tool (https://github.com/kr-g/mpycntrl) which does nothing more than sitting on the serial interface and handing over files to my board. this worked fine so far. its slower than mpfshell but when i increase the blocks size its ok.

but coming back to your issue. i already thought to enable this tool for using web-repl (what is already in place) so that it would be possible to send over this channel. also controlled by own code, since installation might need to do some more than just downloading source code.
but that would be a different approach then OTA, since one needs to trigger the installation in the sense of pushing a button.
automatic OTA is great, but i dont know if it fits all "cases".
probably there are some yet undiscovered other tools...

Re: Announcing pygitgrab

Posted: Wed Dec 04, 2019 2:02 pm
by jedie
My own source code OTA update works fine and is fast, see: viewtopic.php?f=2&t=7300#p41765
But it sends only local files to the device... That's great for developing.

But i think it's not "end user friendly" ;)

The idea behind https://github.com/rdehuyss/micropython-ota-updater is to grab the source from github directy.
Theoretically this is super user-friendly. It's possible to add a "update" button on the device-web-page and the OTA runs... Or something like a long-press on device button. Because a end user must not have a clone of the sources and runs a OTA-Server.

Think https://github.com/rdehuyss/micropython-ota-updater can run on a ESP32 with more RAM then the ESP8266...

But maybe this pygitgrab use less RAM?

Re: Announcing pygitgrab

Posted: Wed Dec 04, 2019 2:07 pm
by kr-g
pygitgrab is _NOT_ supporting OTA !!!
sorry for that confusion - i mixed 2 tools (and an outlook for next development) in my former post.
pygitgrab is just collecting files from github and saves them your local project folder.
the purpose is just to keep the project itself small
the next step like OTA, or sync to a board is _NOT_ covered

but thanks for sending the link to your tool, i will have a look at it lateron