Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
Post Reply
jeffmakes
Posts: 1
Joined: Tue Jun 12, 2018 8:31 pm

Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by jeffmakes » Tue Jun 12, 2018 8:51 pm

Hi all,

I've just created a new tool that allows seamless code synchronisation from a serial REPL session.
Please check it out and let me know what you think
https://github.com/jeffmakes/mpy-miniterm

I just started working with MicroPython on an ESP-32 dev board, and was finding the existing options for downloading code onto the board to be rather clunky, especially given that I had to close my serial console application so that mpy-sync/ampy/etc could connect. Yes, I know mpfshell exists, but I found that pretty clunky too, particularly when working with a large project with many files.

It's essentially a modified version of PySerial's miniterm, with an added menu option to synchronise a local folder of code on your PC to the filesystem on the MicroPython device. Simply press Ctrl+T Ctrl-G and your code is downloaded to the target. Before transferring anything, mpy-miniterm hashes the files locally and on the remote to avoid downloading unmodified files. This really saves time when working with a large project with many packages, etc.

So far it's only been tested on Linux with an ESP-32 board, but should work fine on other platforms since the original miniterm works cross-platform. Let me know whether I'm right!

I should credit Nick Moore - I'm using his ReplControl class to manage the communication.

Cheers!
Jeff

shaoziyang
Posts: 195
Joined: Sun Apr 17, 2016 1:55 pm

Re: Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by shaoziyang » Wed Jun 13, 2018 1:23 pm

It is very useful.

cefn
Posts: 205
Joined: Tue Aug 09, 2016 10:58 am

Re: Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by cefn » Wed Jun 13, 2018 8:18 pm

If you care about it running on Windows, you might want to track my https://github.com/vgkits/vgkits-vangua ... initerm.py which has fixes for some major cross-platform issues in miniterm, ( https://github.com/pyserial/pyserial/pull/351 ).

You can test it out as 'vanguard shell' via https://pypi.org/project/vgkits-vanguard/ if you have access to a windows machine and want to compare behaviour (the forked version of miniterm is bundled into vgkits-vanguard). I say you might 'track' since Windows seems also to need the following change for line endings not to be junked on copy-paste into Command Prompt https://github.com/pyserial/pyserial/issues/353 and that will be landing in my version in a day or so.

As an alternative I have been puzzling about dropping out of the shell and having console commands which bundle named collections to the board (meaning the learner doesn't have to curate a single folder and know how all the necessary packages are laid out, and that we can sync in from multiple independent 'roots' like https://github.com/adafruit/Adafruit_Ci ... es/drivers ). Potentially there could be a watchdog option https://pypi.org/project/watchdog/ which leaves a daemon to halt, upload and restart anything from the bundles when the origin files are changed.

As a concept your approach sounds brilliant and I definitely want to test it. I've been puzzling about the same issues as part of educational resources I'm developing at http://vgkits.org for the Vanguard board ( https://vgkits.org/blog/projects/vanguard/ ) and this part of the workflow is key.

User avatar
pythoncoder
Posts: 2806
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by pythoncoder » Thu Jun 14, 2018 6:37 am

@jeffmakes An interesting approach.

But for MicroPython I long ago settled on rshell. Have you tried this? It supports copying and synchronisation and even supports direct editing of files on the target hardware using an editor of your choice. Here is how I invoke it on esp8266:

Code: Select all

rshell -p /dev/ttyUSB0 --editor nano --buffer-size=30 -a
Peter Hinch

cefn
Posts: 205
Joined: Tue Aug 09, 2016 10:58 am

Re: Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by cefn » Tue Jun 19, 2018 12:37 pm

Been trying to organise in my mind the process which I use, to be able to share it with learners and establish the simplest possible tool support for it.

It's something like the diagram below, with the key aspect for learners being the interaction between the editor (where you have 'banked' some python commands) and the REPL, where you can prove what happens when you execute those commands.

I am wondering whether it's better for the "exit to terminal" step (freeing the serial link for upload) is a worthwhile step to be explicit, and perhaps always associated with a restart to prevent false positives or false negatives to do with older modules or object instances still being referred to in the VM.

Reflecting on this has made me question the value of suggesting to the learner that they are still 'in the same REPL session' when perhaps they need to explicitly realise they are not (e.g. that's why any prior references have now disappeared, unless the steps to create them were banked into a main.py or imported module). @jeffmakes what is the cost you perceive for learners having to exit the REPL, trigger a recursive upload, and re-enter the REPL? What is the benefit of doing it from within a single tool? How impactful do you think would be the potential issues from e.g. a module already being loaded, then recursively rewritten on the board's filesystem, but not reloaded since the same VM session is running. Can you see the value for learners if there was a forced restart on entering/re-entering the REPL by default?

To be more explicit, I am wondering whether it makes sense for the 'Restart Board' flowchart node to be placed permanently before the 'Launch REPL' node. However, this would only be proposed as a default setting affect those using the vgkits-vanguard toolchain, when invoking 'vanguard shell' (as per https://pypi.org/project/vgkits-vanguard/ ) . I am not proposing this change at any other level.
interactive-debugging-flowchart.gif
interactive-debugging-flowchart.gif (12.16 KiB) Viewed 311 times

cefn
Posts: 205
Joined: Tue Aug 09, 2016 10:58 am

Re: Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by cefn » Tue Jun 19, 2018 1:17 pm

This version of the flowchart might make relationships clearer. Original graphml edited using yed is at https://gist.github.com/cefn/8927a98264 ... d8618175a7
interactive-debugging-flowchart.gif
interactive-debugging-flowchart.gif (19.83 KiB) Viewed 285 times

nickzoic
Posts: 1
Joined: Tue Feb 07, 2017 12:42 pm

Re: Announcing mpy-miniterm, a serial terminal with integrated file synchronisation

Post by nickzoic » Sun Jul 01, 2018 12:33 pm

[quote=jeffmakes post_id=28087 time=1528836660 user_id=4066]
I should credit Nick Moore - I'm using his ReplControl class to manage the communication.
[/quote]

Thanks! I'm glad you found it useful!

-----Nick

Post Reply