pybkick - tools to make deploying code to pyboard easier

Discussion about programs, libraries and tools that work with MicroPython. Mostly these are provided by a third party.
Target audience: All users and developers of MicroPython.
salimfadhley
Posts: 22
Joined: Thu Jun 19, 2014 10:18 pm

Re: pybkick - tools to make deploying code to pyboard easier

Post by salimfadhley » Sun Oct 19, 2014 11:27 am

do a reset after files are transfered to run the code
Yeah, that was always part of the plan. You could specify an "entry point", basically something callable to invoke from main.py. It would adjust the main.py file to ensure that this happens on reset.
show files on pyboard am sd card
I was thinking, much further downstream that there should be a Pyboard FUSE filesystem that works over serial, not USB Mass Storage.
some kind of debugging the code (for example copy files, start connection to /dev/ttyASM0 and import main.py)
Being able to remote debug would we awesome!

User avatar
shell
Posts: 15
Joined: Tue Aug 12, 2014 8:31 pm
Location: Germany
Contact:

Re: pybkick - tools to make deploying code to pyboard easier

Post by shell » Sun Oct 19, 2014 11:51 am

A soft reset can be done with something like this on the command line:

echo -e "\0004" >> /dev/ttyACM0

May be it helps.

User avatar
dhylands
Posts: 3261
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: pybkick - tools to make deploying code to pyboard easier

Post by dhylands » Sun Oct 19, 2014 6:13 pm

I also proposed soft and hard reset commands to be added to pyb.

https://github.com/micropython/micropython/pull/826

So it would be useful to add a comment on that thread indicating that they would be useful (more likely that they'll get merged if multiple people want the functionality)

User avatar
shell
Posts: 15
Joined: Tue Aug 12, 2014 8:31 pm
Location: Germany
Contact:

Re: pybkick - tools to make deploying code to pyboard easier

Post by shell » Mon Oct 20, 2014 7:30 pm

I get the following erro runnin pybkick:
[shell@xolotl:~/git/pcspinnt/pyboard] (master)$ pybkick --src dev

Traceback (most recent call last):
File "/usr/lib64/python3.3/site-packages/serial/serialposix.py", line 475, in read
raise SerialException('device reports readiness to read but returned no data (device disconnected or multiple access on port?)')
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/bin/pybkick", line 9, in <module>
load_entry_point('pybkick==0.0.2', 'console_scripts', 'pybkick')()
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 58, in main
entry_point = opts.entry_point
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 22, in kick
with pb.raw_repl():
File "/usr/lib64/python3.3/contextlib.py", line 48, in __enter__
return next(self.gen)
File "/usr/lib64/python3.3/site-packages/pybkick/pyboard.py", line 81, in raw_repl
self.enter_raw_repl()
File "/usr/lib64/python3.3/site-packages/pybkick/pyboard.py", line 68, in enter_raw_repl
data = self.read_until(1, b'to exit\r\n>')
File "/usr/lib64/python3.3/site-packages/pybkick/pyboard.py", line 47, in read_until
data = self.serial.read(min_num_bytes)
File "/usr/lib64/python3.3/site-packages/serial/serialposix.py", line 480, in read
if e[0] != errno.EAGAIN:
TypeError: 'SerialException' object does not support indexing

User avatar
dhylands
Posts: 3261
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: pybkick - tools to make deploying code to pyboard easier

Post by dhylands » Mon Oct 20, 2014 9:20 pm

I seem to recall seeing something like that if I try and execute a pyb.bootloader() command.

Basically, I think what's happening is that the serial port gets closed and pyboard.py isn't dealing with that properly.

You should wrap the read in a try/except block, like https://github.com/dhylands/usb-ser-mon ... #L121-L127

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: pybkick - tools to make deploying code to pyboard easier

Post by pfalcon » Mon Oct 20, 2014 11:06 pm

I'd also like to be able to make a 'sdist_pyboard' setuptools extension that automates pushing code to the pyboard in one shot from a python project.
For the next feature, I was thinking of making a micropython-lib installer.

It would be able to install/update specfic modules from micropython-lib directly onto the pyboard, for example:

CODE: SELECT ALL
kick_micropython_lib(['os', 'json']


Would download the appropriate modules from https://github.com/micropython/micropython-lib, and then transfer them over to the pyboard.

Would anybody actually use this kind of feature?


Before you spend too much time on that, I'd suggest to study "prior art", however little we have of it in that regard. So, how I envisioned deployment process for uPy applications: There're developed on a host machine in comfort. When they're ready to be pushed to target, you run pip-micropython tool in a special mode which creates snapshot of all modules used by an app in a subdir. Then you just copy files to target by whatever means you have/like - be it cp, scp, rsync or whatever. Then you just run with sys.path properly configured. If you need to automate this, shell and make are your best friends.

I made (and announced this on forum) couple of apps made in this vein:
They're made not for pyboard, but for embedded linux systems, but then the only difference is how you transfer files to the target (and well, maybe the way to setup sys.path - on POSIX system, this can be done with MICROPYPATH environment var, on pyboard - in different, perhaps less convenient way, but then it's something to improve for pyboard).


I'm personally very happy with this process. What I'm thinking about further, is how to make PyPI modules installable right on target. I'm not going to port pip to uPy, so it needs to be rewritten from scratch (to do only what's needed to do of course). That's why I'm working on builtin zlib module for uPy (to be able to uncompress tarballs). But then again it's more useful fot bigger systems like Linux, for pyboard, the process above should be good enough - the only thing to elaborate and optimize is means to transfer files to pyboard.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

salimfadhley
Posts: 22
Joined: Thu Jun 19, 2014 10:18 pm

Re: pybkick - tools to make deploying code to pyboard easier

Post by salimfadhley » Tue Oct 21, 2014 10:32 pm

I see what you mean - replacing micropython-pip is not a good idea.

I think it would be better to focus on making this more reliable and faster.

salimfadhley
Posts: 22
Joined: Thu Jun 19, 2014 10:18 pm

Re: pybkick - tools to make deploying code to pyboard easier

Post by salimfadhley » Tue Oct 21, 2014 10:34 pm

Did you ever find a solution to this? The pybkick script assumes that the pyboard is in a normal (not raw) repl. It might be that a simple factory reset will be good enough to make this error go away.
shell wrote:I get the following erro runnin pybkick:
[shell@xolotl:~/git/pcspinnt/pyboard] (master)$ pybkick --src dev

Traceback (most recent call last):
File "/usr/lib64/python3.3/site-packages/serial/serialposix.py", line 475, in read
raise SerialException('device reports readiness to read but returned no data (device disconnected or multiple access on port?)')
serial.serialutil.SerialException: device reports readiness to read but returned no data (device disconnected or multiple access on port?)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/bin/pybkick", line 9, in <module>
load_entry_point('pybkick==0.0.2', 'console_scripts', 'pybkick')()
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 58, in main
entry_point = opts.entry_point
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 22, in kick
with pb.raw_repl():
File "/usr/lib64/python3.3/contextlib.py", line 48, in __enter__
return next(self.gen)
File "/usr/lib64/python3.3/site-packages/pybkick/pyboard.py", line 81, in raw_repl
self.enter_raw_repl()
File "/usr/lib64/python3.3/site-packages/pybkick/pyboard.py", line 68, in enter_raw_repl
data = self.read_until(1, b'to exit\r\n>')
File "/usr/lib64/python3.3/site-packages/pybkick/pyboard.py", line 47, in read_until
data = self.serial.read(min_num_bytes)
File "/usr/lib64/python3.3/site-packages/serial/serialposix.py", line 480, in read
if e[0] != errno.EAGAIN:
TypeError: 'SerialException' object does not support indexing

User avatar
shell
Posts: 15
Joined: Tue Aug 12, 2014 8:31 pm
Location: Germany
Contact:

Re: pybkick - tools to make deploying code to pyboard easier

Post by shell » Sun Oct 26, 2014 1:26 pm

Sorry for the late answer:

I get a new error:
1 [shell@xolotl:~/git/pcspinnt/pyboard] (master)$ pybkick --src dev
Traceback (most recent call last):
File "/usr/bin/pybkick", line 9, in <module>
load_entry_point('pybkick==0.0.2', 'console_scripts', 'pybkick')()
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 58, in main
entry_point = opts.entry_point
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 27, in kick
kick_single_file(pb, file_path, src, dst)
File "/usr/lib64/python3.3/site-packages/pybkick/kick.py", line 32, in kick_single_file
target_path = posixpath.join(dst, rel_path)
File "/usr/lib64/python3.3/posixpath.py", line 83, in join
path += b
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

salimfadhley
Posts: 22
Joined: Thu Jun 19, 2014 10:18 pm

Re: pybkick - tools to make deploying code to pyboard easier

Post by salimfadhley » Sun Oct 26, 2014 1:41 pm

What was the command line settings or function arguments you used?

Post Reply