writing files to /sd goes wrong

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
klankschap
Posts: 16
Joined: Tue Feb 02, 2016 11:20 am

writing files to /sd goes wrong

Post by klankschap » Mon Feb 22, 2016 9:39 pm

While trying to log some data into /sd/log.csv by appending samples

Code: Select all

fp = open('log.csv', 'a')

the file is created and filled with samples, so it seems.

Code: Select all

os.listdir()
will list the file.
However it remains invisible when checking the sd storage from OSX.
No sync() will help
Sometimes it shows later with corrupted content, sometimes it remains invisible, even after remounting.
Any hints?
.F

User avatar
Roberthh
Posts: 3667
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: writing files to /sd goes wrong

Post by Roberthh » Tue Feb 23, 2016 6:59 am

That's a mess I constantly fight with. It results from the fact, that both the host and the PyBoard have a different view of what is in the filesystem. The host will eventually 'sync' the files on the PyBoard and then write to it what it's view of the content. When you are writing files locally on PyBoard, In my experience the only safe way seems to be to disable mass storage mode on PyBoard in boot.py.
WiPy is more robust, since the file system is accessed by ftp, where WiPy is the master of files.
Maybe for mass storage a different protocol would be more safe, like mtp. But I have no idea whether that is available, and under which conditions or licenses. And mtp has it's own problems. There seem to be quite a few incompatible implementations in the world.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: writing files to /sd goes wrong

Post by dhylands » Tue Feb 23, 2016 7:55 am

The real issue is that you should disable mass storage if you're planning on writing to the sdcard from the pyboard.
You can disable mass storage by adding the line:

Code: Select all

pyb.usb_mode('CDC') # act as a serial only
to your boot.py (and commenting out any existing pyb.usb_mode call).

Mas Storage assumes that while the pyboard is mounted on the Mac, that the Mac has exclusive access to the file system.

I wrote a tool call rshell which allows files to be copied into and out of the pyboard filesystem without needing to use mass storage.

The released version of rshell has a minor issue on the Mac, but you can edit the installed main.py script and comment out one line and it will then work on the Mac. The line you need to comment is this one:
https://github.com/dhylands/rshell/blob ... n.py#L2234
(the call to autoconnect).

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: writing files to /sd goes wrong

Post by pythoncoder » Tue Feb 23, 2016 8:19 am

In my view @dhylands' approach is the way to go. rshell is a powerful program and its options are well worth exploring.
Peter Hinch
Index to my micropython libraries.

klankschap
Posts: 16
Joined: Tue Feb 02, 2016 11:20 am

Re: writing files to /sd goes wrong

Post by klankschap » Tue Feb 23, 2016 9:35 am

dhylands wrote:The released version of rshell has a minor issue on the Mac, but you can edit the installed main.py script and comment out one line and it will then work on the Mac. The line you need to comment is this one:
https://github.com/dhylands/rshell/blob ... n.py#L2234
(the call to autoconnect).
that seems a nice tool.
I did install it using pip
but when i run it, or import it, i get an error:

Code: Select all

In [1]: import rshell
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-fd3019c8af2a> in <module>()
----> 1 import rshell

/Users/vm/Desktop/pyboard/rshell.py in <module>()
     12 # from __future__ import print_function
     13 
---> 14 from rshell.getch import getch
     15 from rshell.pyboard import Pyboard
     16 

ImportError: No module named 'rshell.getch'; 'rshell' is not a package

Then i installed it from the github distort using

Code: Select all

sudo python setup.py install
Now i get a different error:

Code: Select all

$ rshell 
Connecting to /dev/cu.usbmodem262442 ...
Traceback (most recent call last):
  File "/Users/vm/anaconda/bin/rshell", line 9, in <module>
    load_entry_point('rshell==0.0.1', 'console_scripts', 'rshell')()
  File "/Users/vm/anaconda/lib/python3.5/site-packages/rshell-0.0.1-py3.5.egg/rshell/command_line.py", line 4, in main
    rshell.main.main()
  File "/Users/vm/anaconda/lib/python3.5/site-packages/rshell-0.0.1-py3.5.egg/rshell/main.py", line 2234, in main
    autoconnect()
  File "/Users/vm/anaconda/lib/python3.5/site-packages/rshell-0.0.1-py3.5.egg/rshell/main.py", line 178, in autoconnect
    context = pyudev.Context()
  File "/Users/vm/anaconda/lib/python3.5/site-packages/pyudev/core.py", line 65, in __init__
    self._libudev = load_udev_library()
  File "/Users/vm/anaconda/lib/python3.5/site-packages/pyudev/_ctypeslib/libudev.py", line 284, in load_udev_library
    raise ImportError('No library named udev')
ImportError: No library named udev
Exception ignored in: <bound method Context.__del__ of <pyudev.core.Context object at 0x1013aba58>>
Traceback (most recent call last):
  File "/Users/vm/anaconda/lib/python3.5/site-packages/pyudev/core.py", line 69, in __del__
    self._libudev.udev_unref(self)
AttributeError: 'Context' object has no attribute '_libudev'
hints?

User avatar
platforma
Posts: 258
Joined: Thu May 28, 2015 5:08 pm
Location: Japan

Re: writing files to /sd goes wrong

Post by platforma » Tue Feb 23, 2016 12:05 pm

Did you do what Dave already suggested?
The line you need to comment is this one:
https://github.com/dhylands/rshell/blob ... n.py#L2234
(the call to autoconnect).

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: writing files to /sd goes wrong

Post by dhylands » Tue Feb 23, 2016 2:55 pm

The second error needs the autoconnect line commented out. I also think it needs python3 so I'll need to figure out how to check for that.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: writing files to /sd goes wrong

Post by pythoncoder » Tue Feb 23, 2016 6:03 pm

I rather think 3.4 is the minimum.
Peter Hinch
Index to my micropython libraries.

User avatar
ynohtna
Posts: 4
Joined: Thu Jan 07, 2016 7:11 pm

Re: writing files to /sd goes wrong

Post by ynohtna » Tue Feb 23, 2016 8:15 pm

dhylands wrote:I wrote a tool call rshell which allows files to be copied into and out of the pyboard filesystem without needing to use mass storage.

The released version of rshell has a minor issue on the Mac, but you can edit the installed main.py script and comment out one line and it will then work on the Mac. The line you need to comment is this one:
https://github.com/dhylands/rshell/blob ... n.py#L2234
(the call to autoconnect).
Here's a pull request for you, Dave, that fixes the OSX issue: https://github.com/dhylands/rshell/pull/4

klankschap
Posts: 16
Joined: Tue Feb 02, 2016 11:20 am

Re: writing files to /sd goes wrong

Post by klankschap » Tue Feb 23, 2016 9:55 pm

platforma wrote:Did you do what Dave already suggested?
Thanks for pointing out, that made a difference ;-)

.F

Post Reply