Accessing the SD card

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
buteman
Posts: 6
Joined: Mon Mar 10, 2014 9:41 am

Accessing the SD card

Post by buteman » Tue Mar 11, 2014 9:38 pm

I am intending to use my board ( which i do not have yet ) as the front end to a simple oscilloscope.
What I am hoping I will be able to do is write code on the board which controls one or more of the ADC's and gathers the readings and stores them in a file on the SD card.
I have written code which runs on my laptop and, at present, reads data from a text file I have generated using python's random.
I can get it to display a dynamic plot of the data which looks just like the display on an oscilloscope.
I can select how many points to display and how long it waits between screen updates. Also which of the points it starts from.
I can also display a static plot and this can display as many points as I choose and which point to start from.
I believe I can simply write code to take the data from the file the board generates and initially I will be happy to use the board to grab a certain amount of data then use the laptop to examine it 'offline'.
What I am hoping to do is develop some protocol, probably using files written to the SD card, to tell the board how long to sample for and then carry out the task then somehow signal that it has finished.

For anyone interested here https://app.box.com/s/4171nkmydw5glfostei6 is a screenshot of what I have so far.
This is the code which produces it: https://app.box.com/s/lx8htg3t48kzmctk2lsb
As I am not a programmer any suggestions for improvement are most welcome.

All of this running on Ubuntu 13.10 and python 2.7.5+

Now there are a few unknowns at present as far as I an concerned.
1/ how fast the data can be gathered.
2/ how long it will take to write the data to the SD card.
3/ If reading the data from the laptop causes or needs any interaction from the board.
and more but if anyone can give me some pointers I would appreciate it,
thanks

torwag
Posts: 209
Joined: Fri Dec 13, 2013 9:25 am

Re: Accessing the SD card

Post by torwag » Wed Mar 12, 2014 6:11 pm

Hi,

I can see some challenges in your approach which you might want to consider:

1. Writing to SD Card is relatively slow. An oscilloscope would require you to e.g. write 1 million samples / sec to the SD card reading them in by 12 bits would make (taken a 16 bit value for storage) 1.9 MB/sec of binary data without any overhead. If I understood your code, you plan to write the data in ASCII which would be equal to be 4 byte for the data and at least one byte for a separator, resulting in 4.7MB/sec .
Thus, if you have no need to store the data locally (e.g. recording in the absence of a PC connection). I would suggest to stream the data directly to your python program via USB resp. via serial.
2. If I remember correctly, I think there is not yet a solution how to deal with read-write access to the SD card from both sides (PC and micropython). Basically, only one device should have access to the SD card at any time to avoid data and file system corruption, e.g. what would happen if the PC starts reading a several MB large file and in-between it would be deleted by micropython and other data would be written in place. This is something which still needs a solution.

Enjoy your project and keep us updated.

Torsten

buteman
Posts: 6
Joined: Mon Mar 10, 2014 9:41 am

Re: Accessing the SD card

Post by buteman » Wed Mar 12, 2014 8:20 pm

Hi Torsten,
Thanks for your input.
I realise that getting it working so as to be viewing it so quickly after taking readings could be a problem.
If you look at my code so far you will see it is only designed to display data from the file but reading via USB
could be a better way even to do what I am aiming for.
Perhaps I could send an instruction to micropython telling it how many samples to take then read them in and write them to a file on my PC. No doubt I will need to develop some code to do this but I don't see that as being a major problem.
After that the rest of my code could still do what it does now.
I am looking at expanding that code so that if the readings are all of a similar value I can choose an option which would find minimum and maximum values then display a shifted subset.
I'm probably not explaining this very well, say values are between 1000 and 1050 then displaying them raw would not show much useful but subtracting say 990 from the results and then displaying the new 'values' would, in effect magnify them.
As it stands I am happy to get something like this working as an initial project and it will still be better than having nothing other than a digital multimeter and a low range frequency meter which is all I have at present.
Aside from which doing the project has already given me a lot of pleasure.
Norman

buteman
Posts: 6
Joined: Mon Mar 10, 2014 9:41 am

Re: Accessing the SD card

Post by buteman » Fri Mar 14, 2014 6:04 pm

Okay,
I expect that I will use serial over USB then. I assume that this is implemented in micropython so it shouldn't be a problem doing the comms. Looking at the specs for pySerial I came across this:
The parameter baudrate can be one of the standard values: 50, 75, 110, 134, 150, 200, 300, 600, 1200, 1800, 2400, 4800, 9600, 19200, 38400, 57600, 115200. These are well supported on all platforms.

Standard values above 115200, such as: 230400, 460800, 500000, 576000, 921600, 1000000, 1152000, 1500000, 2000000, 2500000, 3000000, 3500000, 4000000 also work on many platforms and devices.
So I should be able to get a reasonable throughput.

User avatar
JonHylands
Posts: 64
Joined: Sun Dec 29, 2013 1:33 am

Re: Accessing the SD card

Post by JonHylands » Fri Mar 14, 2014 9:02 pm

buteman wrote:Okay,
I expect that I will use serial over USB then. I assume that this is implemented in micropython so it shouldn't be a problem doing the comms. Looking at the specs for pySerial I came across this: (snip) So I should be able to get a reasonable throughput.
I talk to an AVR through pyserial (over a USB virtual serial port) at 1 megabit (1000000 baud) all the time, with no issues. On Linux, Windows, and Mac.

- Jon

bittware
Posts: 16
Joined: Mon Aug 18, 2014 3:27 am

Re: Accessing the SD card

Post by bittware » Sat Sep 13, 2014 2:15 am

buteman wrote: For anyone interested here https://app.box.com/s/4171nkmydw5glfostei6 is a screenshot of what I have so far.
This is the code which produces it: https://app.box.com/s/lx8htg3t48kzmctk2lsb
As I am not a programmer any suggestions for improvement are most welcome.
Hi buteman,
I'm interested in your implementation of this idea. The links are broken. Could you post them again. Thanks.

User avatar
dhylands
Posts: 3212
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Accessing the SD card

Post by dhylands » Sat Sep 13, 2014 8:16 am

I just clicked on both links and they both seem to be working fine for me.

Post Reply