OSError: [Errno 5]

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
nelfata
Posts: 74
Joined: Wed Apr 30, 2014 10:50 pm

OSError: [Errno 5]

Post by nelfata » Fri Jul 04, 2014 3:02 pm

I am having some issues reading large files (2MB) in small blocks on the SD card
After reading about 200K I get OSError: [Errno 5]
Accessing the SD card afterwards is impossible (form Windows), and Ctrl-D in the terminal hangs. Pressing Reset does not recover, 2 resets are needed.

I have attached the test file, just copy them onto the SD card and perform "import SDIO"
NOTE: create a large file (2MB) and copy it to the SD card as well (I could not uploaded it)

Please if someone had experienced this take a look into it and let me know. Thank you.
Attachments
SDIO.zip
(530 Bytes) Downloaded 570 times

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: OSError: [Errno 5]

Post by pfalcon » Fri Jul 04, 2014 5:04 pm

Please report this at bug tracker: https://github.com/micropython/micropython
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

nelfata
Posts: 74
Joined: Wed Apr 30, 2014 10:50 pm

Re: OSError: [Errno 5]

Post by nelfata » Sat Jul 05, 2014 6:35 pm

Sure, already done, issue #743

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: OSError: [Errno 5]

Post by dhylands » Mon Jul 07, 2014 10:05 pm

Just thought I'd report that this was fixed yesterday.

nelfata
Posts: 74
Joined: Wed Apr 30, 2014 10:50 pm

Re: OSError: [Errno 5]

Post by nelfata » Wed Oct 15, 2014 2:53 am

Hello again,
it seem that bug came back.
I am having the same read file issues. Please could someone try the zip file I sent earlier.
Changing the buffer size from 1024 to 512 gives another behavior.
With 1024 size, I get:
SmartCup>import SDIO
Bytes to read 25517
Read block 1024 so far 0 next 1024
Read block 1024 so far 1024 next 2048
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "SDIO.py", line 41, in <module>
File "SDIO.py", line 33, in Test
OSError: [Errno 5]


With 512 size:
...
Read block 512 so far 24576 next 25088
Read block 429 so far 25088 next 25517
Bytes to read 25517
Read block 512 so far 0 next 512
Read block 512 so far 512 next 1024

It reads all the file in chunks, when it reaches the end, I would expect an exception that it reached the end of file, but instead it start over.

Some simpler examples to try:

l=0
aFile = open('/flash/boot.py', 'r')
while True:
try:
a=aFile.read(512)
except:
print('error')
l = l + len(a)
print(l)

aFile.close()


l=0
aFile = open('/flash/boot.py', 'r')
while True:
try:
a=aFile.read(10)
except:
print('error')
l = l + len(a)
print(l)

aFile.close()

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: OSError: [Errno 5]

Post by dhylands » Wed Oct 15, 2014 3:46 am

I tried your original SDIO.py and changed the 'rU' passed to open to be 'rb', removed the sys.path.append("1://") line, and changed 1://data.bin to be /sd/data.bin.

If you open the file using 'r' then it will assume text mode (i.e. same as 'rt'). If you pass in a binary file, then you might get errors as part of the unicode parser. Using 'rb' opens the file in binary mode, which is how you should be doing ot for binary files.

I let it run for a few minutes and eventually hit Control-C:

Code: Select all

Read block 1024 so far 15511552 next 15512576
Read block 1024 so far 15512576 next 15513600
Read block 1024 so far 15513600 next 15514624
Read block 1024 so far 15514624 next 15515648
Read block 1024 so far 15515648 next 15516672
Read block 1024 so far 15516672 next 15517696
I also had some errors, but I had a corrupted sdcard. After running fsck on it and fixing it up, then everythng worked fine.

nelfata
Posts: 74
Joined: Wed Apr 30, 2014 10:50 pm

Re: OSError: [Errno 5]

Post by nelfata » Wed Oct 15, 2014 4:50 am

Ok, that changed some behavior.

Here is one to try:

l=0
aFile = open('/sd/cup.jpg', 'rb')
while True:
try:
a = aFile.read(512)
if a == '':
print('done')
break
except OSError:
print('OSError')
break
except :
print('Some error')
break
l = l + len(a)
print(l)

aFile.close()

---
You can see that after it reaches the end of the file, a is not breaking out of the loop and no exceptions occur, is this normal?
If I check for len(a) == 0, then it breaks out of the loop.
The file size I used is 25517 bytes.
Tried block size of 1024 with the same result, but block size 2048 causes an exception (possibly memory error).

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: OSError: [Errno 5]

Post by dhylands » Wed Oct 15, 2014 4:59 am

I tried a smaller file, 1012987 bytes long.

Code: Select all

import sys, pyb

def Test():

    print("About to open /sd/data.bin")
    aFile = open("/sd/data.bin",'rb')
    print("After opening /sd/data.bin")
    bytesToRead = aFile.seek(0,2)

    print('Bytes to read %d' % bytesToRead)

    bytesRead  = 0
    chunkCount = 0
    chunkSize  = 0
    blockSize  = 1024

    aFile.seek(0,0)

    # Read/send a file block by block
    while bytesRead < bytesToRead:

        if bytesToRead - bytesRead > blockSize:
            chunkSize = blockSize
        else:
            chunkSize = bytesToRead - bytesRead

        # Should never happen but if it does just exit the file read/send loop
        if chunkSize<0:
            break

        print('Read block %d so far %d next %d' % (chunkSize, bytesRead, bytesRead + chunkSize))

        # READ THE FILE
        chunkToRead = aFile.read(chunkSize)

        bytesRead  = bytesRead + chunkSize
        chunkCount = chunkCount + 1

    aFile.close()

while True:
    Test()
and it seems to be working fine.

Code: Select all

Read block 1024 so far 1010688 next 1011712
Read block 1024 so far 1011712 next 1012736
Read block 251 so far 1012736 next 1012987
About to open /sd/data.bin
After opening /sd/data.bin
Bytes to read 1012987
Read block 1024 so far 0 next 1024
Read block 1024 so far 1024 next 2048
Read block 1024 so far 2048 next 3072
Read block 1024 so far 3072 next 4096
Read block 1024 so far 4096 next 5120
What size was your file?

nelfata
Posts: 74
Joined: Wed Apr 30, 2014 10:50 pm

Re: OSError: [Errno 5]

Post by nelfata » Wed Oct 15, 2014 5:02 am

I wrote the size in my previous comment: 25517 bytes.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: OSError: [Errno 5]

Post by dhylands » Wed Oct 15, 2014 5:03 am

I also tried 2048, 4096, and 8192 and they all worked fine:

Code: Select all

Read block 8192 so far 991232 next 999424
Read block 8192 so far 999424 next 1007616
Read block 5371 so far 1007616 next 1012987
About to open /sd/data.bin
After opening /sd/data.bin
Bytes to read 1012987
Read block 8192 so far 0 next 8192
Read block 8192 so far 8192 next 16384
Read block 8192 so far 16384 next 24576
Read block 8192 so far 24576 next 32768

Post Reply