Page 1 of 3
OSError: [Errno 5]
Posted: Fri Jul 04, 2014 3:02 pm
by nelfata
I am having some issues reading large files (2MB) in small blocks on the SD card
After reading about 200K I get OSError: [Errno 5]
Accessing the SD card afterwards is impossible (form Windows), and Ctrl-D in the terminal hangs. Pressing Reset does not recover, 2 resets are needed.
I have attached the test file, just copy them onto the SD card and perform "import SDIO"
NOTE: create a large file (2MB) and copy it to the SD card as well (I could not uploaded it)
Please if someone had experienced this take a look into it and let me know. Thank you.
Re: OSError: [Errno 5]
Posted: Fri Jul 04, 2014 5:04 pm
by pfalcon
Re: OSError: [Errno 5]
Posted: Sat Jul 05, 2014 6:35 pm
by nelfata
Sure, already done, issue #743
Re: OSError: [Errno 5]
Posted: Mon Jul 07, 2014 10:05 pm
by dhylands
Just thought I'd report that this was fixed yesterday.
Re: OSError: [Errno 5]
Posted: Wed Oct 15, 2014 2:53 am
by nelfata
Hello again,
it seem that bug came back.
I am having the same read file issues. Please could someone try the zip file I sent earlier.
Changing the buffer size from 1024 to 512 gives another behavior.
With 1024 size, I get:
SmartCup>import SDIO
Bytes to read 25517
Read block 1024 so far 0 next 1024
Read block 1024 so far 1024 next 2048
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "SDIO.py", line 41, in <module>
File "SDIO.py", line 33, in Test
OSError: [Errno 5]
With 512 size:
...
Read block 512 so far 24576 next 25088
Read block 429 so far 25088 next 25517
Bytes to read 25517
Read block 512 so far 0 next 512
Read block 512 so far 512 next 1024
It reads all the file in chunks, when it reaches the end, I would expect an exception that it reached the end of file, but instead it start over.
Some simpler examples to try:
l=0
aFile = open('/flash/boot.py', 'r')
while True:
try:
a=aFile.read(512)
except:
print('error')
l = l + len(a)
print(l)
aFile.close()
l=0
aFile = open('/flash/boot.py', 'r')
while True:
try:
a=aFile.read(10)
except:
print('error')
l = l + len(a)
print(l)
aFile.close()
Re: OSError: [Errno 5]
Posted: Wed Oct 15, 2014 3:46 am
by dhylands
I tried your original SDIO.py and changed the 'rU' passed to open to be 'rb', removed the sys.path.append("1://") line, and changed 1://data.bin to be /sd/data.bin.
If you open the file using 'r' then it will assume text mode (i.e. same as 'rt'). If you pass in a binary file, then you might get errors as part of the unicode parser. Using 'rb' opens the file in binary mode, which is how you should be doing ot for binary files.
I let it run for a few minutes and eventually hit Control-C:
Code: Select all
Read block 1024 so far 15511552 next 15512576
Read block 1024 so far 15512576 next 15513600
Read block 1024 so far 15513600 next 15514624
Read block 1024 so far 15514624 next 15515648
Read block 1024 so far 15515648 next 15516672
Read block 1024 so far 15516672 next 15517696
I also had some errors, but I had a corrupted sdcard. After running fsck on it and fixing it up, then everythng worked fine.
Re: OSError: [Errno 5]
Posted: Wed Oct 15, 2014 4:50 am
by nelfata
Ok, that changed some behavior.
Here is one to try:
l=0
aFile = open('/sd/cup.jpg', 'rb')
while True:
try:
a = aFile.read(512)
if a == '':
print('done')
break
except OSError:
print('OSError')
break
except :
print('Some error')
break
l = l + len(a)
print(l)
aFile.close()
---
You can see that after it reaches the end of the file, a is not breaking out of the loop and no exceptions occur, is this normal?
If I check for len(a) == 0, then it breaks out of the loop.
The file size I used is 25517 bytes.
Tried block size of 1024 with the same result, but block size 2048 causes an exception (possibly memory error).
Re: OSError: [Errno 5]
Posted: Wed Oct 15, 2014 4:59 am
by dhylands
I tried a smaller file, 1012987 bytes long.
Code: Select all
import sys, pyb
def Test():
print("About to open /sd/data.bin")
aFile = open("/sd/data.bin",'rb')
print("After opening /sd/data.bin")
bytesToRead = aFile.seek(0,2)
print('Bytes to read %d' % bytesToRead)
bytesRead = 0
chunkCount = 0
chunkSize = 0
blockSize = 1024
aFile.seek(0,0)
# Read/send a file block by block
while bytesRead < bytesToRead:
if bytesToRead - bytesRead > blockSize:
chunkSize = blockSize
else:
chunkSize = bytesToRead - bytesRead
# Should never happen but if it does just exit the file read/send loop
if chunkSize<0:
break
print('Read block %d so far %d next %d' % (chunkSize, bytesRead, bytesRead + chunkSize))
# READ THE FILE
chunkToRead = aFile.read(chunkSize)
bytesRead = bytesRead + chunkSize
chunkCount = chunkCount + 1
aFile.close()
while True:
Test()
and it seems to be working fine.
Code: Select all
Read block 1024 so far 1010688 next 1011712
Read block 1024 so far 1011712 next 1012736
Read block 251 so far 1012736 next 1012987
About to open /sd/data.bin
After opening /sd/data.bin
Bytes to read 1012987
Read block 1024 so far 0 next 1024
Read block 1024 so far 1024 next 2048
Read block 1024 so far 2048 next 3072
Read block 1024 so far 3072 next 4096
Read block 1024 so far 4096 next 5120
What size was your file?
Re: OSError: [Errno 5]
Posted: Wed Oct 15, 2014 5:02 am
by nelfata
I wrote the size in my previous comment: 25517 bytes.
Re: OSError: [Errno 5]
Posted: Wed Oct 15, 2014 5:03 am
by dhylands
I also tried 2048, 4096, and 8192 and they all worked fine:
Code: Select all
Read block 8192 so far 991232 next 999424
Read block 8192 so far 999424 next 1007616
Read block 5371 so far 1007616 next 1012987
About to open /sd/data.bin
After opening /sd/data.bin
Bytes to read 1012987
Read block 8192 so far 0 next 8192
Read block 8192 so far 8192 next 16384
Read block 8192 so far 16384 next 24576
Read block 8192 so far 24576 next 32768