Exception handling with files

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
Post Reply
Abraxas
Posts: 15
Joined: Mon Jan 01, 2018 5:53 pm

Exception handling with files

Post by Abraxas » Tue Jan 09, 2018 5:28 pm

Below is some code I wrote to log punches. Also included is some code to handle errors that might occur if a file cannot be read for whatever reason. However when I run this code the files seem to be corrupted (I am basing this off of some of the weird characters inputed to the file) or the file does not save leaving a blank file still. One observation noticed is if I eliminate the error handling the code works as it should. What am I doing wrong here?

Code: Select all

"""
logPunches(fileName)
- a function that opens, reads, appends data to, and rewrites a file given fileName
- returns nothing
"""
def logPunches(fileName):
    # Define variables
    time = []
    time = clock.datetime()
    log = []
    monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

    # See if there is a file to read
    try:
        readLog = open(fileName,'r')
        for line in readLog:
            log.append(line)
            print("Read line: {0}".format(line))
        readLog.close()
        print()
        print(log)
    # If no file found, do nothing
    except:
        pass
    
    # Append new entry to log
    log.append('{0}:{1},{2}{3},\n'.format(time[4], time[5], monthNames[time[1]-1], time[2]))
    print(log)
    
    # Replace/create file with an updated file using log
    writeLog = open(fileName, 'w')
    for line in range(len(log)):
        writeLog.write(log[line])
    writeLog.close()

User avatar
deshipu
Posts: 1316
Joined: Thu May 28, 2015 5:54 pm

Re: Exception handling with files

Post by deshipu » Tue Jan 09, 2018 6:06 pm

I'm not sure what is happening there exactly, but you might want to use "try-finally" or "with" to make sure the files are always closed, even when an exception happens. It's also a good idea to put the minimum amount of code in the "try" section, catch only the one specific exception that you expect, and put the rest in an "else" clause of the "try".

User avatar
pythoncoder
Posts: 2706
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Exception handling with files

Post by pythoncoder » Tue Jan 09, 2018 6:34 pm

While agreeing 100% with the above, another potential source of corruption is USB MSC (mass storage) mode. This is designed on the basis that mass storage devices are dumb devices like hard disks rather than active devices whose contents can change regardless of the connected PC. I always disable MSC mode in boot.py and use rshell to maintain the Pyboard filesystem.

Code: Select all

# boot.py -- run on boot-up
import pyb
pyb.usb_mode('CDC') # act as a serial (CDC) and not a storage device (MSC)
Peter Hinch

Abraxas
Posts: 15
Joined: Mon Jan 01, 2018 5:53 pm

Re: Exception handling with files

Post by Abraxas » Wed Jan 10, 2018 9:42 pm

Thanks for all your advice. I decided to try using a with statement (See code below.) since it sounded like the easiest to implement but ended up getting an "OSError: [Errno 2] ENOENT." I tried looking up what this meant but my searches came up with irrelevant information.

Code: Select all

"""
logPunches(fileName)
- a function that opens, reads, appends data to, and rewrites a file given fileName
- returns nothing
"""
def logPunches(fileName):
    # Define variables
    time = []
    time = clock.datetime()
    log = []
    monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

    # See if there is a file to read
    with open(fileName, 'r') as readLog:
        for line in readLog:
            log.append(line)
            print("Read line: {0}".format(line))
        
    print("I made it to this point")
    # Append new entry to log
    log.append('{0}:{1},{2}{3},\n'.format(time[4], time[5], monthNames[time[1]-1], time[2]))
    print(log)
    
    # Replace/create file with an updated file using log
    writeLog = open(fileName, 'w')
    for line in range(len(log)):
        writeLog.write(log[line])
    writeLog.close()
I then tried using the try statement once again with exception, else, and finally clauses (see code below). Using finally threw a "local variable referenced before assignment" error. I deleted the finally statement which fixed the error but caused the same corrupted data to show up.

Code: Select all

"""
logPunches(fileName)
- a function that opens, reads, appends data to, and rewrites a file given fileName
- returns nothing
"""
def logPunches(fileName):
    # Define variables
    time = []
    time = clock.datetime()
    log = []
    monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

    # See if there is a file to read
    try:
        readLog = open(fileName, 'r')
    except:
        print("Error reading file")
    else:
        for line in readLog:
            log.append(line)
            print("Read line: {0}".format(line))
    finally:
        readLog.close()
        
    print("I made it to this point")
    # Append new entry to log
    log.append('{0}:{1},{2}{3},\n'.format(time[4], time[5], monthNames[time[1]-1], time[2]))
    print(log)
    
    # Replace/create file with an updated file using log
    writeLog = open(fileName, 'w')
    for line in range(len(log)):
        writeLog.write(log[line])
    writeLog.close()
I think I'm still missing something here. Thoughts?

Thanks for the MSC suggestion. Disabling MSC might not be a bad idea but what I really need is for the pyboard to not open a non-existent file but instead create the file with the new data if the designated file doesn't exist. If the file does exist I want it to read the file add data and re-save the file.

User avatar
dhylands
Posts: 2789
Joined: Mon Jan 06, 2014 6:08 pm
Location: Shuswap, BC, Canada
Contact:

Re: Exception handling with files

Post by dhylands » Thu Jan 11, 2018 3:28 am

You can use os.stat to test if a file exists or not. Something like this:

Code: Select all

def fileExists(filename):
    try:
        os.stat(filename)
        return True;
    except OSError:
        return False

User avatar
Roberthh
Posts: 938
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: Exception handling with files

Post by Roberthh » Thu Jan 11, 2018 8:07 am

Please have also a look at the standard Python file open modes, as e.g. explained here: https://www.tutorialspoint.com/python/p ... les_io.htm. You might need the "w+" or "a+" mode.

efahl
Posts: 5
Joined: Sat Dec 23, 2017 2:02 pm

Re: Exception handling with files

Post by efahl » Thu Jan 11, 2018 4:25 pm

Looking at your code, it appears you simply want to append a timestamp to it? If that's the case, then there's no need to read the current contents, simply open it in append mode as Robert alludes. This probably won't help with the corruption issue, don't know what's going on there...

[code]import time
monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

def appendTimeStamp(fileName):
t = time.localtime(time.time())
timestamp = '{0}:{1},{2}{3},\n'.format(t[4], t[5], monthNames[t[1]-1], t[2])
try:
with open(fileName, 'a+') as logfile:
logfile.write(timestamp)
except OSError as exc:
print("Error writing to %r: %s" % (fileName, exc))
[/code]

Eric

(Sorry about the lack of post count => no bbcode.)

Abraxas
Posts: 15
Joined: Mon Jan 01, 2018 5:53 pm

Re: Exception handling with files

Post by Abraxas » Fri Jan 12, 2018 3:01 am

@Dave
Thanks so much! Your suggestion worked for checking to see if a file actually exists. Then I just use an if statement to condition whether reading is necessary. Code below:

Code: Select all

# Don't forget to import uos
def fileExists(fileName):
    try:
        uos.stat(fileName)
        print("File found!")
        return True
    except OSError:
        print("No file found!")
        return False
@Robberthh
@Eric
Thank you for the reference and suggestions. Appending would be easy and I'd be totally all for it but my project is more complicated than just creating a simple logging txt file. I guess it would help if I actually told you guys what it is I am trying to accomplish (I just realized I may have been a little vague on details). I want to allow multiple users to use the same pyboard to log data. Each user would have their own csv file(s) which can easily be selected with the push of a few buttons. Once a csv file is selected, the pyboard would record data in an organized manner. Thus a simple appending routine would not work if I want to keep information sorted.

That being said I still seem to be having some issues with data corruption but I think that needs to be another forum posting (I think I'll write that post in a day or two because I want to do some more testing) as the solution to the current posting has been found! Thanks for all your replies!

User avatar
dhylands
Posts: 2789
Joined: Mon Jan 06, 2014 6:08 pm
Location: Shuswap, BC, Canada
Contact:

Re: Exception handling with files

Post by dhylands » Fri Jan 12, 2018 5:51 pm

Having USB Mass Storage enabled at the same time as the pyboard writes to any files will eventually lead to corruption of your files.

Post Reply