Exception handling with files

[Abraxas]

Below is some code I wrote to log punches. Also included is some code to handle errors that might occur if a file cannot be read for whatever reason. However when I run this code the files seem to be corrupted (I am basing this off of some of the weird characters inputed to the file) or the file does not save leaving a blank file still. One observation noticed is if I eliminate the error handling the code works as it should. What am I doing wrong here?

Code: Select all

"""
logPunches(fileName)
- a function that opens, reads, appends data to, and rewrites a file given fileName
- returns nothing
"""
def logPunches(fileName):
    # Define variables
    time = []
    time = clock.datetime()
    log = []
    monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

    # See if there is a file to read
    try:
        readLog = open(fileName,'r')
        for line in readLog:
            log.append(line)
            print("Read line: {0}".format(line))
        readLog.close()
        print()
        print(log)
    # If no file found, do nothing
    except:
        pass
    
    # Append new entry to log
    log.append('{0}:{1},{2}{3},\n'.format(time[4], time[5], monthNames[time[1]-1], time[2]))
    print(log)
    
    # Replace/create file with an updated file using log
    writeLog = open(fileName, 'w')
    for line in range(len(log)):
        writeLog.write(log[line])
    writeLog.close()

[deshipu]

I'm not sure what is happening there exactly, but you might want to use "try-finally" or "with" to make sure the files are always closed, even when an exception happens. It's also a good idea to put the minimum amount of code in the "try" section, catch only the one specific exception that you expect, and put the rest in an "else" clause of the "try".

[pythoncoder]

While agreeing 100% with the above, another potential source of corruption is USB MSC (mass storage) mode. This is designed on the basis that mass storage devices are dumb devices like hard disks rather than active devices whose contents can change regardless of the connected PC. I always disable MSC mode in boot.py and use rshell to maintain the Pyboard filesystem.

Code: Select all

# boot.py -- run on boot-up
import pyb
pyb.usb_mode('CDC') # act as a serial (CDC) and not a storage device (MSC)

[Abraxas]

Thanks for all your advice. I decided to try using a with statement (See code below.) since it sounded like the easiest to implement but ended up getting an "OSError: [Errno 2] ENOENT." I tried looking up what this meant but my searches came up with irrelevant information.

Code: Select all

"""
logPunches(fileName)
- a function that opens, reads, appends data to, and rewrites a file given fileName
- returns nothing
"""
def logPunches(fileName):
    # Define variables
    time = []
    time = clock.datetime()
    log = []
    monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

    # See if there is a file to read
    with open(fileName, 'r') as readLog:
        for line in readLog:
            log.append(line)
            print("Read line: {0}".format(line))
        
    print("I made it to this point")
    # Append new entry to log
    log.append('{0}:{1},{2}{3},\n'.format(time[4], time[5], monthNames[time[1]-1], time[2]))
    print(log)
    
    # Replace/create file with an updated file using log
    writeLog = open(fileName, 'w')
    for line in range(len(log)):
        writeLog.write(log[line])
    writeLog.close()

I then tried using the try statement once again with exception, else, and finally clauses (see code below). Using finally threw a "local variable referenced before assignment" error. I deleted the finally statement which fixed the error but caused the same corrupted data to show up.

Code: Select all

"""
logPunches(fileName)
- a function that opens, reads, appends data to, and rewrites a file given fileName
- returns nothing
"""
def logPunches(fileName):
    # Define variables
    time = []
    time = clock.datetime()
    log = []
    monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

    # See if there is a file to read
    try:
        readLog = open(fileName, 'r')
    except:
        print("Error reading file")
    else:
        for line in readLog:
            log.append(line)
            print("Read line: {0}".format(line))
    finally:
        readLog.close()
        
    print("I made it to this point")
    # Append new entry to log
    log.append('{0}:{1},{2}{3},\n'.format(time[4], time[5], monthNames[time[1]-1], time[2]))
    print(log)
    
    # Replace/create file with an updated file using log
    writeLog = open(fileName, 'w')
    for line in range(len(log)):
        writeLog.write(log[line])
    writeLog.close()

I think I'm still missing something here. Thoughts?

Thanks for the MSC suggestion. Disabling MSC might not be a bad idea but what I really need is for the pyboard to not open a non-existent file but instead create the file with the new data if the designated file doesn't exist. If the file does exist I want it to read the file add data and re-save the file.

[dhylands]

You can use os.stat to test if a file exists or not. Something like this:

Code: Select all

def fileExists(filename):
    try:
        os.stat(filename)
        return True;
    except OSError:
        return False

[Roberthh]

Please have also a look at the standard Python file open modes, as e.g. explained here: https://www.tutorialspoint.com/python/p ... les_io.htm. You might need the "w+" or "a+" mode.

[efahl]

Looking at your code, it appears you simply want to append a timestamp to it? If that's the case, then there's no need to read the current contents, simply open it in append mode as Robert alludes. This probably won't help with the corruption issue, don't know what's going on there...

[code]import time
monthNames = ('Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sept', 'Oct', 'Nov', 'Dec')

def appendTimeStamp(fileName):
t = time.localtime(time.time())
timestamp = '{0}:{1},{2}{3},\n'.format(t[4], t[5], monthNames[t[1]-1], t[2])
try:
with open(fileName, 'a+') as logfile:
logfile.write(timestamp)
except OSError as exc:
print("Error writing to %r: %s" % (fileName, exc))
[/code]

Eric

(Sorry about the lack of post count => no bbcode.)

[Abraxas]

@Dave
Thanks so much! Your suggestion worked for checking to see if a file actually exists. Then I just use an if statement to condition whether reading is necessary. Code below:

Code: Select all

# Don't forget to import uos
def fileExists(fileName):
    try:
        uos.stat(fileName)
        print("File found!")
        return True
    except OSError:
        print("No file found!")
        return False

@Robberthh
@Eric
Thank you for the reference and suggestions. Appending would be easy and I'd be totally all for it but my project is more complicated than just creating a simple logging txt file. I guess it would help if I actually told you guys what it is I am trying to accomplish (I just realized I may have been a little vague on details). I want to allow multiple users to use the same pyboard to log data. Each user would have their own csv file(s) which can easily be selected with the push of a few buttons. Once a csv file is selected, the pyboard would record data in an organized manner. Thus a simple appending routine would not work if I want to keep information sorted.

That being said I still seem to be having some issues with data corruption but I think that needs to be another forum posting (I think I'll write that post in a day or two because I want to do some more testing) as the solution to the current posting has been found! Thanks for all your replies!

[dhylands]

Having USB Mass Storage enabled at the same time as the pyboard writes to any files will eventually lead to corruption of your files.