Page 1 of 1

Timers crash file IO

Posted: Sat Sep 08, 2018 2:14 am
by rhubarbdog
I have written a data logger. It writes data to the SD card, I mount the sd card manually as i have a SKIPSD file in my flash storage. Firstly it loads a `boot_file.txt` then deletes it. Then it loops writing data once every few minutes, press the switch to terminate gracefully. Writing the saved state to `boot_file.txt` for next time and unmount the sd card. In the progam's initial sequence i use a timer to flash an LED using a callback. when done i cancel the callback with `timer.callback(None)`.
I let the program run for a few loops then press the switch. The lights change indicating the loop is terminating. it then just hangs for a bit, terminates my repl session on my laptop, continues to hang eventually the lights switch off, but my file probably will be corrupted.
Here's a minimium example

Code: Select all

import pyb
import os

pressed = False

def switch_cb():
    global pressed

    pressed = True

def timer_cb(timer):
    pyb.LED(1).toggle()
    
def main():

    pyb.LED(2).on()
    os.mount(pyb.SDCard(), '/sd')
    os.chdir('/sd')

    try:
        with open("boot_file.txt","r") as f:
            info=f.readline()
    except OSError:
        info=None
        
    if "boot_file.txt" in os.listdir():
        os.remove("boot_file.txt")
        
    blue=pyb.LED(4)
    blue.off()
    yellow=pyb.LED(3)
    yellow.on()

    print("")
    
    with open("data_file.txt","a") as f:
        while not pressed:
            f.write("hello world.\n")

            print(".",end="")
            
            for _ in range(60):
                if pressed:
                    break
                pyb.delay(1000)

            yellow.toggle()
            blue.toggle()

    print("")        
    blue.off()
    yellow.off()

    with open("boot_file.txt", "w") as f:
        f.write("saved state for next time\n")
        
    os.umount('/sd')
    pyb.LED(2).off()


if __name__ == "__main__":

    switch = pyb.Switch()
    switch.callback(switch_cb)

    timer = pyb.Timer(14, freq=3)
    timer.callback(timer_cb)
    pyb.delay(5000)
    timer.callback(None)
    
    main()
For this bug to happen it appears i need all the events
1) a previously used timer
2) a boot file which is deleted on boot and re written on exit
3) writing of data in a sleepy loop

This is a bug and i haven't done some thing wrong?

update:
if i re start repl whilst there are LEDs still on I get the message unable to open `/dev/ttyACM0`
if i wait until they extinguish the repl session is totally un responsive except to pressing CTRL+D (soft reboot). This just causes the pyboard to flash the red and green lights (error?), no message printed.

Re: Timers crash file IO

Posted: Sat Sep 08, 2018 5:15 pm
by pythoncoder
I wonder if you should chdir back to /flash before unmounting the SD card?

Also note that (I'm pretty sure) removing the SD card while powered up is not supported.

Re: Timers crash file IO

Posted: Tue Sep 11, 2018 3:54 am
by rhubarbdog
Got it. I chipped away at the problem and isolated the bug. There is a bug with file append on pyboard v 1.1 . Keeping the file open for a while and making multiple writes causes the (implicit) close file to hang. I've logged it on github.

Re: Timers crash file IO

Posted: Tue Sep 18, 2018 8:02 pm
by thejoker
rhubarbdog wrote:
Tue Sep 11, 2018 3:54 am
Got it. I chipped away at the problem and isolated the bug. There is a bug with file append on pyboard v 1.1 . Keeping the file open for a while and making multiple writes causes the (implicit) close file to hang. I've logged it on github.
I am also working on a project where I write sensor data to a text file, but I do it slightly different.
In order to speed up the data collection I create a data buffer, which is just an empty list filled with empty lists with length the amount of time instances of data. After I'm done collecting data I write all data to the .txt file. Hope this is useful in some way to you:

Code: Select all

sample_amount = 50
data_buffer = [[] for i in range(sample_amount)]  # List of lists to collect data
for loop_value in range(sample_amount):
    #Read sensor or get data values
    data = read_sensor()
    data_buffer[loop_value] = '{},'.format(data)
#Write data to txt file after enough samples are taken:
data_file = open('measurements.txt', 'w')
for each_data_value in data_buffer:
    data_file.write(each_data_value)
data_file.close()