Saving data on ramdisk before a WDT event

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Saving data on ramdisk before a WDT event

Post by davef » Sun Jun 13, 2021 10:55 pm

I modified this snippet found elsewhere on the forum so that I could log the time of random timeouts (freezes) on a ESP32. Then I thought I would try to save what was in /ramdisk before I did a machine.reset()

Code: Select all

## Simple software WDT implementation
import machine
import utime
from shutil import copyfileobj


wdt_counter = 0

def wdt_callback():
    global wdt_counter
    wdt_counter += 1
    if (wdt_counter >= 70): #  70 seconds
        epoch = utime.time()
        local_time = utime.localtime(epoch)
        seconds = local_time[5]
        minutes = local_time[4]
        hours = local_time[3]
        day = local_time[2]

        time = str(hours) + ':' + str(minutes) + ':' + str(seconds)

        try:
            with open('errors.txt', 'a') as outfile:
                outfile.write(str(day) + ' ' + time + '\n')
                outfile.write('had a wdt event' + '\n')
        except OSError:
            pass

     #  save sbms_datalog.csv
        src_file = open('/ramdisk/sbms_datalog.csv', 'rb')
        dest_file = open('sbms_datalog.csv', 'wb')
        copyfileobj(src_file, dest_file, length=512)

        print('Contents of src copied to dest')
        
        src_file.close()
        dest_file.close()

        machine.reset()


def wdt_feed():
    global wdt_counter
    wdt_counter = 0

wdt_timer = machine.Timer(3) #  hardware timer for ESP32
wdt_timer.init(period=1000, mode=machine.Timer.PERIODIC, callback=lambda t:wdt_callback())
## END Simple software WDT implementation
However, what seems to happen when the callback occurs is that the contents of /ramdisk disappear. I have tested the copyfileobj part in a stand-alone program. The write to errors.txt continues to work after the callback.

Is this expected behaviour? Any thoughts on how to save the data other than periodic saves to flash?

Thanks.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Saving data on ramdisk before a WDT event

Post by pythoncoder » Mon Jun 14, 2021 1:42 pm

If the ESP32 is crashing, RAM contents will be lost. You could use the network to log data to another machine.

In general ramdisks don't make much sense in a microcontroller context because RAM is scarce. If you want to save data in RAM, you're best off saving it in Python objects (lists, dicts or arrays). But keep an eye on size.
Peter Hinch
Index to my micropython libraries.

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: Saving data on ramdisk before a WDT event

Post by davef » Mon Jun 14, 2021 8:51 pm

Point taken on RAM being scarce. My solution was to buy a ESP32 SPIRAM variant.

What I don't understand is how when the crash occurs (and RAM contents are lost) that I can still save an error message to flash.

I'll look into saving the data as you suggest.

Thanks

User avatar
karfas
Posts: 193
Joined: Sat Jan 16, 2021 12:53 pm
Location: Vienna, Austria

Re: Saving data on ramdisk before a WDT event

Post by karfas » Wed Jun 16, 2021 8:15 pm

You might be able to save something in the RTC slow memory, e.g. machine.RTC().memory().

This will most likely not survive the reboot caused by the watchdog (as the memory attributes are wrong in current micropython), but it CAN be used to remember (small, up to 2048 bytes) things.

One of my PRs - https://github.com/micropython/micropython/pull/7133 - corrects the memory attribute so the RTC memory keeps its content after a watchdog reset.
You can also have a look at https://github.com/karfas/upy-esp32-lib - there I have some functions using the RTC memory as FIFO queue.

Regards,
Thomas
A few hours of debugging might save you from minutes of reading the documentation! :D
My repositories: https://github.com/karfas

davef
Posts: 811
Joined: Thu Apr 30, 2020 1:03 am
Location: Christchurch, NZ

Re: Saving data on ramdisk before a WDT event

Post by davef » Wed Jun 16, 2021 8:37 pm

Hi Thomas,

Thanks for those suggestions.

As the data is not critical I am now saving what is on ramdisk to flash every hour. Worst-case is that I lose up to an hours worth of data. If something becomes "critical" I'll check out RTC memory.

Cheers,
Dave

Post Reply