How to workaround the file system becoming corrupt by interrupted file IO

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
Post Reply
andydrizen
Posts: 7
Joined: Wed Jan 08, 2020 10:10 am

How to workaround the file system becoming corrupt by interrupted file IO

Post by andydrizen » Sat May 02, 2020 2:19 am

I have a little IoT device that stores key/value pairs in the file system. The device will read/write to this store at regular intervals through the day. I have noticed that if the power is interrupted during this IO, the file system becomes corrupt (in the same way as described here: viewtopic.php?t=3102#p18335) with lots of files like:

Code: Select all

\x00\x00\x00\x00
Additionally, the only way I know how to fix this issue is also described in that post: recreate the file system. Unfortunately, this means my IoT device loses it's configuration and must be setup again.

I'm trying to think how to workaround this issue, so my questions for the group are:

‣ Is it really file IO that causes this, or have I misattributed the cause?
‣ Is there a way to prevent this corruption?
‣ Or is there a better location for storing key value pairs that will not be affected by power cycling?
‣ Is it a hardware or software issue? I'm using Micropython on a Wemos D1 Mini.
‣ Is there software/hardware that can fix this issue, e.g. detecting a power outage and finalising file IO before the power is totally lost.

Best,
Andy

Online
User avatar
tve
Posts: 197
Joined: Wed Jan 01, 2020 10:12 pm
Location: Santa Barbara, CA
Contact:

Re: How to workaround the file system becoming corrupt by interrupted file IO

Post by tve » Sat May 02, 2020 4:51 am

1. Use LittleFS instead of FAT
2. If using LittleFS you may want to write a new file and then delete the old one, when reading, use the filename to tell which is newest (e.g. conf0.json, conf1.json, etc)
3. Instead of using the filesystem, use two flash blocks in an alternating manner and keep a seq number in them so you know which of the two is most recent.

andydrizen
Posts: 7
Joined: Wed Jan 08, 2020 10:10 am

Re: How to workaround the file system becoming corrupt by interrupted file IO

Post by andydrizen » Thu May 07, 2020 10:08 am

That looks perfect, thanks so much.

Post Reply