Remove '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00' files?

[kwiley]

My flash drive seems to have accumulated numerous files named '\x00\x00\x00\x00\x00\x00\x00\x00.\x00\x00\x00'. I don't understand where they came from, and I have no idea how to get rid of them. os.remove() doesn't work.

[kevinkk525]

I had those as part of a filesystem corruption. You could erase the flash and flash the firmware again.

[kwiley]

I suspected as much. I was hoping for a less nuclear solution.

Thanks.

[pythoncoder]

I think it's the only safe solution to a corrupted filesystem.

[nebelgrau]

Why does this happen, what is the mechanism behind the creation of these "files"? I ran into this a few times and I thought the reason might have been some error that occurred while my script was writing into a file: is that it? Or it "just happens sometimes"? Is it a general MP problem or specific to ESP8266?

Thanks!

[jimmo]

The most common cause of this sort of filesystem corruption is if writes get interrupted (due to power off or reset).

The way flash works is you can't just write a single byte, you always have to erase an entire "page", then overwrite the page with new data. The reasons for this is that writes can only set bits to zero, and a different page-erase mechanism can set the entire page to ones. (Or the other way around).

So when you write to a file in MicroPython, what actually happens is:
-Read into memory the entire page containing the area ylu want to write (this may be cached)
- Erase the page
- Modify the in-memory copy with your changes
- Overwrite the page

(This is the same on any computer using flash memory or SD cards etc)

If you lose power (or reset) after step 2, then you will end up in a corrupted state. If the page is just your file data, then you'll see blank sections in your file. If the page is a directory table (or the FAT), then you'll see much more serious corruption, potentially including what you're seeing (likely a corrupted directory table).

Other devices (e.g. your PC) use better filesystems (e.g. with journalling) that can recover from failures. MicroPython uses FAT because it works well over USB and can be implemented with a small amount of code, although on ESP8266 the former isn't relevant.

So is it possible that your program is doing lots of filesystem activity and could be reset while it's running?

[pythoncoder]

The commonest cause of corruption is when the flash drive is configured for MSC (mass storage) mode. The nature of mass storage devices such as USB sticks is that the PC expects to have full control over the (presumed dumb) drive. If the device makes any changes to the filesystem all bets are off and corruption is very likely.

I advise disabling MSC mode in boot.py. I very rarely experience corruption on any MicroPython device.

[kevinkk525]

This does however not apply to the ESP8266 unless that's a new feature I have never heard of.

[pythoncoder]

Ah, you're right. Been spending too much time with Pyboards