The file system, raw flash and wear levelling
Posted: Thu Jan 16, 2020 10:30 am
Hi,
on a Wemos D1 mini, I want to use the available remaining flash space to regularly store some data and then send it upstream when the conditions permit. I now have a collected a few questions regarding the management of the flash from micropython:
Do I understand it correctly that micropython's file system functionality is essentially a form of FAT over the free flash space on the device and does no further wear levelling, meaning I risk premature wear out if I create (and delete) lots of data files, which would rewrite the FS directory index structures very often?
Given that I essentially want to keep (self-contained) data chunks around until I have send them away and then delete them, I think I can implement a simple and sufficient wear-levelling approach myself in python. I want to store my chunks sequentially in a modular wrap-around fashion in the available flash space, keeping track of what I sent out by simply deleting them after the upstream host has acknowledged reception.
However, this approach depends on how the exposed APIs actually manage the underlying flash:
Is there an easy way to use the available raw flash access functions and be sure to not poke in the wrong parts of the flash (e.g. the micropython installation itself or the filesystem data structures)? I see no "erase" function to call - is erasing the flash done by writing 0xff data to a particular location? Or is the logic inverted somehow and I should write 0x00? (This is not a particularly critical question as getting this wrong would reduce flash life by only a factor of 2).
If I allocate a big file (filling most of the available FS space) and then go and only rewrite sections of that file while not changing its size, does that mean that only the place in flash corresponding to where I rewrite is touched? Or is micropython also automatically updating anything in the file system index structures (e.g. like Linux would be updating atime)?
Cheers and thanks for any answers!
on a Wemos D1 mini, I want to use the available remaining flash space to regularly store some data and then send it upstream when the conditions permit. I now have a collected a few questions regarding the management of the flash from micropython:
Do I understand it correctly that micropython's file system functionality is essentially a form of FAT over the free flash space on the device and does no further wear levelling, meaning I risk premature wear out if I create (and delete) lots of data files, which would rewrite the FS directory index structures very often?
Given that I essentially want to keep (self-contained) data chunks around until I have send them away and then delete them, I think I can implement a simple and sufficient wear-levelling approach myself in python. I want to store my chunks sequentially in a modular wrap-around fashion in the available flash space, keeping track of what I sent out by simply deleting them after the upstream host has acknowledged reception.
However, this approach depends on how the exposed APIs actually manage the underlying flash:
Is there an easy way to use the available raw flash access functions and be sure to not poke in the wrong parts of the flash (e.g. the micropython installation itself or the filesystem data structures)? I see no "erase" function to call - is erasing the flash done by writing 0xff data to a particular location? Or is the logic inverted somehow and I should write 0x00? (This is not a particularly critical question as getting this wrong would reduce flash life by only a factor of 2).
If I allocate a big file (filling most of the available FS space) and then go and only rewrite sections of that file while not changing its size, does that mean that only the place in flash corresponding to where I rewrite is touched? Or is micropython also automatically updating anything in the file system index structures (e.g. like Linux would be updating atime)?
Cheers and thanks for any answers!