littlefs (and fatfs) block size

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
Post Reply
User avatar
pythoncoder
Posts: 3925
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

littlefs (and fatfs) block size

Post by pythoncoder » Mon Dec 16, 2019 4:18 pm

I am updating my FRAM driver to take account of various improvements to MicroPython including littlefs, also developing drivers for EEPROM chips. This has thrown up an interesting issue related to block size.

FRAMs are nonvolatile devices with true random access: they have no concept of pages. Consequently the block size in the ioctl is arbitrary. Currently I use 512 bytes which has worked with FAT. Is there any size which is optimum in terms of performance or RAM usage? The device driver doesn't "know" what filesystem is to be used so the block size must not cause FAT to croak.

I also have drivers for EEPROM chips. As an alternative to filesystem usage the drivers allow single or multiple devices to be addressed as arrays of bytes using slice syntax. This means that if someone writes, with an arbitrary value of addr,

Code: Select all

data = b'the quick brown fox'
device[addr:addr + len(data)] = data
the device driver must handle block and chip boundaries. Consequently the block size in the ioctl is again arbitrary.

To gain the benefit of wear levelling it would seem prudent that the block size in the ioctl should be related to the physical block size: in the chips I use this is 128 or 256 bytes so I guess that an integer multiple of these would be wise.

Any recommendations?
Peter Hinch

User avatar
Roberthh
Posts: 1840
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: littlefs (and fatfs) block size

Post by Roberthh » Mon Dec 16, 2019 8:13 pm

As a trial I made a little block device driver using the RTC memory and Littlefs. It worked, but was of little real use. The memory is cleared on boot. But for that I used a block size of 128. There is a minimum block size, which is slightly above 100, as far as I recall. Damien mentioned that somewhere.

User avatar
pythoncoder
Posts: 3925
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: littlefs (and fatfs) block size

Post by pythoncoder » Tue Dec 17, 2019 7:10 am

The block size I choose makes no difference to my drivers: it's just a number. While it's good to know the minimum, my question is whether there is an underlying optimum value; either in terms of speed, RAM usage or wear levelling.

If I'm going to stick constants in my code it's good to be able to justify them ;) Or if there's a good reason, I could make it a config option but I'd want to be able to explain the reason to choose one value over another.
Peter Hinch

User avatar
MostlyHarmless
Posts: 145
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: littlefs (and fatfs) block size

Post by MostlyHarmless » Tue Dec 17, 2019 3:39 pm

pythoncoder wrote:
Tue Dec 17, 2019 7:10 am
Or if there's a good reason, I could make it a config option but I'd want to be able to explain the reason to choose one value over another.
"Nobody will ever need more than 640 ..." is good enough of a reason for me ;)

User avatar
pythoncoder
Posts: 3925
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: littlefs (and fatfs) block size

Post by pythoncoder » Wed Dec 18, 2019 11:05 am

@Roberthh The minimum size for littlefs is 104 bytes: see this doc. I haven't found any figure for an optimum.
Peter Hinch

User avatar
Roberthh
Posts: 1840
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: littlefs (and fatfs) block size

Post by Roberthh » Wed Dec 18, 2019 11:11 am

Yes, that was the number I had in mind. The other aspect was, that for certain elements of the FS, at least one block is required, like a file occupies at least one block. So for a file system in a small memory space, a small block size is more efficient.

User avatar
pythoncoder
Posts: 3925
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: littlefs (and fatfs) block size

Post by pythoncoder » Thu Dec 19, 2019 7:10 am

That makes sense. I've made the block size user-configurable, so my query is so I can make recommendations in the docs.

It would seem that 128bytes is optimum for littlefs, but what about FAT? This article seems to suggest that 512bytes is the minimum. This value certainly works so I guess I should recommend it. Do you have any comment or know-how on this?

I've had limited success with littlefs. I got it working on an ESP8266, gaining 4K of RAM in the process. But so far I haven't managed to mount my EEPROM devices using it. I'll reserve judgement until its official release.
Peter Hinch

User avatar
Roberthh
Posts: 1840
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: littlefs (and fatfs) block size

Post by Roberthh » Thu Dec 19, 2019 7:43 am

I never have seen FAT with a block size smaller than 512 bytes, except maybe on Floppy disks 40 years ago.

User avatar
MostlyHarmless
Posts: 145
Joined: Thu Nov 21, 2019 6:25 pm
Location: Pennsylvania, USA

Re: littlefs (and fatfs) block size

Post by MostlyHarmless » Thu Dec 19, 2019 3:02 pm

pythoncoder wrote:
Thu Dec 19, 2019 7:10 am
It would seem that 128bytes is optimum for littlefs, but what about FAT?
A small block size makes sense if the underlying storage isn't organized in larger units and if the filesystem does not have any per block overhead, like checksums for example.


Regards, Jan

User avatar
pythoncoder
Posts: 3925
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: littlefs (and fatfs) block size

Post by pythoncoder » Fri Dec 20, 2019 1:48 pm

I have now tested the new release version and littlefs on STM32 uses a block size of 512. I guess @Damien chose this value for a good reason.
Peter Hinch

Post Reply