littlefs - advantages & disadvantages

[mattyt]

At the November Melbourne MicroPython Meetup Damien gave a talk and demo about LittleFS.
Although there were some problems during the demo (it highlighted a bug that was fixed ten minutes later!) the explanation ought to be useful.

[MostlyHarmless]

It all seems to me that it's not finished yet.

Open Source Software is never "finished". I am contributing to one Open Source project (PostgreSQL) for 23 years now. We started by keeping the damn thing from crashing all the time. We eventually succeeded and it now is one of the most reliable data platforms available. Which leads to new problems, like that code we wrote in the late 90's and early 2000's doesn't scale well to 8-socket, 192-core machines with 2 TB of RAM. So we didn't "finish" yet.

I still shy away from the effort to compile the firmware myself.

Have you tried? What is the worst thing that can happen? That you run into a problem and have to ask here for help?

Come on, give it a try. Adding your first frozen module to the build will make you feel really good. Trust me.


I know I am evil, Jan

[pythoncoder]

...
It all seems to me that it's not finished yet...

That is a simple statement of fact. Littlefs on ESP8266 is not yet a part of a release build. Those of us who are using it are living life at the "bleeding edge": some complication is to be expected. In time it will doubtless be formally released and it will "just work".

As for building firmware, the ESP8266 hardware is severely resource constrained. If you want to run substantial amounts of code on it, there is no option other than to acquire the skills to build firmware and freeze Python code.

[jedie]

Question : what's the difference between littlefs 1 and 2 ?

[jimmo]

I think v1 support is there mostly for people who wanted to migrate existing / devices that were using v1 for whatever purpose or had some specific reason for using it. But that's why support for it isn't compiled in by default.

[jedie]

Current, official release esp8266-20191220-v1.12.bin from http://micropython.org/download#esp8266 has still no littlefs2 support:

Code: Select all

MicroPython v1.12 on 2019-12-20; ESP module with ESP8266
Type "help()" for more information.
>>> import os
>>> dir(os)
['__class__', '__name__', 'remove', 'VfsFat', 'chdir', 'dupterm', 'dupterm_notify', 'getcwd', 'ilistdir', 'listdir', 'mkdir', 'mount', 'rename', 'rmdir', 'stat', 'statvfs', 'umount', 'uname', 'urandom']

The release note says:

The esp8266 port has switched to use per-board configurations and builds, as well as the new frozen manifest feature. It also has optional littlefs support (requires a separate build).

Why?!?

[pythoncoder]

The release notes state

Support for littlefs filesystems is added through the MicroPython VFS interface, and it works on the unix, stm32, esp8266 and esp32 ports.

I tested this on STM with a standard build of firmware built from source and also tried it on ESP8266 a few days ago with a special build.

But standard V1.12 release firmware indeed doesn't support it:

Code: Select all

MPY: soft reboot
MicroPython v1.12 on 2019-12-20; ESP module with ESP8266
Type "help()" for more information.
>>> import os
>>> dir(os)
['__class__', '__name__', 'remove', 'VfsFat', 'chdir', 'dupterm', 'dupterm_notify', 'getcwd', 'ilistdir', 'listdir', 'mkdir', 'mount', 'rename', 'rmdir', 'stat', 'statvfs', 'umount', 'uname', 'urandom']
>>> 

Odd.

[jedie]

Think it must be enabled by config change. Have you done this?
What I not understand: why it it not enabled by default and fat deactivated. Because it saves RAM and this ist very helpfully on ESP8266.

[yeti]

I tried two custom builds (FAT+LFS2, LFS2-only) and both build without problems.
Nobody ever expects the spani... eeeeh... something different!

The FAT+LFS2 build comes up with the typical contents in a FATFS ('mkfs' it manually to LFS2), the LFS2-only build looks like coming up with an unformatted block device. This aspect needs a closer look. And even if keeping both FSes, there should be a way to select LFS2 to be initialised and pre-filled ('boot.py') by default. Or am I just overlooking the right knob for this in the config?

It really is a noticeable gain in RAM (LFS2-only v1.12 build on 'NodeMCU Amica' (or clone)):

Code: Select all

>>> import gc
>>> gc.mem_free()
36704
>>> _

Great!
The less than 16k free days are keeping to drift away more and more:

Code: Select all

MicroPython v1.5-4-g9ebd4da on 2015-10-22; ESP module with ESP8266
Type "help()" for more information.
>>> import gc
>>> gc.mem_free()
15760
>>> _

[jedie]

It really is a noticeable gain in RAM (LFS2-only v1.12 build on 'NodeMCU Amica' (or clone)):

Code: Select all

>>> import gc
>>> gc.mem_free()
36704
>>> _

A empty ESP8266 with official build:

Code: Select all

MicroPython v1.12 on 2019-12-20; ESP module with ESP8266
Type "help()" for more information.
>>> import gc
>>> gc.mem_free()
30384

Around 6KB more free RAM. Sounds good.
It was always talked about saving 4KB.