mmap

[deshipu]

Maybe this is a crazy idea, or maybe there is something like this there already. I got thinking that since the flash memory on ESP8266 is cached in RAM anyways, and the files live in flash, maybe it would be possible to have a mmap-like function, that could give us an object with a buffer protocol for a fragment of a file, which we could then use efficiently with things like SPI and I²C without having to worry about running out of RAM. This would greatly simplify storing things like images, sounds, fonts, etc. on the filesystem -- instead of mucking around with precompiled bytecode, just use mmap!

How feasible would something like this be?

[dhylands]

I'm confused. Don't we already have that? When you open a file, it just reads buffers from the file into RAM, not the entire file.

I guess for flash based filesystems and read-only files, we could bypass RAM altogether and just have a pointer directly into flash (maybe that's what you were alluding to?)

[deshipu]

I'm confused. Don't we already have that? When you open a file, it just reads buffers from the file into RAM, not the entire file.

I guess for flash based filesystems and read-only files, we could bypass RAM altogether and just have a pointer directly into flash (maybe that's what you were alluding to?)

Yes, that's what I was trying to say, sorry for not making it clear.

[jms]

Can't be done without an MMU or explicit SPI mapping support in the device (and I use devices that do this).

[pfalcon]

ESP8266 can map only 1MB window of a flash at time. We're already crossing border of 512K, while implementing stretch goals probably will grow to 600K-700K, then for @micropython.native support, will need 100-200K buffer with executable permissions, and as RAM is super-scarce, that will be flash. That leaves only 100-200K of FS to fit into directly-mappable window. But a normal FS doesn't give any guarantee that file storage is contiguous, or what size of chunks is used, or relative placement. Some special FSes may, there's work on making arbitrary (and multiple) FSes easily pluggable into MicroPython, interested parties are welcome to explore and to contribute to this work.