Flash ROM, external QSPI, v1.1 vs. D's three variants

The official PYBD running MicroPython, and its accessories.
Target audience: Users with a PYBD
Post Reply
kwiley
Posts: 102
Joined: Wed May 16, 2018 5:53 pm
Contact:

Flash ROM, external QSPI, v1.1 vs. D's three variants

Post by kwiley » Mon May 11, 2020 6:46 pm

It's hard to get a really sold understanding of how nonvolatile storage works on PyBoard. The old v1.1 is described as 1024KiB ROM. But direct experience quickly reveals that the mounted drive has a capacity of 97KB. An old discussion on the forums (viewtopic.php?f=6&t=870&p=4997) clarifies how the ROM is used, including reservations for firmware, boot sectors etc., and also claims that a large portion is simply unusable by PyBoard because of some conflict with mem-mapping that I haven't taken too much time to try to understanding.

Not having purchased a PyBoard D yet, this confusion gives me concern that I can't really determine in advance what sort of working environment I will have if I purchase a D. Namely, I can't make purchasing decisions about *which* D model I need. For example, the two lower D models actually advertise only *half* the ROM of the much older v1.1, claiming to offer 512KiB ROM, while the third D is documented as providing twice the v1.1's at 2MiBs. But I don't have any clear picture of what these values really mean at the user-level. The v1.1 ambiguities (in which 1024 on paper becomes a mere 97 when boots hit the ground) suggests that I really don't know what sort of environment these three boards will give me. Do the two lower D models have more or less user ROM than the v1.1? How could it be less when the v1.1 only has 97 of real, usable space, but they certainly claim to have half as much on paper, so...? I dunno...?

Furthermore, there is a tantalizing one-liner in the descriptions of all three D models claiming "2MiB external QSPI flash with execute capabilities to extend internal flash" and second 2MiBs QSPI flash for file system and storage. Discussions elsewhere (viewtopic.php?t=6637) have clarified that one of these is read-only, for storing firmware or read-only data. Presumably that is in reference to the flash that with execute capabilities and internal flash extension, but I'm not sure what that really *means*. Are OS firmware images written to this external flash, thereby freeing up some of the internal flash to offer a larger user-accessible file system? Does this mean that 512 on paper is actually more usable space to the user than the v1.1's 1024 on paper by pushing most of the system-level storage to the external 2M module? I don't know! Help!

And how does one use the read-only data feature of this external 2MBs? I see some documentation about wiring up the board to write data to that flash, but even used as read-only data, how would my program "read it only" if that flash doesn't even mount as a visible file system (even in a read-only state).

And like the v1.1, is there some huge sector of the internal flash that is completely unusable by PyBoard so that the documented flash size is simply...overlooking such a gap? It would be nice to document what fraction of the described 2MiBs is simply "gone", just for clarification.

Lastly, as to the second external 2MiB flash, may I presume that appears as a nearly full-capacity (2MiBs) mounted volume for my own use? How large is that volume in reality once it is mounted?

I don't mean to complain. I understand resources and personnel are limited, and I'm super excited about trying out the D, but I can't make an informed decision about which of the three D models to buy based on the current information.

Thank you very much.

Cheers!

chuckbook
Posts: 119
Joined: Fri Oct 30, 2015 11:55 pm

Re: Flash ROM, external QSPI, v1.1 vs. D's three variants

Post by chuckbook » Tue May 12, 2020 3:41 pm

PYBV11 has 1024kB of internal flash. Due to the it's sector organization only 112kB can be used as FAT filesystem. The FAT area consists of one 3*16kB and 1*64kB sectors. As flash can only erased sector wise, a RAM buffer is needed for buffering content. PYBV11 reserves 64kB of SRAM for this purpose. All other sectors are 128kB in size and using them for a RW FAT filesystem would require 128kB of RAM.

QSPI flash on PYBD use uniform 4kB sectors. Therefore only 4kB of RAM have to allocated to buffer filesystem operations.

PYBD SF2 & 3 provide 512kB internal flash. SF6 provide 2048kB internal flash. As PYBD can execute code directly form external QSPI flash, the 2nd QSPI flash on SF2 & 3 is used to store code that exceeds internal flash space.
Note that extra internal flash space is very often used to hold frozen code or other RO data areas.

Anyhow, SFx modules always provide a 2MB FAT filesystem if default firmware is used.
On SF6 the 2nd QSPI flash can be utilized in various ways. The only limitation is that if the 2nd flash is configured for direct execution, it is practically read only.

Note:
FAT filesystems on internal or external flash are only provided for their simple usage. They are not meant for permanent rw operations like data logging as they don't obey any wear leveling rules. Littlefs, which is supported in MP, would be a better solution but it doesn't provide easy methods for host access.

kwiley
Posts: 102
Joined: Wed May 16, 2018 5:53 pm
Contact:

Re: Flash ROM, external QSPI, v1.1 vs. D's three variants

Post by kwiley » Tue May 12, 2020 6:02 pm

Thanks. That seems to agree with and further clarify the earlier posts on this topic. So, is is correct to assume that if your code exceeds the capacity of the internal flash, the PYBD automatically spills over to the external flash seamlessly and executes from the flash seamlessly, or does the user have to organize or manage code in various ways to make use of the executable external flash?

User avatar
pythoncoder
Posts: 4232
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Flash ROM, external QSPI, v1.1 vs. D's three variants

Post by pythoncoder » Wed May 13, 2020 5:45 am

Normal MicroPython code is loaded from the filesystem. It is compiled to bytecode which runs from RAM.

It is possible to "freeze" the bytecode. This involves precompiling it as part of the firmware build process. In this case it is executed from Flash, saving RAM. On the SF2 and SF3 one of the external QSPI flash chips can be used to store this. The SF6 has sufficient internal flash for this to be unnecessary.
Peter Hinch

Post Reply