nrf52: external file system

Discussion and questions about boards that can run MicroPython but don't have a dedicated forum.
Target audience: Everyone interested in running MicroPython on other hardware.
ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

nrf52: external file system

Post by ttmetro » Wed Feb 10, 2021 3:44 am

Are external file systems (i.e. on an attached flash chip) supported for the nrf52?

The on-chip flash is only 1MBytes that are shared between the interpreter and user files. An external file system would leave more space for the interpreter.
Bernhard Boser

User avatar
Roberthh
Posts: 2462
Joined: Sat May 09, 2015 4:13 pm
Location: Rhineland, Europe

Re: nrf52: external file system

Post by Roberthh » Wed Feb 10, 2021 9:22 am

You can use @pythoncoder's spi flash driver to connect a flash device on an spi bus and mount it as a drive. Works well.
https://github.com/peterhinch/micropyth ... ster/flash

edit: A sample script on usage, here for rp2040:

Code: Select all

from machine import SPI, Pin
import os
spi = SPI(0, baudrate=10_000_000, polarity=1, phase=1,
            sck=Pin(18), mosi=Pin(19), miso=Pin(16))

from flash_spi import FLASH
cspins = (Pin(17, Pin.OUT, value=1),)
flash = FLASH(spi, cspins, cmdset=False)
os.mount(flash, "/spi")
print(os.listdir("/spi"))

Before mounting, you have to run os.VfsLfs2.mkfs(flash) to create the file system.

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: nrf52: external file system

Post by ttmetro » Thu Feb 11, 2021 3:21 am

Roberthh wrote:
Wed Feb 10, 2021 9:22 am
You can use @pythoncoder's spi flash driver to connect a flash device on an spi bus and mount it as a drive. Works well.
https://github.com/peterhinch/micropyth ... ster/flash
...
Thanks! I'll give this a try & see how far I'll get.

The CircuitPython nrf port is much more full-featured, except that it lacks interrupts, uasyncio, etc. Is it easier to add those to CircuitPython or get the missing features to the MicroPython port ...
Bernhard Boser

User avatar
jimmo
Posts: 2244
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: nrf52: external file system

Post by jimmo » Thu Feb 11, 2021 5:33 am

ttmetro wrote:
Thu Feb 11, 2021 3:21 am
The CircuitPython nrf port is much more full-featured, except that it lacks interrupts, uasyncio, etc. Is it easier to add those to CircuitPython or get the missing features to the MicroPython port ...
We are keen to add more to the nRF port. In particular making it use the same modbluetooth API as the other ports is high on my radar. Definitely would welcome contributions to improve it.

In this case, MicroPython has tried to make it possible to support as much of these sort of filesystem features as possible from within Python (rather than requiring each port to implement dedicated drivers). Although it would be good to make it easier to find things like Peter's flash driver.

I don't know a huge amount about CircuitPython although Scott does show up in this forum quite often. Perhaps worth asking in their Discord though. I know they were working on async/await and scheduler-related features recently. I don't know what their plan is for interrupts (although since I started using asyncio in MicroPython I've been trying to use as few interrupts as possible!!).

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: nrf52: external file system

Post by ttmetro » Thu Feb 11, 2021 5:46 pm

jimmo wrote:
Thu Feb 11, 2021 5:33 am
We are keen to add more to the nRF port. In particular making it use the same modbluetooth API as the other ports is high on my radar. Definitely would welcome contributions to improve it.
That's great to hear! I've submitted a small "trial" PR (https://github.com/micropython/micropython/pull/6753) and am waiting to see if it gets merged. Once that happens I may submit other PRs.
jimmo wrote:
Thu Feb 11, 2021 5:33 am
I don't know a huge amount about CircuitPython although Scott does show up in this forum quite often. Perhaps worth asking in their Discord though. I know they were working on async/await and scheduler-related features recently. I don't know what their plan is for interrupts (although since I started using asyncio in MicroPython I've been trying to use as few interrupts as possible!!).
I see two use cases for asyncio: IO - that's what C-Python uses it for and that's what's implemented in uasyncio.

The other interesting case is for power management: whenever no task is ready to run the processor goes into LightSleep. I've tried this (https://github.com/iot49/eventio) with an nrf52 which has excellent low-power support. With the right hardware and taking care of disabling high-power peripherals it's indeed possible to get impressive power savings almost for "free". Polling of course is not compatible with this. I wonder if something like this is in the future of a MicroPython specific uasyncio?

It looks like CircuitPython takes a different angle at this (https://circuitpython.readthedocs.io/en ... index.html), organizing interrupts and power management around "Alarms". Since I use this for teaching I have a preference for conventional interrupt based approaches (otherwise students will be puzzled in the next course or job), but perhaps that's a too narrow view.
Bernhard Boser

User avatar
jimmo
Posts: 2244
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: nrf52: external file system

Post by jimmo » Fri Feb 12, 2021 2:08 am

ttmetro wrote:
Thu Feb 11, 2021 5:46 pm
The other interesting case is for power management: whenever no task is ready to run the processor goes into LightSleep. I've tried this (https://github.com/iot49/eventio) with an nrf52 which has excellent low-power support. With the right hardware and taking care of disabling high-power peripherals it's indeed possible to get impressive power savings almost for "free". Polling of course is not compatible with this. I wonder if something like this is in the future of a MicroPython specific uasyncio?
Yes, absolutely. I hope asyncio will eventually become a really convenient way of integrating sleep modes (especially light sleep). One of the main blockers to progress on this is coming up with a good way to integrate peripheral i/o with asyncio, and in general having a common path for ways for tasks to be woken. See https://github.com/micropython/micropython/pull/6125 for some initial work in this area.

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: nrf52: external file system

Post by ttmetro » Fri Feb 12, 2021 7:16 pm

jimmo wrote:
Fri Feb 12, 2021 2:08 am
Yes, absolutely. I hope asyncio will eventually become a really convenient way of integrating sleep modes (especially light sleep). One of the main blockers to progress on this is coming up with a good way to integrate peripheral i/o with asyncio, and in general having a common path for ways for tasks to be woken. See https://github.com/micropython/micropython/pull/6125 for some initial work in this area.
That's fantastic!

I've looked a bit at this PR and the ones that preceded it.
In case that has not been considered - a possibly simpler solution is to permit ISRs to schedule tasks for execution. Other schemes like poll or uevent can then be layered on top of this capability.

I've implemented that with an Atomic FIFO queue that is interrupt safe and does not disable interrupts (I'd used the https://infocenter.nordicsemi.com/index ... tfifo.html in the nrf52 library, but there are alternatives).

Interrupts wake the processor from light sleep. The ISR adds task(s) to the queue (similar to MicroPython's schedule). After it returns, the processor resumes where it entered sleep.

Maybe I'm missing something; at least it's simple and flexible.
Caveat: won't work with treads, at least the fifo I used does not.
Bernhard Boser

User avatar
jimmo
Posts: 2244
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: nrf52: external file system

Post by jimmo » Sun Feb 14, 2021 10:21 pm

ttmetro wrote:
Fri Feb 12, 2021 7:16 pm
In case that has not been considered - a possibly simpler solution is to permit ISRs to schedule tasks for execution.
Yes, basically you're implementing micropython.schedule but in Python.

A possibly even simpler way to do exactly this is to add support for loop.call_soon_threadsafe -- https://docs.python.org/3/library/async ... threadsafe

This would allow a (hard or soft) ISR to just go:

Code: Select all

def irq():
  asyncio.get_event_loop().call_soon_threadsafe(my_handler_task())
Behind the scenes, this is doing exactly the same thing (a thread-safe queue of things to add to the loop at a safe non-ISR point later).

I'm looking forward to this functionality because I'd like a way to safely set a asyncio.Event from ISR context.

ttmetro
Posts: 94
Joined: Mon Jul 31, 2017 12:44 am

Re: nrf52: external file system

Post by ttmetro » Tue Feb 16, 2021 5:30 pm

jimmo wrote:
Sun Feb 14, 2021 10:21 pm

Code: Select all

def irq():
  asyncio.get_event_loop().call_soon_threadsafe(my_handler_task())
I'm looking forward to this functionality because I'd like a way to safely set a asyncio.Event from ISR context.
I was not aware of call_soon_threadsafe; would be great to have in MicroPython!

Is this what uevent is about? Looks more like a MicroPython specific extension. I'm sure Damian knows what he's doing, but from my limited understanding I'd "vote" for call_soon_threadsafe and then layer (custom) scheduling on top of that.
Bernhard Boser

User avatar
jimmo
Posts: 2244
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia
Contact:

Re: nrf52: external file system

Post by jimmo » Wed Feb 17, 2021 12:59 am

ttmetro wrote:
Tue Feb 16, 2021 5:30 pm
I was not aware of call_soon_threadsafe; would be great to have in MicroPython!

Is this what uevent is about? Looks more like a MicroPython specific extension. I'm sure Damian knows what he's doing, but from my limited understanding I'd "vote" for call_soon_threadsafe and then layer (custom) scheduling on top of that.
Yes, uevent is part of the low-level implementation required to support call_soon_threadsafe in an efficient way, and also a bunch of other things that uasyncio needs to wait on.

The idea is kind of two parts:
- Move all polling into a single place, written in C so that Python code doesn't have to be running when nothing is ready.
- Make it possible to figure out that "nothing is going to be ready, stop polling and go to sleep / wfi / wev"

You can implement the functionality of call_soon_threadsafe right now by creating a global queue of queued coroutines, that and a task can poll that queue and turn them into tasks. But it's crazy inefficient because you have a task that has to burn a lot of time just polling that queue.

It would be something like:

Code: Select all

queue = []

async def dispatch_task():
  while True:
    if queue:
      asyncio.create_task(queue.pop())
    await asyncio.sleep(0)
then in your soft ISR (or schedule context):

Code: Select all

queue.push(my_task())
Note this doesn't work from hard IRQ because the queue list isn't hard-IRQ safe.

uevent would give a generic way to replace that sleep(0) with a "wake me up when something modified the queue". And then instead of having a dedicated task for this, it would just be part of the core uasyncio loop where it also waits for i/o, etc.

FWIW, asyncio.ThreadSafeFlag was merged last night -- it's also inefficient for the same reason, but it's a starting point for a uevent-based implementation in the future. And it should be hard-IRQ safe.

Post Reply