Page 1 of 1

ANN: ufarc - state machine lib for uasyncio

Posted: Sat Feb 02, 2019 4:19 pm
by dwhall256
Introduction
ufarc is a QP-like framework to help you build an async/concurrent application using hierarchical state machines with message passing and run-to-completion semantics (in less than 500 sloc).

Why use ufarc?
I became a bit disillusioned with CPython's asynchronous syntax in its early days. I was confused on how I was to architect my program when yield was such a clumsy method of flow control. Now, I admit the more recent async/await syntax has greatly improved things, but I still find myself preferring the familiarity of state machines (I learned them in college). State machines let me design an architecture by decomposing the problem by drawing nested boxes (states) and arrows (events). The state machine becomes a class. A state is coded as a method. The framework passes Events (a tuple of signal & data) as an argument to the handler method. This is perfect for embedded programming: nearly all my code is written in compact paragraphs; almost like writing an interrupt handler for each event. I create a state machine for each resource that needs protection against concurrent access. And I code each event handler without worrying about pre-emptive threads, semaphores or deadlock. There's a lot more to the goodness that is state-machine programming that I haven't said here, but I didn't want to ramble too long.

The point is, you can use ufarc to design a concurrent system using state machines and you don't need to know anything about uasyncio. You create state machine classes, start them in their initial state. Then you call ufarc.run_forever() to run uasyncio's event loop.

Honesty
I just got this ported last night. Only half of the code examples run. I'm seeing bugs in the other examples that are a matter of the differences between CPython and MicroPython (and asyncio and uasyncio). But I'm hoping some of you will help me expose the bugs and iron them out.

Backstory
I wrote the farc framework for CPython to help me explore state-machine-oriented programming over the last two years. With farc, I created HeyMac, a 100% CPython app that ran on a raspberry pi, and communicated with GPS over UART, a LoRa modem over SPI and GPIO and provided a layered communications stack... all while running on one Raspbian CPython process. But the Pi was just my prototyping hardware. My ultimate goal is to get HeyMac running on a microcontroller. And that's why I made ufarc (pronounced: you-fark).

Re: ANN: ufarc - state machine lib for uasyncio

Posted: Sun Feb 03, 2019 7:07 am
by pfalcon
Known Issue: Some examples fail due to

Code: Select all

File "/Users/dwhall/.micropython/lib/ufarc/__init__.py", line 513, in run_forever
  File "/Users/dwhall/.micropython/lib/ufarc/__init__.py", line 532, in stop
  File "uasyncio/core.py", line 183, in stop
  File "uasyncio/core.py", line 48, in call_soon
IndexError: full
Well, you're supposed to adjust runq_len/waitq_len params to uasyncio.get_event_loop() based on needs of a particular app. The defaults are such as to not waste memory on small systems for typical apps on small systems like ESP8266.

Code: Select all

def get_event_loop(runq_len=16, waitq_len=16):

Re: ANN: ufarc - state machine lib for uasyncio

Posted: Sun Feb 03, 2019 7:56 am
by pfalcon
And in all fairness, I don't understand why you use asyncio/uasyncio there at all, given that you perform your own scheduling. It would seam they provide convenient abstraction for time accounting, but it's actually rather different in asyncio vs uasyncio.

Code: Select all

    # Desktop debug:
    if not hasattr(_event_loop, "call_at_"):
        _event_loop.call_at_ = _event_loop.call_at
uasyncio call_at_() takes milliseconds, while asyncio call_it() takes seconds, so unclear how much you'd desktop-debug in such way.

Code: Select all

expiration = now + tm_event.interval
...
elif expiration < Framework._time_events[0][0]:
Can't do anything like that. uasyncio is honest down to the last tick, and doesn't fool its users with "infiniteness of time" illusion. MicroPython's, and thus uasyncio's, time comes from something like 24-bit ARM SystTick register, how on earth can it be infinite? It's not even 32-bit, so if you add up too much, you can reach "forever" quite easily.

uasyncio README is quite explicit about those points: https://pypi.org/project/micropython-uasyncio/ , but fairly speaking, it assumes familiarity with MicroPython timing ("utime" module) API, which is itself detailedly documented: http://docs.micropython.org/en/latest/l ... .ticks_add

Re: ANN: ufarc - state machine lib for uasyncio

Posted: Sun Feb 03, 2019 10:24 pm
by dwhall256
@pfalcon, thank you for the feedback.

farc uses the asyncio event loop so as to not re-invent the wheel. I enjoy the non-blocking ability, the timer and spawning execution o a function via run(). I may not be using it as intended, but it's what worked for me on the desktop.

I am experimenting with ufarc and uasyncio to see how much of my CPython code I can re-use on my microcontroller target. I don't have much familiarity yet with micropython, but I am very familiar with embedded software and fitting Python into small spaces.

My example apps shouldn't be overflowing a queue of 16. So I will search to see if anything is getting backed up. Or if a coroutine isn't dissolved after it returns. I don't yet have a good debugging setup that lets me step through the python code running on the unix port on my desktop. Debugging at the C level is possible, but more time consuming. I will likely have to do both at one point or another.

The "Desktop debug:" code is an artifact of me jumping back and forth between MP and CPython for debugging. It probably doesn't make sense to the outsider; and it will go away when I get things running smoothly. I do switch my time constants from seconds to millis.

regards,

!!Dean

Re: ANN: ufarc - state machine lib for uasyncio

Posted: Mon Feb 04, 2019 8:19 pm
by pfalcon
dwhall256 wrote:
Sun Feb 03, 2019 10:24 pm
I don't have much familiarity yet with micropython, but I am very familiar with embedded software and fitting Python into small spaces.
So, that's why I tried to underline some of the points of MicroPython API above, with reasoning behind them. Nothing of that should come as a surprise to an embedded programmer. (But there can be "cognitive dissonance" of high-level nature of Python vs not trying to hide some low-level things, like wrap-around timers, but that's again the point of MicroPython - give users powerful instruments, but maintain faithful representation of hardware, and let users build abstractions on top of that like they want.)
My example apps shouldn't be overflowing a queue of 16. So I will search to see if anything is getting backed up. Or if a coroutine isn't dissolved after it returns.
There definitely can be bugs. Reports are welcome.

What should be mentioned is that the whole idea of MicroPython is that people actually look at the code they use, because that code is high-level and easy to read (because it's Python), and at the same time small enough to grasp quickly (because it's MicroPython).

In that regard, queue of size 16 doesn't mean that there're 16 items fit there, because some items make take more than 1 slot. That still leaves enough space for a simple demo, so yeah, worth looking further.