Standard API for drivers?

Discuss development of drivers for external hardware and components, such as LCD screens, sensors, motor drivers, etc.
Target audience: Users and developers of drivers.
pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Standard API for drivers?

Post by pfalcon » Sat Apr 30, 2016 7:34 pm

pythoncoder wrote:Firstly there seems to be a cost in terms of RAM usage and performance. Secondly accessing a property can trigger code execution which isn't evident to the caller, whereas accessing a method evidently leads to code running.

I'll bow to Damien and Paul's expertise on the first point. I'm only partly convinced by the second - triggering execution is, after all, the purpose of properties - but I'm willing to go along with the convention in future.
Properties look like normal member variables, and give false feeling that they're are fast and cheap to access. A sensor may be on slow bus like I2C, require big block of data to transfer, and then slow floating-point calculations to get real-world units. All that may be very slow, which you may have no idea about and use "sensor.temperature" 10 times in row. If it's method call, it's pretty natural to think that it has its cost, and that value may need to be cached.

Properties also require deep searches for lookup, so super-efficient ports may simply disable properties support, we want to design API which works even for them.

If that all still doesn't ring a bell, let me say it like this: we don't want to limit MicroPython-based battery-powered sensors to 1 year of battery life due to uncareful design of API, if otherwise 3 years are achievable. (It's hard to quantify the numbers, so well, we're just staying on conservative side and avoid using things which are known to be less efficient than the other).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Standard API for drivers?

Post by pythoncoder » Sun May 01, 2016 6:11 am

pfalcon wrote:...Properties also require deep searches for lookup...
Thanks for the clarification.

I've thought for some time that we could do with a new document in the Micropython Language section detailing the best "micro" idioms. In many cases Python offers more than one way of achieving the same results. We've discussed @property vs. a function call. Another which came to mind recently was

Code: Select all

 super().__init__(args)
vs.

Code: Select all

base_class_name.__init__(self, args)
From my point of view as a mere Python coder in classes with single inheritance both work identically: I've no idea which is more efficient in code size or performance. Doubtless there are numerous cases where those with understanding of the compiler could identify idioms which are best avoided.

Alas I imagine it's only you or Damien who could write such a doc. But it would be useful.
Peter Hinch
Index to my micropython libraries.

User avatar
kfricke
Posts: 342
Joined: Mon May 05, 2014 9:13 am
Location: Germany

Re: Standard API for drivers?

Post by kfricke » Sun May 01, 2016 9:17 am

Regarding floating-point vs integer, one should consider the fact that floats of course do feel more natural to us /humans). But first of all MicroPython is for microcontrollers and second the next thing a developer faces after reading a sensor is its binay representation, where floats do consume a lot of space when only tackling small values (float: 4 bytes, double: 8 bytes). The fine fractional resolution of a float also is not needed, because most sensors do have a fairly low resolution which should be truncated after 2-3 digits of the fraction part.

e.g.: when sending my decimal sensor readings to my servers i tend to send 2 char bytes only instead of one float. The first char does only represent the integer value and the second simply the "milli"-fraction.

In my opinion a solid udecimal module should be reasonable to cope with such char-tuples and maybe implement basic float arithmetic as even softener floating-point support. That way sensor drivers might get implemented completely missing real float type-arithmetic, but exposing a reasonable micro interface.
At the end of the day a driver is only worth the number of MicroPython platforms it does support.

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Standard API for drivers?

Post by pfalcon » Sun May 01, 2016 2:23 pm

pythoncoder wrote: Another which came to mind recently was

Code: Select all

 super().__init__(args)
vs.

Code: Select all

base_class_name.__init__(self, args)
From my point of view as a mere Python coder in classes with single inheritance both work identically: I've no idea which is more efficient in code size or performance. Doubtless there are numerous cases where those with understanding of the compiler could identify idioms which are best avoided.

Alas I imagine it's only you or Damien who could write such a doc. But it would be useful.
I personally can't answer which would be the more efficient (I mean, significantly) out of top of my head. I'd err on the side that BaseClass.__init__() would be at least somewhat more efficient. But it has drawback in case of refactoring: you need to change base class name in more than one name. Anyway, as I say, it's hard to answer such questions, and even grounded today answer may become outdated as code evolves. To get empirical, always-current answers, there's tests/run-bench-tests . I added it on May 5th, 2014 - 2 years ago. Since then, nobody contributed more testcases to it beyond those I wrote. That should be good indication how much people are interested in that stuff.

Anyway, this is off-topic in this thread ;-).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: Standard API for drivers?

Post by pfalcon » Sun May 01, 2016 2:32 pm

kfricke wrote:because most sensors
The problem is that we would need to support all sensors, not just "most".
e.g.: when sending my decimal sensor readings to my servers i tend to send 2 char bytes only instead of one float. The first char does only represent the integer value and the second simply the "milli"-fraction.
Then you make your reflow oven and figure out that temperature no longer fits in one byte. That's how well-designed API differs from an adhoc solution you describe. (You can build adhoc solution on a general one, the other way is complicated.)
In my opinion a solid udecimal module
Depends on what it would do. If it introduce whole new type, I'd say it's big scope creep. If it's just set of convenience functions to deal with fixed-point numbers encoded as standard int's, sounds like a good idea.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

PinkInk
Posts: 65
Joined: Tue Mar 11, 2014 3:42 pm

Re: Standard API for drivers?

Post by PinkInk » Sat May 07, 2016 5:16 pm

udecimal would be useful for a more comprehensive ntp library, and potentially other network protocols? (flame-war: on).

In general, as network is more readily available in micropython (baremetal) than it was before (with esp8266), should network protocols fall into the category of 'drivers', and their peculiarities be considered in a standard driver model?

miltmobley
Posts: 30
Joined: Mon Mar 07, 2016 11:44 pm

Re: Standard API for drivers?

Post by miltmobley » Mon May 09, 2016 6:41 pm

This stuff about properties and memory usage is pretty far off the OP's original topic, which was standard api's for devices/drivers.

I am surprised to hear that a developer of a new port would have made, for example, an SPI device that did not have the same API
as the standard, documented SPI device. It should have been implemented as an adapter that would translate the standard api
to the port's hardware library, e.g. stmhal/hal. Any features that could not be translated would be marked as extensions.

Back to the original question, I see a motivation for standard api's:

Suppose I want to make a driver for a WiFi device to attach to a PyBoard via SPI bus.

If I place the code in the drivers folder, I will have to make an adapter there to convert SPI operations for specific ports.
This is because there is no standard SPI api in the C layer. Each port has its own flavor of SPI api specific to its processor.
Which part of the adapter was included in a port build would be selected by a header file in the port source, or a flag
defined in its Makefile.

Alternately I can place the code in the stmhal folder. Then anyone who wants to use the wifi device in another port would have
to port my code to the other port.

If the C layer had a standard api for SPI devices, then each port could register its device with the api handler, and requests in
a driver could be port-agnostic. This is approximately the way Linux uses device classes and class core drivers, which accept
device registrations from devices with different underlying hardware interfaces.

Currently each Micropython port that supports networking has a modnetwork.c file that contains the mod_network_register_nic
function that shows how simple the registration could be, given that the nic identifiers are object references.
And the mod_network_nic_type_t struct contains a list of function pointers that is similar to the structs passed to device class
registration functions in Linux. So some of the needed functionality is already there.

The only problem then is to define the api's. Since it appears the socket code is already using a file like interface, with open,
close, read, write and ioctl functions, you might want to use that as a basis for the api definitions. In Unix (and RTOS systems
that support it), ioctl is used to handle device specific operations that don't map to the file mode. Thus ioctl's could be used to handle
device features that are not supported by the standard api.

User avatar
dhylands
Posts: 3821
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: Standard API for drivers?

Post by dhylands » Mon May 09, 2016 11:41 pm

And the usual humor about standards comes into play:
standards.png
standards.png (23.74 KiB) Viewed 7768 times

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: Standard API for drivers?

Post by deshipu » Tue May 10, 2016 8:22 am

miltmobley wrote: If I place the code in the drivers folder, I will have to make an adapter there to convert SPI operations for specific ports.
This is because there is no standard SPI api in the C layer. Each port has its own flavor of SPI api specific to its processor.
Which part of the adapter was included in a port build would be selected by a header file in the port source, or a flag
defined in its Makefile.
That actually is not such a big problem. While you may be creating the SPI object differently in different ports (because the underlying hardware requires you to either specify which SPI port to use, or which pins to use, or nothing at all because it only has one spi), all SPI objects will have the same methods across all ports (at least that's the goal already being worked towards). That means that if you let your drivers take an SPI object as the parameter and let the user create it, instead of creating it in the driver, your driver suddenly becomes portable across the ports.

User avatar
kfricke
Posts: 342
Joined: Mon May 05, 2014 9:13 am
Location: Germany

Re: Standard API for drivers?

Post by kfricke » Tue May 10, 2016 2:45 pm

dhylands wrote:And the usual ...
..staying off-topic: Congrats to the Hackaday entry!

Post Reply