file backed settings class with dot access

General discussions and questions abound development of code with MicroPython that is not hardware specific.
Target audience: MicroPython Users.
Post Reply
bitrat
Posts: 41
Joined: Fri Jul 26, 2019 4:13 am

file backed settings class with dot access

Post by bitrat » Tue Sep 28, 2021 11:21 pm

file backed settings class with dot access

Hi all,

I was about to write a class for this, then wondered if there's one in common use for MP?

I'm looking for an object that can be initialised from a config file by reading into a dictionary, accessed using dot notation using __getattr__ and finally backed up to a file again.

Simple would be good. :-)

Bitrat

bitrat
Posts: 41
Joined: Fri Jul 26, 2019 4:13 am

Re: file backed settings class with dot access

Post by bitrat » Thu Sep 30, 2021 9:32 pm

Update:

following the KICS (keep it complicated) principle, I was playing around with attributes that work like dictionary entries. This reminds me of a prototyped language like JavaScript. My motivation is that I wanted a quicker syntax to create, access and save arbitrary config options during development.

This is a first cut that works in CPython3:

Code: Select all

class SettingDict(object):

    def __init__(self, d = {}):
        object.__setattr__(self, 'dict_',  {})
        for k, v in d.items():
            self.do_setting(k, v)

    def __getattr__(self, k):
        if k is 'dict_':
            return dict_
        try:
            return self.dict_[k]
        except KeyError:
            self.do_setting(k, {})
            return self.dict_[k]

    def do_setting(self, k, v):
        if isinstance(v, dict):
            d = SettingDict(v)
            self.dict_[k] = d
        else:
            self.dict_[k] = v

    def __setattr__(self, k, v):
        self.do_setting(k, v)
        
    def items(self):
        return self.dict_.items()

    def __str__(self, ):
        return 'SD'
        
    def toDict(self):
        rec = set([])
        return self.toDict1(rec)

    def toDict1(self, rec):
        d = {}
        for k, v in self.items():
            if set([v]) & rec:
                raise Exception("circular ref detected")
            rec.update([v])
            if isinstance(v, SettingDict):
                d[k] = v.toDict1(rec)
            else:
                d[k] = v
            rec.discard(v)
        return d

It works like this:

[code]
>>> from settings import SettingDict, pp
>>> sd0 = SettingDict({"aaa": {"BBB": {"sss": {"w": 11232334}, "x": 11232334, "z": 888888888}}})
>>> pp(sd0)
aaa:
   BBB:
      sss:
         w=11232334
      x=11232334
      z=888888888
>>> 
>>> sd0.aaa.BBB.newThing = {'a':1, 'b':2, 'c':3}
>>> 
>>> pp(sd0)
aaa:
   BBB:
      sss:
         w=11232334
      x=11232334
      z=888888888
      newThing:
         a=1
         b=2
         c=3
>>> sd0.aaa.BBB.a.b.c.d = 999
>>> pp(sd0)
aaa:
   BBB:
      sss:
         w=11232334
      x=11232334
      z=888888888
      newThing:
         a=1
         b=2
         c=3
      a:
         b:
            c:
               d=999
>>> sd0.toDict()
{'aaa': {'BBB': {'sss': {'w': 11232334}, 'x': 11232334, 'z': 888888888, 'newThing': {'a': 1, 'b': 2, 'c': 3}, 'a': {'b': {'c': {'d': 999}}}}}}
>>> 
>>> import json
>>> json.dumps(sd0.toDict())
'{"aaa": {"BBB": {"sss": {"w": 11232334}, "x": 11232334, "z": 888888888, "newThing": {"a": 1, "b": 2, "c": 3}, "a": {"b": {"c": {"d": 999}}}}}}'
>>> 

It won't work in MP due to the absence of object.__setattr__() and it's hard to find a way to compose an object without it.

The issues around this are traversed here and here

A side issue is my use of recursion, which may be solvable using a tail call. I may also need to guard against some other attribute names...

My question is.. does anybody have any ideas on how the initial creation of the 'dict_' attribute could be accomplished using MP? In the meantime, I'm moving on with a conventional dictionary! :D

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: file backed settings class with dot access

Post by pythoncoder » Fri Oct 01, 2021 8:25 am

I may be missing something here but your example seems to behave identically in CPython 3.8.10 and

Code: Select all

MicroPython v1.17-74-gd42cba0d2 on 2021-09-28; linux version
. How do you provoke a MP failure? The only problem I've seen is that CPython issues a syntax warning on import:

Code: Select all

>>> import rats93
/home/adminpete/rats93.py:9: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if k is 'dict_':
Example MP session:

Code: Select all

>>> import rats93
>>> s = rats93.SettingDict({"aaa": {"BBB": {"sss": {"w": 11232334}, "x": 11232334, "z": 888888888}}})
>>> s.aaa.BBB.sss.w
11232334
>>> s.aaa.BBB.x
11232334
>>> s.aaa.BBB.z
888888888
>>> s.aaa.BBB.z=42
>>> s.aaa.BBB.z
42
Peter Hinch
Index to my micropython libraries.

bitrat
Posts: 41
Joined: Fri Jul 26, 2019 4:13 am

Re: file backed settings class with dot access

Post by bitrat » Sat Oct 02, 2021 6:34 am

Hi Peter,

thanks for having a look at my code! :)

On

Code: Select all

MicroPython v1.11-361-g4ba0aff47-dirty on 2019-11-05; ESP32 module with ESP32
I get ...

Code: Select all

>>> from settings import SettingDict, pp
>>> 
>>> 
>>> sd0 = SettingDict({"aaa": {"BBB": {"sss": {"w": 11232334}, "x": 11232334, "z": 888888888}}})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "settings.py", line 100, in __init__
AttributeError: type object 'object' has no attribute '__setattr__'
The links I provided discuss the absence in MP of:

Code: Select all

object.__setattr__()
I realise I'm using a superannuated build, and I have no idea why it's dirty, as I got it from somewhere official... I better check out the esp-idf and build a new image...

But, are you saying my code should run on the ESP32 build?

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: file backed settings class with dot access

Post by pythoncoder » Sat Oct 02, 2021 8:38 am

I like this code, it's quite ingenious! Your firmware is truly ancient, you should definitely upgrade, either to release build 1.17 or to the latest daily build. As far as I know your code should run on most if not all platforms, but there have been firmware fixes to __setattr__ which you probably haven't got.

Why are you concerned about recursion? MP supports both normal and tail recursion. I guess tail might use less stack space, but in this application I can't see depth getting beyond four or five in practice so I wouldn't worry about it.
Peter Hinch
Index to my micropython libraries.

bitrat
Posts: 41
Joined: Fri Jul 26, 2019 4:13 am

Re: file backed settings class with dot access

Post by bitrat » Sun Oct 03, 2021 7:35 pm

pythoncoder wrote:
Sat Oct 02, 2021 8:38 am
I like this code, it's quite ingenious!
Thanks! :-)
pythoncoder wrote:
Sat Oct 02, 2021 8:38 am
Your firmware is truly ancient, you should definitely upgrade, either to release build 1.17
Lol, I just remembered, it's the same firmware I was hacking on in 2019.
pythoncoder wrote:
Sat Oct 02, 2021 8:38 am
Why are you concerned about recursion?
I've been prone to recursivity since using xlisp and scheme. I have to remember to keep it in check on constrained architectures...

PS: Happy to say this code works with MicroPython v1.17 on 2021-09-02; ESP32 module with ESP32 from this page.
Last edited by bitrat on Wed Oct 06, 2021 5:21 am, edited 1 time in total.

Post Reply