store config/settings...

[jedie]

I would like to store config/settings... But how?

Currently i save WIFI credentials in a json, looks like:

Code: Select all

{
    "wifi": {
        "your-ssid-1": "Your-Password-1",
        "your-ssid-2": "Your-Password-2"
    }
}

(It is planed, that the user can enter SSID/Password via web interface someday.)

In my project exists some other dynamic settings. The user can change this via web interface: The timers and his time zone offset.

Basically, I see these possibilities to save these kind of data:

• save/restore json files
• create/parse a own format
• Create .py files with e.g.:

  Code: Select all

  FOO=const(123)

  and use __import__ and getattr

I think json would be the most universal. But I guess it's not always the most efficient way. e.g.: Save the time zone offset is just a integer -12 - 12... I guess it would be most effective to save it as a .py file, right?

Any comments, suggestions?

EDIT: If I just think about it: If i respect DRY and KISS then only json comes into consideration, isn't it?

[jedie]

What's about this?

Code: Select all

import gc
import sys

_CONFIG_FILENAME_PATTERN = 'config_%s.json'


def get_json_config(key):
    from ujson import load
    try:
        with open(_CONFIG_FILENAME_PATTERN % key, 'r') as f:
            cfg = load(f)[key]
    except OSError:
        return  # e.g.: file not exists

    del load
    del sys.modules['ujson']
    gc.collect()

    return cfg


def save_json_config(key, cfg):
    if cfg == get_json_config(key):
        print('Skip save json config: Already exists with same data.')
        return

    from ujson import dump
    with open(_CONFIG_FILENAME_PATTERN % key, 'w') as f:
        dump({key: cfg}, f)

    del dump
    del sys.modules['ujson']
    gc.collect()

    if get_json_config(key=key) != cfg:
        raise AssertionError('Verify config failed!')

[jimmo]

I highly recommend your third option -- write out your config as Python (using `repr` helps here) and load it directly by importing.

[jedie]

What do you see as the advantages? RAM consumption?

[pythoncoder]

I agree with @jimmo. The only reason to use JSON is where data is to be written by machine. If it's going to be written by you, why introduce a needless level of encoding and decoding? In your example I'd simply create a file config.py containing:

Code: Select all

wifi = {"your-ssid-1": "Your-Password-1",
        "your-ssid-2": "Your-Password-2"}

[jedie]

An important reason to prefer json: code injections

So i will implement both variants: I will use "python code files" only for validated values. In the end: only numbers

[jedie]

Created py config code:

Code: Select all

_PY_MODULE_NAME_PATTERN = 'config_%s'
_PY_VALUE_ATTRIBUTE_NAME = 'value'
_PY_FILE_PATTERN = 'config_%s.py'

def restore_py_config(module_name, default=None):
    """
    IMPORTANT: Enables code injections! So use with care!
    """
    module_name = _PY_MODULE_NAME_PATTERN % module_name
    try:
        module = __import__(module_name, None, None)
    except ImportError:
        # e.g: py file not created, yet.
        return default

    value = getattr(module, _PY_VALUE_ATTRIBUTE_NAME, default)

    del module
    del sys.modules[module_name]
    gc.collect()
    return value


def save_py_config(module_name, value):
    """
    Will create files like: 'config_{key}.py'
    IMPORTANT: Enables code injections! So use with care!
    """
    if restore_py_config(module_name) == value:
        print('Skip save py config: Already exists with this value')
        return

    with open(_PY_FILE_PATTERN % module_name, 'w') as f:
        if isinstance(value, (int, float)):
            f.write('from micropython import const\n')
            f.write('%s = const(%r)' % (_PY_VALUE_ATTRIBUTE_NAME, value))
        else:
            f.write('%s = %r' % (_PY_VALUE_ATTRIBUTE_NAME, value))

    verify = restore_py_config(module_name)
    if verify != value:
        raise AssertionError('Py config verify error: %r != %r' % (verify, value))

EDIT: Code change: i remove the "key", because it's useless

[stijn]

An important reason to prefer json: code injections

Can you elaborate what this means?

Aslo, another vote for using code for config. Main advantage: well, it's code, which is infinitely more versatile than JSON. Also note: JSON (in standard MicroPython) does not encode inf/nan. Horrendous, if you think about it

[jedie]

Can you elaborate what this means?

Pass a string that will execute code on import, because it's not covert by repr()... Don't know if this is really possible.
But maybe the ujson implementation is safer than the repr() one?

Basically, json have been implemented more in focus on security, I assume.

[MostlyHarmless]

Json is a good default for machine to machine communication, because it is almost a standard and works across languages. An excellent format for API calls.

When I write Flask applications, all my API routes usually exchange Json back and forth via POST request. That makes it very easy to use the exact same API call from a Python CLI as well as from a JS timer loop that updates a couple status fields in my Web dashboard. Getting and sending the entire config.json would be two very simple API routes in picoweb.

If none of the above is of concern, then @jimmo has IMHO the best answer. repr() is probably a bit lighter on the resources than Json. I would not be too worried about injection in this case. If something that was a string or int all of the sudden executes code through a repr() -> import sequence, then there is a bug in repr(). A bug like the roach in MiB.


My $.02, Jan