best approach for an application config file

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
User avatar
on4aa
Posts: 70
Joined: Sat Nov 11, 2017 8:41 pm
Location: Europe
Contact:

Re: best approach for an application config file

Post by on4aa » Sat Jan 20, 2018 3:15 pm

There is one caveat with pickle, though.
Quoting from the documentation:
The Python funcitons eval and exec invoke the compiler at runtime, which requires significant amounts of RAM. Note that the pickle library from micropython-lib employs exec. It may be more RAM efficient to use the ujson library for object serialisation.
Serge

User avatar
pythoncoder
Posts: 3130
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: best approach for an application config file

Post by pythoncoder » Sat Jan 20, 2018 6:07 pm

Agreed. There is also this issue which should be borne in mind. I've admired the elegance of Pickle since I first discovered Python but it has its limitations. But Pickle can handle virtually all Python objects whereas JSON cannot. Horses for courses, but I too would use JSON where possible.
Peter Hinch

User avatar
dhylands
Posts: 2908
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: best approach for an application config file

Post by dhylands » Mon Jan 22, 2018 6:10 pm

You can also use good ole python. This works well for read-only configuration information.

Just create a python module and import it.

ghusson
Posts: 9
Joined: Tue Jan 23, 2018 4:41 pm

Re: best approach for an application config file

Post by ghusson » Mon Jan 29, 2018 2:19 pm

After reading this post, I wrote a sample class here under. Feel free to share, test, enhance !
Example :
my_jsonfile = jsonfile("./test.json", default_data = {"a": "porty", "b": "portx"})
print(my_jsonfile.get_data())
update_data = {"c": "portc", "b": "portb"}
my_jsonfile.update_data_dict(update_data)
print(my_jsonfile.get_data())
my_jsonfile.store_data()

Code: Select all

# Copyright 2017 Gautier HUSSON - Liberasys

# The MIT License (MIT)
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.

try:
    import ujson as json
except ImportError:
    try:
        import json
    except ImportError:
        print("No json library not found.")
        raise SystemExit


class jsonfile:
    ''' This class loads and stores a data from and to a file with json formalism.'''

    def __init__(self, file_path, default_data = {}):
        ''' mandatory argument : file path
            optional argument : default data
              if none provided, an empty dictionnary is set'''

        self.file_path = file_path
        self.data = default_data
        self.load_file()


    def load_file(self):
        ''' Returns the data loaded from json file, or the default one '''

        try:
            with open(self.file_path) as json_file:
                data_from_file = json.loads(json_file.read())
        
        except (OSError, ValueError, IOError):
            print("Couldn't load " + self.file_path + ". Creating a default one.")
            self.store_data()
        
        else:
            if type(self.data) is dict and type(data_from_file) is dict:
                self.data.update(data_from_file)
            else:
                self.data = data_from_file
            print("Loaded config from " + self.file_path + ".")
            json_file.close()


    def get_data(self):
        ''' Returns the data '''
        return(self.data)


    def set_data(self, new_data):
        ''' Stores the data in the instance '''
        self.data = new_data


    def update_data_dict(self, dict_to_be_merged):
        ''' Merges the dictionnary given in argument with self data dictionnary '''
        if type(self.data) is dict and type(dict_to_be_merged) is dict:
            self.data.update(dict_to_be_merged)
        else:
            print("Trying to update dictionnary, but non dict found.")
            raise SystemExit


    def store_data(self):
        ''' Saves the data to json file.'''
        try:
            with open(self.file_path, "w") as json_file:
                json_file.write(json.dumps(self.data))
        except OSError:
            print("Couldn't save " + self.file_path + ".")


#if __name__ == "__main__":
#    my_jsonfile = jsonfile("./test.json", default_data = {"a": "porty", "b": "portx"})
#    print(my_jsonfile.get_data())
#    update_data = {"c": "portc", "b": "portb"}
#    my_jsonfile.update_data_dict(update_data)
#    print(my_jsonfile.get_data())
#    my_jsonfile.store_data()






jimako
Posts: 14
Joined: Fri Dec 23, 2016 6:36 pm

Re: best approach for an application config file

Post by jimako » Mon Apr 23, 2018 4:46 pm

dhylands wrote:
Mon Jan 22, 2018 6:10 pm
You can also use good ole python. This works well for read-only configuration information.

Just create a python module and import it.
That is my preferred method as well. And here is an example on how I use it, in case somebody is looking how to use this method.

Create a `config.py` file:

Code: Select all

WIFI_SSID = "myssid"
WIFI_PASSWORD = "mysecretpassword"
Then in you main script file:

Code: Select all

import config
import network

sta_if = network.WLAN(network.STA_IF)
sta_if.active(True)
sta_if.connect(config.WIFI_SSID, config.WIFI_PASSWORD)

stijn
Posts: 280
Joined: Thu Apr 24, 2014 9:13 am

Re: best approach for an application config file

Post by stijn » Mon Apr 23, 2018 6:43 pm

Another vote for Python config files here: it's simple, just works with all possible types, convenient, and moreover, it's actual code: we have a lot of configuration which requires mathematical calculations so with a Python config the whole calculation becomes part of the file and serves as documentation. Otherwise we'd have to use a seperate script to generate a config file..

User avatar
kamikaze
Posts: 142
Joined: Tue Aug 16, 2016 10:10 am
Location: СССР
Contact:

Re: best approach for an application config file

Post by kamikaze » Tue Apr 24, 2018 3:26 pm

The best config for Python app is Python code! it is much faster to "translate" than any third-party stuff. And you don't need to spend extra CPU and RAM to translate it in runtime

Post Reply