best approach for an application config file

The official pyboard running MicroPython.
This is the reference design and main target board for MicroPython.
You can buy one at the store.
Target audience: Users with a pyboard.
Sokrates
Posts: 15
Joined: Mon Dec 14, 2015 11:24 pm

best approach for an application config file

Post by Sokrates » Sun Jan 17, 2016 2:38 pm

Hello,

I need a config file for the application running on my pyboard.

I need to read/modify a specific attribute on the file and possible create new file if it is missing.

My initial approach would be using xml and I'm wondering if anybody could suggest a lib to use. I found the following xmltok but it seems to be just a reader, not a writer.
https://pypi.python.org/pypi/micropython-xmltok

Any suggestion would be more than welcome.

User avatar
dhylands
Posts: 3465
Joined: Mon Jan 06, 2014 6:08 pm
Location: Peachland, BC, Canada
Contact:

Re: best approach for an application config file

Post by dhylands » Sun Jan 17, 2016 4:49 pm

I've always thought that xml was too wordy, so I personally wouldn't use it unless I had to for compatability.

ujson is builtin to MicroPython
http://docs.micropython.org/en/latest/l ... ujson.html

That works well for simple types (dicts, lists, string, numbers).

I've also used python source code as a configuration format (although typically for read-only config)

dastultz
Posts: 20
Joined: Mon Jan 11, 2016 2:56 am

Re: best approach for an application config file

Post by dastultz » Sun Jan 17, 2016 5:32 pm

I'm with Dave on the wordy XML thing. I'd certainly give JSON a try but I'd favor YAML for super clean syntax.

/Daryl

User avatar
pythoncoder
Posts: 4573
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: best approach for an application config file

Post by pythoncoder » Sun Jan 17, 2016 5:53 pm

My preferred option is pickle (in the micropython library). Unlike (u)json all native Python data types are supported. The principal drawback is security: if the pickled file can be modified by ill-intentioned people they can force the execution of arbitrary Python code. If that's a problem I'd use json.
Peter Hinch

Sokrates
Posts: 15
Joined: Mon Dec 14, 2015 11:24 pm

Re: best approach for an application config file

Post by Sokrates » Mon Jan 18, 2016 10:08 pm

thanks to everybody. I thinks I will go with ujson.

Following some examples that I found on line for future reference:

Write data to a file:

Code: Select all

import json

config = {'key1': 'value1', 'key2': 'value2'}

f = open('config.json', 'w')
f.write(ujson.dumps(config))
f.close()
Read data from a file:

Code: Select all

import json

f = open('config.json', 'r')
c = ujson.loads(f.readall())

#edit the data
config['key3'] = 'value3'

#write it back to the file
f = open('config.json', 'w')
f.write(ujson.dumps(config))
f.close()
 
Some more info about nested dictionaries:

Code: Select all

>>> d = {}
>>> d['dict1']['innerkey1'] = 'value1'
>>> d['dict2']['innerkey2'] = 'value2'
{'dict1': {'innerkey1': 'value1'},'dict2': {'innerkey2': 'value2'}}
 

User avatar
pythoncoder
Posts: 4573
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: best approach for an application config file

Post by pythoncoder » Tue Jan 19, 2016 7:21 am

@Sokrates Your nested dictionary example won't work as posted. Try:

Code: Select all

>>> d = {}
>>> d['dict1'] ={}
>>> d['dict2'] ={}
>>> d['dict1']['innerkey1'] = 'value1'
>>> d['dict2']['innerkey2'] = 'value2'
>>> d
{'dict2': {'innerkey2': 'value2'}, 'dict1': {'innerkey1': 'value1'}}
>>> 
Peter Hinch

dastultz
Posts: 20
Joined: Mon Jan 11, 2016 2:56 am

Re: best approach for an application config file

Post by dastultz » Tue Jan 19, 2016 10:29 pm

pythoncoder wrote:@Sokrates Your nested dictionary example won't work as posted.
defaultdict to the rescue!
https://www.accelebrate.com/blog/using- ... ct-python/

Sokrates
Posts: 15
Joined: Mon Dec 14, 2015 11:24 pm

Re: best approach for an application config file

Post by Sokrates » Tue Jan 19, 2016 10:50 pm

pythoncoder wrote:@Sokrates Your nested dictionary example won't work as posted. Try:

Code: Select all

>>> d = {}
>>> d['dict1'] ={}
>>> d['dict2'] ={}
>>> d['dict1']['innerkey1'] = 'value1'
>>> d['dict2']['innerkey2'] = 'value2'
>>> d
{'dict2': {'innerkey2': 'value2'}, 'dict1': {'innerkey1': 'value1'}}
>>> 
Yes, you are right. Thanks for pointing it out.

maruel
Posts: 1
Joined: Mon Jul 10, 2017 10:35 pm

Re: best approach for an application config file

Post by maruel » Mon Jul 10, 2017 10:44 pm

[quote="pythoncoder"]My preferred option is pickle (in the micropython library). Unlike (u)json all native Python data types are supported. The principal drawback is security: if the pickled file can be modified by ill-intentioned people they can force the execution of arbitrary Python code. If that's a problem I'd use json.[/quote]

Hi, sorry for reviving this old thread but I looked everywhere and couldn't find an answer.

You mention that you use pickle and it's part of the micropython standard library. However, I could only find a dummy implementation of pickle (at least for the ESP8266 version which is what I'm using). See https://github.com/micropython/micropyt ... ter/pickle

Any ideas on if and how I could use pickle? I'm currently using json but it would make my life so much easier to have pickle...

Many thanks,

User avatar
pythoncoder
Posts: 4573
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: best approach for an application config file

Post by pythoncoder » Wed Jul 12, 2017 12:58 pm

I'm not sure what you mean by a "dummy implementation". The version in the link provided works. Note that the pickle files aren't interchangeable with those created by CPython. There is also this issue https://github.com/micropython/micropython/issues/2280. Otherwise I'm not aware of any problem with the version in the library.
Peter Hinch

Post Reply