2nd ESP8266 projects stretch goal - Embedded Database (btree module)

All ESP8266 boards running MicroPython.
Official boards are the Adafruit Huzzah and Feather boards.
Target audience: MicroPython users with an ESP8266 board.
pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by pfalcon » Sun Apr 09, 2017 8:44 pm

SpotlightKid wrote:One remark about the docs: I don't see any mention of what the implicit sort order of keys is. If it is alphabetical, which I assume, which encoding is used and is it by code point or lexicographical?
Keys are Python bytes objects and sorted like Python bytes objects (which use lexicographic sorting).
Can the sort method be changed?
It will be interesting to revisit this question when we have a lot of software leveraging the current capabilities of the module. I'm personally waiting for a killer app (say, 1000 github stars).
Apart from that the docs have the (usual ;-)) inconsistent use of single and double backticks, which is all over the docs. I admit, it's easy to get this wrong, especially when one often writes Markdown at other times.
Patches welcome.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by pfalcon » Sat May 13, 2017 11:13 pm

I'm happy to report that "Notes Pico" web application, which is the reference test application for MicroPython's Picoweb web framework, has been finally ported to use btree database backend. The latest version wasn't yet released to PyPI, waiting for last-minute fixes, but is available at https://github.com/pfalcon/notes-pico .

Note that it still remains tested only with Unix version of MicroPython, but yes, the idea is to keep porting and optimizing it to be able to run on ESP8266 (which will guaranteedly require freezing it as bytecode).
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by pfalcon » Sun May 14, 2017 10:03 pm

The latest version of Notes Pico published to PyPI: https://pypi.python.org/pypi/notes-pico/
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

crizeo
Posts: 42
Joined: Sun Aug 06, 2017 12:55 pm
Location: Germany

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by crizeo » Sun Aug 06, 2017 1:02 pm

I wanted to use the btree module to map ~500000 words to integers. Therefore I would have to transform the dict (that I have on my Windows/Linux machine saved via pickle and in standard text file) into a btree.

Is there any possible way to transform the dict to btree on Windows/Linux (and then send the btree database file to the esp8266 via ampy)?
Or... is there any chance of creating a btree database on Windows/Linux?

pfalcon
Posts: 1155
Joined: Fri Feb 28, 2014 2:05 pm

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by pfalcon » Sun Aug 06, 2017 6:48 pm

Yes, btree module is included in Unix build, and database file format should be portable across systems.
Awesome MicroPython list
Pycopy - A better MicroPython https://github.com/pfalcon/micropython
MicroPython standard library for all ports and forks - https://github.com/pfalcon/micropython-lib
More up to date docs - http://pycopy.readthedocs.io/

crizeo
Posts: 42
Joined: Sun Aug 06, 2017 12:55 pm
Location: Germany

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by crizeo » Sun Aug 06, 2017 10:00 pm

I could not find the btree module (Oracle Berkeley DB 1.8.5). The only thing I can install on windows or Unix is Oracle Berkeley 6.x or something... Is there a python script or another simple possibility for creating a btree file on Unix (as it is exists in Micropython)?

jcea
Posts: 27
Joined: Fri Mar 18, 2016 5:28 pm

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by jcea » Sun Aug 06, 2017 11:32 pm

@crizeo, you must compile Micropython in a regular Unix/Linux host, use its btree module to generate the file you what and then copy that file to the ESP8266 device.

crizeo
Posts: 42
Joined: Sun Aug 06, 2017 12:55 pm
Location: Germany

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by crizeo » Sun Aug 06, 2017 11:41 pm

@jcea: Thanks! Is there a tutorial or something on compiling MicroPython on Linux/UNIX?

User avatar
deshipu
Posts: 1388
Joined: Thu May 28, 2015 5:54 pm

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by deshipu » Mon Aug 07, 2017 7:22 am

See the README.txt in the top of MicroPython repository.

crizeo
Posts: 42
Joined: Sun Aug 06, 2017 12:55 pm
Location: Germany

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Post by crizeo » Mon Aug 07, 2017 5:18 pm

Thanks a lot! Got it working on Linux (after apt-get install libffi-dev python-dev).

Post Reply