Page 1 of 3

2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Thu Jun 16, 2016 8:32 pm
by pfalcon
While MQTT client waits for more feedback and testing, work on another stretch goal - a simple embedded database suitable even for the smallest devices - has started. As with MQTT, this is more of a progress report than announcement of ready to use feature, but if you're a developer interested in this area, you can find more details at https://github.com/micropython/micropython/issues/2188 . As usual, initial prototyping and development happens with unix port, and bringing this down to esp8266 will certainly require additional porting and optimization, so may take some time.

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Sat Jul 02, 2016 2:47 pm
by pfalcon
The "btree" module is now enabled for "unix" port. Note that it has "provisional" status and some API details WILL change - like way(s) to construct btree object and constant names. But the main data access interface should be quite stable - it follow Python's "dictionary protocol" as close as possible, with extensions done in a natural way to supported ordered key range enumeration. You can see usage example in a unit test for the module, https://github.com/micropython/micropyt ... /btree1.py .

It will take more effort (and time) to port module to baremetal ports (like ESP8266), and that's why I say that details of API will change (as a result of this porting). But all in all, if you're interested in usage of MicroPython as a full-stack application framework (and native database support is important part of that), you can start to play with it using the unix port.

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Mon Aug 01, 2016 5:37 pm
by pfalcon
Btree module has now been refactored to support esp8266 and was enabled in the master. It passes basic unit test, but more testing is required for it. If you're interested in database support, feel free to give it a try. A usage example is here: https://github.com/micropython/micropyt ... /btree1.py

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Mon Aug 08, 2016 8:45 pm
by mad474
Congratulations Paul! I've been playing with all available db-methods of todays build since an hour and didn't manage get the ESP8266 in to trouble (besides expected and well-established memory issues: Testing on a 1MB ESP-01).

Heaps of shame on me (who was one of the loudest crying for a DB) for not having time in these months to invest more than a few hours of poking around. For me and others(?) with a bad conscience some kind of sponsoring method may be a big relief ;) - no serious.

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Tue Aug 09, 2016 8:00 pm
by pfalcon
mad474 wrote:Congratulations Paul! I've been playing with all available db-methods of todays build since an hour and didn't manage get the ESP8266 in to trouble (besides expected and well-established memory issues: Testing on a 1MB ESP-01).

Heaps of shame on me (who was one of the loudest crying for a DB) for not having time in these months to invest more than a few hours of poking around. For me and others(?) with a bad conscience some kind of sponsoring method may be a big relief ;) - no serious.
Thanks for trying it out, it's actually quite helpful to know it has worked for somebody before the release. Otherwise, I'm pretty got used to the situation when some feature gets some attention/a kind of adoption month(s) after it was released (when developers are already deep into something else, so can't even give enough/immediate attention to shower of questions starting ;-) ). Anyway, as you know, I was quite happy to work on an embedded DB, and wanted to deliver it early exactly to let people to start playing with it sooner, as it has big potential not just for ESP8266, but for MicroPython as a whole.

Definitely don't feel bad and feel free to look into it at your own pace (it's a summer after all). You can always clear conscience by posting examples and answering other users' questions ;-). And well, currently it's all OK with MicroPython development - the Kickstarter campaign gave enough momentum for now. The key is sustainability, so if later we need to think how to keep up MicroPython further, your ideas may be helpful. Thanks for your support!

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Sat Aug 13, 2016 2:53 pm
by jgriessen
But all in all, if you're interested in usage of MicroPython as a full-stack application framework (and native database support is important part of that), you can start to play with it using the unix port.
And now the ESP8266! This IS getting interesting. I can imagine a vending machine with its own website that connects to a customer's phone or BLE dongle for a low cost money transaction. Maybe something similar to https://www.dotdashpay.com/ but less expensive and probably for bitcoin and ethereum also.

That might want the next ESP32 with BLE, since that is one money dongle comm method.

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Tue Apr 04, 2017 4:05 am
by pfalcon
It occurred to me that all this time btree module stayed undocumented. Sorry for this omission, docs are now added: http://docs.micropython.org/en/latest/e ... btree.html . Proofreading is welcome.

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Tue Apr 04, 2017 12:08 pm
by SpotlightKid
One remark about the docs: I don't see any mention of what the implicit sort order of keys is. If it is alphabetical, which I assume, which encoding is used and is it by code point or lexicographical? Can the sort method be changed?

Apart from that the docs have the (usual ;-)) inconsistent use of single and double backticks, which is all over the docs. I admit, it's easy to get this wrong, especially when one often writes Markdown at other times.

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Tue Apr 04, 2017 4:31 pm
by pythoncoder
Does Python support multiple ways of sorting bytes instances?

Re: 2nd ESP8266 projects stretch goal - Embedded Database (btree module)

Posted: Tue Apr 04, 2017 4:56 pm
by SpotlightKid
I am not sure. Anyway, I was guessing that the sorting would be implemented in C.