I tried btree but this does not seem to work. I keep getting an "OSError: 0" after flushing 4 to 5 kB to the btree. The size of the database does not seem to be the problem. Garbage collection (GC) and heap usage are ok as well. I tried to fix this for days but I can't get rid of the error. Does anybody have an idea? I don't even have to add the values on the ESP, could do it on Windows / Unix instead, but I did not find a way to create a database based on the Berkerley DB 1.8.5 (bsddb185 module not wokring for me).
Or is there a simple alternative to btree? I could even write the values to a standard file (open(..., "w")), but it would take minutes/hours to find a value...
Here is the sample code (sample database with random values just growing until the OSError occurs):
Code: Select all
import btree
import uos, urandom # for random bytestring + db size
import gc # for showing free heap
DATABASE_FILENAME = "btree.db"
def randbytes():
length = 1 + urandom.getrandbits(4) # 1 to 16
return uos.urandom(length)
dbfile = open(DATABASE_FILENAME, 'w+b')
db = btree.open(dbfile)
counter_entries = 0 # number of entries (pairs of k+v) in db
counter_bytes = 0 # number of bytes (keys + values) in database
for i in range(500):
key, value = randbytes(), randbytes()
db[key] = value # here is where the error occurs
counter_entries += 1
counter_bytes += len(key) + len(value)
print("total: %d entries (%d B); db size: %d B; free heap: %d B" %
(counter_entries, counter_bytes, uos.stat(DATABASE_FILENAME)[6], gc.mem_free()))
db.flush()
db.close()
dbfile.close()
Code: Select all
...
total: 249 entries (4168 B); db size: 16384 B; free heap: 512 B
total: 250 entries (4186 B); db size: 16384 B; free heap: 176 B
total: 251 entries (4205 B); db size: 16384 B; free heap: 8608 B
total: 252 entries (4222 B); db size: 16384 B; free heap: 8256 B
total: 253 entries (4242 B); db size: 16384 B; free heap: 7904 B
total: 254 entries (4259 B); db size: 16384 B; free heap: 7536 B
total: 255 entries (4271 B); db size: 16384 B; free heap: 7184 B
total: 256 entries (4280 B); db size: 16384 B; free heap: 6832 B
total: 257 entries (4304 B); db size: 16384 B; free heap: 6480 B
total: 258 entries (4312 B); db size: 16384 B; free heap: 6128 B
total: 259 entries (4329 B); db size: 16384 B; free heap: 5776 B
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "btree_test.py", line 21, in <module>
OSError: 0
Edit: Solution was posted by Roberthh (see page 2):
Code: Select all
db = btree.open(dbfile, pagesize=1024) # or 512
I created a btree database containing 1000 items on Unix and transfered it to the ESP. That's what I get when I search for a entry (OSError: 0):