For Micropython apps, btree is a wonderful boon, allowing a file-backed dict-like database that can be huge compared to the usually trivial memory space of a Micropython board (I'm using an openMV board)
However, I'd love to be able to process big stuff, like Openstreetmaps xml map databases, into btree files using Python on a desktop, then transfer the files to the Micropython board for actual run-time use.
(Low powered, large-ish screen, e-ink based GPS device, anyone?)
My problem is that although I have got code to work on Linux, using berkeleydb.btopen() and on openMV Micropython using btree.open(), I cannot seem to make the files interchangeable.
I kinda expect them to be, because Micropython btree is reportedly based on berkelydb, but 'based on' does not mean compatible necessarily...
Bottom-line is: Does anyone know if this should work and I am just messing up the implementation/file-copy or is it known to be a futile goal?
If it isn't ever going to work, anyone got any suggestion on how I can do this? It does need to be something like btree because the map is never going to fit into the openMV memory all at once... besides, btree on Micropython is kinda lovely. Simple, efficient and available.
Here's some code, because we all love code:
Code: Select all
#For the openMV Micropython
import btree
#now test btree writing a simple file
dbf=open('fred.btr', 'wrb') #wrb so i can write it
db=btree.open(dbf)
nid = 1
nid_bytes=nid.to_bytes(6,'little') # 6 bytes, little endian. To match the Linux implementation
ln='one'
db[nid_bytes]=bytes(ln, 'utf-8') # have to encode everything in bytes
print(db[nid_bytes])
db.close()
dbf.close()
# now see if you can reopen and read it
dbf=open('fred.btr', 'rb') #rb read-only
db=btree.open(dbf) #rb so it is read only
print(db[nid_bytes])
for key, value in db.items():
print('key=',key,'value=',value)
dbf.close()
db.close()b'one'
b'one'
key= b'\x01\x00\x00\x00\x00\x00' value= b'one'
Then on a Linux, the code is fairly similar:
Code: Select all
# for the Linux
import berkeleydb
db=berkeleydb.btopen('testLinuxBerkeleydb.btr', 'n') #always make a new one
nid = 1
nid_bytes=nid.to_bytes(6,'little')
ln='one'
db[nid_bytes]=bytes(ln, 'utf-8') # does setting the size and endianness of the key make a difference? Yes it does.
print(db[nid_bytes])
db.close()
db=berkeleydb.btopen('testLinuxBerkeleydb.btr', 'r') # read-only
print(db[nid_bytes])
for key, value in db.items():
print('key=',key,'value=',value)
db.close()
b'one'
b'one'
key= b'\x01\x00\x00\x00\x00\x00' value= b'one'
Trouble is that when I do a hexdump on the files, they are quite different. Also, when I copy the Linux produced file to the openMV board and try to read it (with code designed to open it read-only and NOT to overwrite it) it fails and gives an OSError 79 (on which I can't find any doco...)
