[SOLVED] Using btree for key-value-database (OSError: 0)
Re: Using btree for key-value-database (OSError: 0)
Sounds bad...
Do you know any alternative that I could use?
Do you know any alternative that I could use?
Re: Using btree for key-value-database (OSError: 0)
I raised an issue in the github repository. Paul or Damien should see it. Maybe they know an answer.
Re: Using btree for key-value-database (OSError: 0)
Got it working: you have to set pagesize in btree.open to a small value, like 512 or 1024, like:
Code: Select all
db = btree.open(dbfile, pagesize=1024)
Re: Using btree for key-value-database (OSError: 0)
You are awesome! Thank you so much for your help!!!
Re: Using btree for key-value-database (OSError: 0)
90 minutes ago I started to create the 500000 entry database on the ESP8266. Being at 125000 now. On the PC it takes like 20 seconds.
P.S.: Finally, it was indeed a RTFM thing. But I understood it only after sneaking through the btree code.
Update: After almost 13 hours now at 400,000 entries, and still building up. It's getting very slow. There's no real value letting it go up to 500,000, but I'll let it go.
P.S.: Finally, it was indeed a RTFM thing. But I understood it only after sneaking through the btree code.
Update: After almost 13 hours now at 400,000 entries, and still building up. It's getting very slow. There's no real value letting it go up to 500,000, but I'll let it go.
Last edited by Roberthh on Wed Aug 09, 2017 5:55 am, edited 1 time in total.
Re: Using btree for key-value-database (OSError: 0)
Fortunately I can create it on my PC and only have to do few changes, so this will work for me. You saved me
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: [SOLVED] Using btree for key-value-database (OSError: 0)
@Roberthh Well done! That really needs documenting.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: [SOLVED] Using btree for key-value-database (OSError: 0)
@pythoncoder Documentation is one good option. I think it's better to change the default value, maybe depending on the port. That's an pretty easy change, and may be better, considering that many people read the docs only as last resort. I do not want to exempt myself from that (although my last resort is digging in the sources).
- pythoncoder
- Posts: 5956
- Joined: Fri Jul 18, 2014 8:01 am
- Location: UK
- Contact:
Re: [SOLVED] Using btree for key-value-database (OSError: 0)
Agreed.
Peter Hinch
Index to my micropython libraries.
Index to my micropython libraries.
Re: [SOLVED] Using btree for key-value-database (OSError: 0)
Another question regarding RAM: Is it possible to increase heap size if I do not need any flash memory?
I modified the firmware so that I got the sdcard module frozen (and deleted some modules that I don't need). The inisetup.py also does not create the boot.py. I modified _boot.py so that my sdcard will be mounted and boot.py/main.py will be ran from there. Btw: Would it be useful to increase the "SDK params storage" value in flashbdev (currently set to 20480)? Simply changing static heap in main.c from 36 to 48 * 4096 did not do the trick of course... Flash memory size is 860*4096 B (which I don't required).
I would like to run even very large scripts (plain python code located on the sd card), but don't need the on-chip flash memory for my files in return. Currently, my 14 kB file fails (11 kB was working). Is there anything (apart from precompiling/freezing) I can do about this?
I modified the firmware so that I got the sdcard module frozen (and deleted some modules that I don't need). The inisetup.py also does not create the boot.py. I modified _boot.py so that my sdcard will be mounted and boot.py/main.py will be ran from there. Btw: Would it be useful to increase the "SDK params storage" value in flashbdev (currently set to 20480)? Simply changing static heap in main.c from 36 to 48 * 4096 did not do the trick of course... Flash memory size is 860*4096 B (which I don't required).
I would like to run even very large scripts (plain python code located on the sd card), but don't need the on-chip flash memory for my files in return. Currently, my 14 kB file fails (11 kB was working). Is there anything (apart from precompiling/freezing) I can do about this?
Last edited by crizeo on Thu Aug 10, 2017 8:38 pm, edited 1 time in total.