Hey everyone,
I'm trying to scale down a prototype that is currently running a python script running on raspberry pi and arduino. The python script opens a csv (which is about 150000 lines long with 26 'columns'), stores it in an array and then closes the file. After this file has loaded, the script then uses Google's speech-text API to listen to conversation. Each word spoken is looked up in the array and to find a sequence of numbers that are then sent over serial to tell the arduino to carry out an action. I've tried this with both the Pi3B+ and Pi0W - on the 3B+ the csv takes s few minutes to open, read and store in the array and can then carry this out. The Pi0W usually crashes trying to do this, presumably because of the limited ram.
In its current iteration the prototype is really rather bulky and as the system is (meant to be) wearable I'm not too keen on it staying that size.
So, I've been looking at these: https://store.micropython.org/product/PYBD-SF6-W4F2 which looks promising - will it have the capabilities to do what I need? If not, do you have any suggestions of other things I might try?
Cheers in advance.
Scaling down a prototype
Re: Scaling down a prototype
Hi,
If it doesn't work on a Pi Zero, then an SF6 is likely not going to be an option.
You could probably get some dramatic improvements though using something other than CSV to store the data. It sounds like a database table -- so perhaps sqlite on the rpi would be a good alternative.
On MicroPython, the btree module might be a good simple alternative to sqlite. http://docs.micropython.org/en/latest/l ... btree.html (Think of it as a file-backed dictionary). For the stm32 you'll have to build your own firmware to enable it (it's only enabled by default on ESP and Unix).
The main idea of both of those options is to avoid loading the whole dataset into RAM - instead it's arranged efficiently on disk. You generate the database file (either sqlite or modbtree) on your real computer, then just copy that onto the device instead.
So if you can get that working on the Pi Zero, then maybe worth looking at the SF6.
If it doesn't work on a Pi Zero, then an SF6 is likely not going to be an option.
You could probably get some dramatic improvements though using something other than CSV to store the data. It sounds like a database table -- so perhaps sqlite on the rpi would be a good alternative.
On MicroPython, the btree module might be a good simple alternative to sqlite. http://docs.micropython.org/en/latest/l ... btree.html (Think of it as a file-backed dictionary). For the stm32 you'll have to build your own firmware to enable it (it's only enabled by default on ESP and Unix).
The main idea of both of those options is to avoid loading the whole dataset into RAM - instead it's arranged efficiently on disk. You generate the database file (either sqlite or modbtree) on your real computer, then just copy that onto the device instead.
So if you can get that working on the Pi Zero, then maybe worth looking at the SF6.
Re: Scaling down a prototype
Thanks jimmo for your response, I'll take a look into slimming stuff down.
The reason I kept it loaded in the RAM was to keep the response as fast as possible as the actions triggered by words are fairly time sensitive. Would organising it as a database on disk have a big effect on latency?
The reason I kept it loaded in the RAM was to keep the response as fast as possible as the actions triggered by words are fairly time sensitive. Would organising it as a database on disk have a big effect on latency?
Re: Scaling down a prototype
As the disk is flash-based I don't think this is going to be a problem. I would imagine the speech-to-text API call would take longer.
Also on the RPi, much of the disk access will be cached automatically by Linux.
Also on the RPi, much of the disk access will be cached automatically by Linux.