Scaling down a prototype

The official PYBD running MicroPython, and its accessories.
Target audience: Users with a PYBD
Post Reply
re_sound
Posts: 2
Joined: Tue Nov 12, 2019 11:57 am

Scaling down a prototype

Post by re_sound » Tue Nov 12, 2019 12:03 pm

Hey everyone,

I'm trying to scale down a prototype that is currently running a python script running on raspberry pi and arduino. The python script opens a csv (which is about 150000 lines long with 26 'columns'), stores it in an array and then closes the file. After this file has loaded, the script then uses Google's speech-text API to listen to conversation. Each word spoken is looked up in the array and to find a sequence of numbers that are then sent over serial to tell the arduino to carry out an action. I've tried this with both the Pi3B+ and Pi0W - on the 3B+ the csv takes s few minutes to open, read and store in the array and can then carry this out. The Pi0W usually crashes trying to do this, presumably because of the limited ram.

In its current iteration the prototype is really rather bulky and as the system is (meant to be) wearable I'm not too keen on it staying that size.

So, I've been looking at these: https://store.micropython.org/product/PYBD-SF6-W4F2 which looks promising - will it have the capabilities to do what I need? If not, do you have any suggestions of other things I might try?

Cheers in advance.

User avatar
jimmo
Posts: 890
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Scaling down a prototype

Post by jimmo » Tue Nov 12, 2019 1:04 pm

Hi,

If it doesn't work on a Pi Zero, then an SF6 is likely not going to be an option.

You could probably get some dramatic improvements though using something other than CSV to store the data. It sounds like a database table -- so perhaps sqlite on the rpi would be a good alternative.

On MicroPython, the btree module might be a good simple alternative to sqlite. http://docs.micropython.org/en/latest/l ... btree.html (Think of it as a file-backed dictionary). For the stm32 you'll have to build your own firmware to enable it (it's only enabled by default on ESP and Unix).

The main idea of both of those options is to avoid loading the whole dataset into RAM - instead it's arranged efficiently on disk. You generate the database file (either sqlite or modbtree) on your real computer, then just copy that onto the device instead.

So if you can get that working on the Pi Zero, then maybe worth looking at the SF6.

re_sound
Posts: 2
Joined: Tue Nov 12, 2019 11:57 am

Re: Scaling down a prototype

Post by re_sound » Tue Nov 12, 2019 3:45 pm

Thanks jimmo for your response, I'll take a look into slimming stuff down.

The reason I kept it loaded in the RAM was to keep the response as fast as possible as the actions triggered by words are fairly time sensitive. Would organising it as a database on disk have a big effect on latency?

User avatar
jimmo
Posts: 890
Joined: Tue Aug 08, 2017 1:57 am
Location: Sydney, Australia

Re: Scaling down a prototype

Post by jimmo » Tue Nov 12, 2019 9:12 pm

As the disk is flash-based I don't think this is going to be a problem. I would imagine the speech-to-text API call would take longer.

Also on the RPi, much of the disk access will be cached automatically by Linux.

Post Reply