Page 2 of 2
Re: Tensorflow and about contributing
Posted: Fri Nov 27, 2020 9:30 am
by pythoncoder
rdagger wrote: ↑Tue Nov 24, 2020 6:27 pm
...I’m currently interested in voice and tone recognition...
AI might be overkill for tone detection. Have you investigated the Goertzel algorithm? I have a MicroPython implementation if that's of any interest.
Re: Tensorflow and about contributing
Posted: Sun Nov 29, 2020 6:59 pm
by rdagger
pythoncoder wrote: ↑Fri Nov 27, 2020 9:30 am
AI might be overkill for tone detection. Have you investigated the Goertzel algorithm? I have a MicroPython implementation if that's of any interest.
I'd definitely like to see your MicroPython implementation, thanks!
I'm not sure it will work because I want to create a smart lock that can open based on a short spoken phrase and/or a short melody.
I'm thinking maybe TensorFlow Lite running on a Teensy 4.1 linked to a MicroPython board via I2C.
Re: Tensorflow and about contributing
Posted: Mon Nov 30, 2020 3:47 am
by rcolistete
MAixBiT with K210 MCU, using Lobo MicroPython firmware or MaixPy, is really fast. With MaixPy there are neural network frameworks to use :
https://maixpy.sipeed.com/en/
https://maixpy.sipeed.com/en/libs/Maix/kpu.html
Re: Tensorflow and about contributing
Posted: Mon Nov 30, 2020 5:39 am
by rdagger
The MAix BiT looks like a great choice. It's inexpensive, has a built-in microphone and has a KPU. Thanks!
I'd definitely go with MaixPy because it looks like it's being actively developed.
Goertzel algorithm
Posted: Mon Nov 30, 2020 8:09 am
by pythoncoder
rdagger wrote: ↑Sun Nov 29, 2020 6:59 pm
...
I'd definitely like to see your MicroPython implementation, thanks!
I'm not sure it will work because I want to create a smart lock that can open based on a short spoken phrase and/or a short melody...
Here it is. Note that it includes a Pyboard-only tone generator so that it can self-test.
Code: Select all
# Tone detection using Goertzel algorithm.
# Requires Pyboard 1.x with X4 and X5 linked.
from array import array
import math
import cmath
from pyb import ADC, DAC, Pin, Timer
import time
import micropython
import gc
micropython.alloc_emergency_exception_buf(100)
# When searching for a specific signal tone the Goertzel algorithm can be more efficient than fft.
# This routine returns magnitude of a single fft bin, which contains the target frequency.
# Constructor args:
# freq (Hz) Target centre frequency
# nsamples (int) Number of samples
# spc (int) No. of samples per cycle of target frequency: sampling freq = freq * spc
# Filter bandwidth as a proportion of target frequency is spc/nsamples
# so 1KHz with 100 samples and 10 samples per cycle bw = 1KHz * 10/100 = 100Hz
# .calc() computes magnitude of one DFT bin, then fires off a nonblocking acquisition of more data.
# Depending on size of sample set, blocks for a few ms.
class Goertzel:
def __init__(self, adc, nsamples, freq, spc, verbose=False):
if verbose:
print('Freq {}Hz +- {}Hz'.format(freq, freq * spc / (2 * nsamples)))
self.sampling_freq = freq * spc
self.buf = array('H', (0 for _ in range(nsamples)))
self.idx = 0
self.nsamples = nsamples
self.adc = adc
self.scaling_factor = nsamples / 2.0
omega = 2.0 * math.pi / spc
self.coeff = 2.0 * math.cos(omega)
self.fac = -math.cos(omega) + 1j * math.sin(omega)
self.busy = False
self.acquire()
def acquire(self):
self.busy = True
self.idx = 0
self.tim = Timer(6, freq=self.sampling_freq, callback=self.tcb)
def tcb(self, _):
buf = self.buf
buf[self.idx] = self.adc.read()
self.idx += 1
if self.idx >= self.nsamples:
self.tim.deinit()
self.busy = False
def calc(self):
if self.busy:
return False # Still acquiring data
coeff = self.coeff
buf = self.buf
q0 = q1 = q2 = 0
for s in buf: # Loop over 200 samples takes 3.2ms on Pyboard 1.x
q0 = coeff * q1 - q2 + s
q2 = q1
q1 = q0
self.acquire() # Start background acquisition
return cmath.polar(q1 + q2 * self.fac)[0] / self.scaling_factor
# Create a sinewave on pin X5
def x5_test_signal(amplitude, freq):
dac = DAC(1, bits=12) # X5
buf = array('H', 2048 + int(amplitude * math.sin(2 * math.pi * i / 128)) for i in range(128))
tim = Timer(2, freq=freq * len(buf))
dac.write_timed(buf, tim, mode=DAC.CIRCULAR)
return buf # Prevent deallocation
def test(amplitude=2047):
freq = 1000
buf = x5_test_signal(amplitude, freq)
adc = ADC(Pin.board.X4)
g = Goertzel(adc, nsamples=100, freq=freq, spc=10, verbose=True)
while True:
time.sleep(0.5)
print(g.calc())
Re: Tensorflow and about contributing
Posted: Sun Jan 17, 2021 5:54 pm
by michael.o
I'm working on a general tensorflow micropython firmware.
https://github.com/mocleiri/tensorflow- ... n-examples
Tensorflow 2.4 and miketeachman's i2s branch (just a bit after 1.13)
There purpose is to create a more generalized python API than what openmv made to the c++ tensorflow lite for microcontrollers library.
I've been starting from openmv's code and adapting to
run inference in the microlite module in the firmware while the tensorflow model specifics coming externally.
Essentially you flash the firmware and then you bring the model and tensor setup logic for setting input data and getting output data.
Eventually I want to use native modules to externalize the tenor ops. As this will reduce the firmware size.
This firmware is not useable yet because I'm working out the details of the python API to call the tflite code needed to setup inputs and extract outputs.
However a lot of the hard parts are done as it builds a firmware for esp32 and the Unix port.
I'm actually using the Unix port so I can debug in visual studio code to try and understand how the tflite structures work.
I'm making great progress so I think inference will be working soon.
If you are interested in tensorflow on micropython you don't need to start from scratch but can start from my work.
I'm working to generalize the hello-world sine wave example but will then implement the other examples:
https://github.com/tensorflow/tensorflo ... o/examples
Re: Tensorflow and about contributing
Posted: Wed Feb 03, 2021 5:35 am
by michael.o
I just wanted to update that I got the hello_world example to work and have moved on to the micro_speech front-end.
For this example I needed to figure out how to run the mfcc logic which transforms the wav file data into a spectorgram which is what is processed by tensor flow.
I'm integrating with ulab now for numpy support but at the moment not enough operations are supported to run the mfcc.py files available publicly so instead I'm wrapping the c/c++ microfrontend library within the tensorflow library.
I hope to come back to this later and try to add the necessary ops to ulab to support running this transform on the micropython side (or at least in a way provided by the application versus the firmware).
Maybe the librosa code although there are others:
https://librosa.org/doc/main/generated/ ... .mfcc.html (scroll to the bottom to see the example spectrogram)
This is the link to the micropython code that when run produces the sine wave output just like the regular C++ examples:
https://github.com/mocleiri/tensorflow- ... ello-world
Re: Tensorflow and about contributing
Posted: Fri Mar 05, 2021 7:52 am
by v923z
michael.o wrote: ↑Wed Feb 03, 2021 5:35 am
I'm integrating with ulab now for numpy support but at the moment not enough operations are supported to run the mfcc.py files available publicly so instead I'm wrapping the c/c++ microfrontend library within the tensorflow library.
You can fix this here:
https://github.com/v923z/micropython-ulab/issues/76
We can also add functions that are not numpy-compatible, but work with, or return ndarrays:
https://github.com/v923z/micropython-ulab/pull/342
You can then pass your own data transformer that I don't even have to know about:
https://github.com/v923z/micropython-ulab/pull/327
If this all is still not enough, then you really have to raise an issue
Re: Tensorflow and about contributing
Posted: Fri Mar 19, 2021 12:29 am
by michael.o
Thanks @v923. I will circle back later and follow those useful links to allow doing the mfcc transform in micropython using ulab.