Tensorflow and about contributing

C programming, build, interpreter/VM.
Target audience: MicroPython Developers.
User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Tensorflow and about contributing

Post by pythoncoder » Fri Nov 27, 2020 9:30 am

rdagger wrote:
Tue Nov 24, 2020 6:27 pm
...I’m currently interested in voice and tone recognition...
AI might be overkill for tone detection. Have you investigated the Goertzel algorithm? I have a MicroPython implementation if that's of any interest.
Peter Hinch
Index to my micropython libraries.

User avatar
rdagger
Posts: 143
Joined: Tue Feb 28, 2017 6:16 pm
Contact:

Re: Tensorflow and about contributing

Post by rdagger » Sun Nov 29, 2020 6:59 pm

pythoncoder wrote:
Fri Nov 27, 2020 9:30 am
AI might be overkill for tone detection. Have you investigated the Goertzel algorithm? I have a MicroPython implementation if that's of any interest.
I'd definitely like to see your MicroPython implementation, thanks!
I'm not sure it will work because I want to create a smart lock that can open based on a short spoken phrase and/or a short melody.
I'm thinking maybe TensorFlow Lite running on a Teensy 4.1 linked to a MicroPython board via I2C.

User avatar
rcolistete
Posts: 352
Joined: Thu Dec 31, 2015 3:12 pm
Location: Brazil
Contact:

Re: Tensorflow and about contributing

Post by rcolistete » Mon Nov 30, 2020 3:47 am

MAixBiT with K210 MCU, using Lobo MicroPython firmware or MaixPy, is really fast. With MaixPy there are neural network frameworks to use :
https://maixpy.sipeed.com/en/
https://maixpy.sipeed.com/en/libs/Maix/kpu.html
My "MicroPython Samples". My "MicroPython Firmwares" with many options (double precision, ulab, etc).

User avatar
rdagger
Posts: 143
Joined: Tue Feb 28, 2017 6:16 pm
Contact:

Re: Tensorflow and about contributing

Post by rdagger » Mon Nov 30, 2020 5:39 am

rcolistete wrote:
Mon Nov 30, 2020 3:47 am
MAixBiT with K210 MCU, using Lobo MicroPython firmware or MaixPy, is really fast. With MaixPy there are neural network frameworks to use :
https://maixpy.sipeed.com/en/
https://maixpy.sipeed.com/en/libs/Maix/kpu.html
The MAix BiT looks like a great choice. It's inexpensive, has a built-in microphone and has a KPU. Thanks!
I'd definitely go with MaixPy because it looks like it's being actively developed.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Goertzel algorithm

Post by pythoncoder » Mon Nov 30, 2020 8:09 am

rdagger wrote:
Sun Nov 29, 2020 6:59 pm
...
I'd definitely like to see your MicroPython implementation, thanks!
I'm not sure it will work because I want to create a smart lock that can open based on a short spoken phrase and/or a short melody...
Here it is. Note that it includes a Pyboard-only tone generator so that it can self-test.

Code: Select all

# Tone detection using Goertzel algorithm.
# Requires Pyboard 1.x with X4 and X5 linked.

from array import array
import math
import cmath
from pyb import ADC, DAC, Pin, Timer
import time
import micropython
import gc
micropython.alloc_emergency_exception_buf(100)

# When searching for a specific signal tone the Goertzel algorithm can be more efficient than fft.
# This routine returns magnitude of a single fft bin, which contains the target frequency.

# Constructor args:
# freq (Hz) Target centre frequency
# nsamples (int) Number of samples
# spc (int) No. of samples per cycle of target frequency: sampling freq = freq * spc
# Filter bandwidth as a proportion of target frequency is spc/nsamples
# so 1KHz with 100 samples and 10 samples per cycle bw = 1KHz * 10/100 = 100Hz

# .calc() computes magnitude of one DFT bin, then fires off a nonblocking acquisition of more data.
# Depending on size of sample set, blocks for a few ms.

class Goertzel:
    def __init__(self, adc, nsamples, freq, spc, verbose=False):
        if verbose:
            print('Freq {}Hz +- {}Hz'.format(freq, freq * spc / (2 * nsamples)))
        self.sampling_freq = freq * spc
        self.buf = array('H', (0 for _ in range(nsamples)))
        self.idx = 0
        self.nsamples = nsamples
        self.adc = adc
        self.scaling_factor = nsamples / 2.0
        omega = 2.0 * math.pi / spc
        self.coeff = 2.0 * math.cos(omega)
        self.fac = -math.cos(omega) + 1j * math.sin(omega)
        self.busy = False
        self.acquire()

    def acquire(self):
        self.busy = True
        self.idx = 0
        self.tim = Timer(6, freq=self.sampling_freq, callback=self.tcb)

    def tcb(self, _):
        buf = self.buf
        buf[self.idx] = self.adc.read()
        self.idx += 1
        if self.idx >= self.nsamples:
            self.tim.deinit()
            self.busy = False

    def calc(self):
        if self.busy:
            return False  # Still acquiring data
        coeff = self.coeff
        buf = self.buf
        q0 = q1 = q2 = 0
        for s in buf:  # Loop over 200 samples takes 3.2ms on Pyboard 1.x
            q0 = coeff * q1 - q2 + s
            q2 = q1
            q1 = q0
        self.acquire()  # Start background acquisition
        return cmath.polar(q1 + q2 * self.fac)[0] / self.scaling_factor

# Create a sinewave on pin X5
def x5_test_signal(amplitude, freq):
    dac = DAC(1, bits=12)  # X5
    buf = array('H', 2048 + int(amplitude * math.sin(2 * math.pi * i / 128)) for i in range(128))
    tim = Timer(2, freq=freq * len(buf))
    dac.write_timed(buf, tim, mode=DAC.CIRCULAR)
    return buf  # Prevent deallocation

def test(amplitude=2047):
    freq = 1000
    buf = x5_test_signal(amplitude, freq)
    adc = ADC(Pin.board.X4)
    g = Goertzel(adc, nsamples=100, freq=freq, spc=10, verbose=True)
    while True:
        time.sleep(0.5)
        print(g.calc())
Peter Hinch
Index to my micropython libraries.

michael.o
Posts: 15
Joined: Sun Oct 25, 2020 12:38 am

Re: Tensorflow and about contributing

Post by michael.o » Sun Jan 17, 2021 5:54 pm

I'm working on a general tensorflow micropython firmware.

https://github.com/mocleiri/tensorflow- ... n-examples

Tensorflow 2.4 and miketeachman's i2s branch (just a bit after 1.13)

There purpose is to create a more generalized python API than what openmv made to the c++ tensorflow lite for microcontrollers library.

I've been starting from openmv's code and adapting to
run inference in the microlite module in the firmware while the tensorflow model specifics coming externally.

Essentially you flash the firmware and then you bring the model and tensor setup logic for setting input data and getting output data.

Eventually I want to use native modules to externalize the tenor ops. As this will reduce the firmware size.


This firmware is not useable yet because I'm working out the details of the python API to call the tflite code needed to setup inputs and extract outputs.


However a lot of the hard parts are done as it builds a firmware for esp32 and the Unix port.

I'm actually using the Unix port so I can debug in visual studio code to try and understand how the tflite structures work.

I'm making great progress so I think inference will be working soon.

If you are interested in tensorflow on micropython you don't need to start from scratch but can start from my work.

I'm working to generalize the hello-world sine wave example but will then implement the other examples:
https://github.com/tensorflow/tensorflo ... o/examples

michael.o
Posts: 15
Joined: Sun Oct 25, 2020 12:38 am

Re: Tensorflow and about contributing

Post by michael.o » Wed Feb 03, 2021 5:35 am

I just wanted to update that I got the hello_world example to work and have moved on to the micro_speech front-end.

For this example I needed to figure out how to run the mfcc logic which transforms the wav file data into a spectorgram which is what is processed by tensor flow.

I'm integrating with ulab now for numpy support but at the moment not enough operations are supported to run the mfcc.py files available publicly so instead I'm wrapping the c/c++ microfrontend library within the tensorflow library.

I hope to come back to this later and try to add the necessary ops to ulab to support running this transform on the micropython side (or at least in a way provided by the application versus the firmware).

Maybe the librosa code although there are others: https://librosa.org/doc/main/generated/ ... .mfcc.html (scroll to the bottom to see the example spectrogram)

This is the link to the micropython code that when run produces the sine wave output just like the regular C++ examples:

https://github.com/mocleiri/tensorflow- ... ello-world

v923z
Posts: 168
Joined: Mon Dec 28, 2015 6:19 pm

Re: Tensorflow and about contributing

Post by v923z » Fri Mar 05, 2021 7:52 am

michael.o wrote:
Wed Feb 03, 2021 5:35 am
I'm integrating with ulab now for numpy support but at the moment not enough operations are supported to run the mfcc.py files available publicly so instead I'm wrapping the c/c++ microfrontend library within the tensorflow library.
You can fix this here: https://github.com/v923z/micropython-ulab/issues/76

We can also add functions that are not numpy-compatible, but work with, or return ndarrays: https://github.com/v923z/micropython-ulab/pull/342

You can then pass your own data transformer that I don't even have to know about: https://github.com/v923z/micropython-ulab/pull/327

If this all is still not enough, then you really have to raise an issue ;)

michael.o
Posts: 15
Joined: Sun Oct 25, 2020 12:38 am

Re: Tensorflow and about contributing

Post by michael.o » Fri Mar 19, 2021 12:29 am

Thanks @v923. I will circle back later and follow those useful links to allow doing the mfcc transform in micropython using ulab.

Post Reply