Hello! I have the ESP-EYE, which comes with a camera and some face recognition algorithms on the chip. Unfortunately, all the software is written in C. I'd like to:
a. Create new firmware for it in Micropython
b. Have that firmware access the onboard camera
c. Detect faces and particularly whether or not a face is smiling.
Does anyone have any recommendations as to how to get started? I'm not finding anything online for machine learning in Micropython.
Really appreciate it!
Thanks,
Tyler
Machine learning on ESP-32?
Re: Machine learning on ESP-32?
Hi,
There's already an ESP32 port of Micropython (see http://micropython.org/download#esp32 and https://github.com/micropython/micropyt ... /README.md ). For accessing the camera module, it's hard to tell which camera module is used on the ESP-EYE but from some quick reading it seemed like it might be an OV2640, in which case perhaps it's possible you might be able to use or adapt https://github.com/namato/micropython-ov2640?
For the machine learning stuff, although it's common to use regular Python for machine learning (e.g. Tensorflow and related projects), the actual number crunching is typically done in C (and hopefully offloaded to GPU), and the Python is just a nice wrapper around it. Even numpy offloads all its matrix calculations to C (well... Fortran coverted to C). You really don't want to be doing massive neural network evaluations in pure Python / MicroPython.
If you're interested in learning more about how Espressif's software does the face recognition, have a look here https://github.com/espressif/esp-face/ and if you're interested in the underlying neural network stuff:
- https://arxiv.org/abs/1801.04381
- https://arxiv.org/abs/1604.02878
- https://arxiv.org/abs/1801.07698
It would of course be possible to make a MicroPython wrapper around the esp-face libraries - I'm not aware of any such thing though.
There's already an ESP32 port of Micropython (see http://micropython.org/download#esp32 and https://github.com/micropython/micropyt ... /README.md ). For accessing the camera module, it's hard to tell which camera module is used on the ESP-EYE but from some quick reading it seemed like it might be an OV2640, in which case perhaps it's possible you might be able to use or adapt https://github.com/namato/micropython-ov2640?
For the machine learning stuff, although it's common to use regular Python for machine learning (e.g. Tensorflow and related projects), the actual number crunching is typically done in C (and hopefully offloaded to GPU), and the Python is just a nice wrapper around it. Even numpy offloads all its matrix calculations to C (well... Fortran coverted to C). You really don't want to be doing massive neural network evaluations in pure Python / MicroPython.
If you're interested in learning more about how Espressif's software does the face recognition, have a look here https://github.com/espressif/esp-face/ and if you're interested in the underlying neural network stuff:
- https://arxiv.org/abs/1801.04381
- https://arxiv.org/abs/1604.02878
- https://arxiv.org/abs/1801.07698
It would of course be possible to make a MicroPython wrapper around the esp-face libraries - I'm not aware of any such thing though.
Re: Machine learning on ESP-32?
Hi,
It's been 3+ years since the question was raised regarding the use of Micropython and ESP-EYE.
Any news about this?
Has anyone managed to do that and access the camera HW?
Looking for such a solution myself.
Thanks,
Eyal
It's been 3+ years since the question was raised regarding the use of Micropython and ESP-EYE.
Any news about this?
Has anyone managed to do that and access the camera HW?
Looking for such a solution myself.
Thanks,
Eyal