Recording and sending audio through MQTT?

All ESP32 boards running MicroPython.
Target audience: MicroPython users with an ESP32 board.
Post Reply
Quernon
Posts: 8
Joined: Wed Aug 29, 2018 9:08 am

Recording and sending audio through MQTT?

Post by Quernon » Thu Sep 13, 2018 2:55 pm

I'm trying to set up remote monitoring of some beehives. I have the environmental monitoring hardware and scripts in place (a simple DHT22 setup and script using umqtt.simple).

The next step would be for the ESP32 to send a short recording, ideally 30 seconds long, in mp3 every X minutes. Hourly would be sufficient, more frequently may not be useful.

Is this even possible with the current Micropython build? What hardware would be require to record the audio? What would be the best way of sending the audio through the internet to a waiting Raspberry Pi? I can see that the ESP32 has DAC input/output pins but I'm not sure where to go beyond that. If it turns out that it's not possible then at least I can get on with implementing the environmental monitoring.

Any and all advice would be very much appreciated.

SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: Recording and sending audio through MQTT?

Post by SpotlightKid » Thu Sep 13, 2018 3:17 pm

Micropython currently has no good support for recoding audio. There was a branch in the repo once to implement support for I2S on the STM32F4 Discovery board, but unfortunately it was never finished. Also, none of the officially supported MicroPython boards have an audio-quality DAC on board. There is the AMP skin for the Pyboard from the MicroPython store, but support for it seems very limited.

You would be probably better of to use something like the Teensy 3.2/3.5/3.6 for your project, which comes with an extensive audio library. Here's an example for an audio recorder.

Also, be aware that audio data can get big very fast, depending on the sample rate, bits per sample and channel count. You're looking at ~172 kB/second for 44110 Hz, 16bit stereo audio. For your purpose you can probably get away with half the sample rate or less, mono, and maybe even 12bit. Still, you need to make sure that you have enough storage space and bandwidth to record and transmit the audio in time.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Recording and sending audio through MQTT?

Post by pythoncoder » Fri Sep 14, 2018 7:42 am

@SpotlightKid The I2S branch was for the Pyboard. The guy also developed an I2S shield for the Pyboard (I have one here). The code seemed near to completion but alas was never finished.

@Quernon I'm doubtful that a simple MQTT publication is the right protocol for sending large files over wireless networks, regardless of your physical hardware. Wireless is inherently unreliable. With qos == 1 MQTT is supposed to be reliable but any failure will require the whole packet to be sent again. If the packet takes a long time to transmit, the probability of error goes up. If the signal is weak you could end up continuously retransmitting.

You need a protocol which splits the packet into short segments and reassembles them at the destination. MQTT could still be used as the transport mechanism. If you plan to use official MicroPython MQTT I suggest you look here to learn about its limitations.

I'd consider an alternative approach. Would it be possible to analyse the audio at the hive and send back the results of the analysis? You might, for example, perform a DFT and send back the amplitude value of each frequency bin. The data compression would be very large indeed.
Peter Hinch
Index to my micropython libraries.

Quernon
Posts: 8
Joined: Wed Aug 29, 2018 9:08 am

Re: Recording and sending audio through MQTT?

Post by Quernon » Tue Sep 18, 2018 10:59 am

@pythoncoder Thanks for the pointers. I was imagining that the audio file would need to be chopped up prior to sending but if this seems unnecessarily complicated, is there a more suitable file transfer protocol that you would recommend?

Unfortunately, I can't process the audio file on the ESP32 as I'm hoping to make use of the OSBeehive analysis (more info and APIs can be found here). I've been focused on getting my side working first but will try diving into their APIs a bit more to see if anything can be done on the board to simplify things/reduce the amount of data to be transmitted.

Thanks again.

User avatar
pythoncoder
Posts: 5956
Joined: Fri Jul 18, 2014 8:01 am
Location: UK
Contact:

Re: Recording and sending audio through MQTT?

Post by pythoncoder » Tue Sep 18, 2018 4:53 pm

I don't have a definitive answer but I'd start out by investigating FTP.
Peter Hinch
Index to my micropython libraries.

SpotlightKid
Posts: 463
Joined: Wed Apr 08, 2015 5:19 am

Re: Recording and sending audio through MQTT?

Post by SpotlightKid » Tue Sep 18, 2018 5:26 pm

I ported the FTP client from the CPython standard library to MicroPython, the code can be found here:

https://github.com/SpotlightKid/micropython-ftplib

Post Reply