One machine learning subgenre that I find exciting is listening models. I find it so cool the way that they handle sound. There are a million different ways to represent acoustic data, but for machine learning purposes- the engineer trains the model on visual cues to make classifications. Last summer at Cornell, I got to collect and process some avian bioacoustic data for Merlin training which was beyond a dream come true (I’ve written 2 other blogs about that experience: here & here).
Data collection can be done autonomously via a passive acoustic monitoring unit, but we used shotgun & parabolic microphones to hone in on specific birds with less training data. Parabolic mic we used to scan for noise, and then shotgun we used for high-quality recording.
After the data collection is complete, you go through and delete areas of extended silence, not all silence though, because the space between each bird’s chirp can indicate a different kind of call. Spectrograms were the dataform of choice as you can see above. Within each spectrogram there can be several birds, and so you need to box the frequency signature that is associated with each specifc bird. The model distingushes between birds by the frequency signature of their call! Such a cool and non-invasive way to log instances of this biological class.