Sharky Pickles

One machine learning subgenre that I find exciting is listening models. I find it so cool the way that they handle sound. There are a million different ways to represent acoustic data, but for machine learning purposes- the engineer trains the model on visual cues to make classifications. Last summer at Cornell, I got to collect and process some avian bioacoustic data for Merlin training which was beyond a dream come true (I’ve written 2 other blogs about that experience: here & here).

Data collection can be done autonomously via a passive acoustic monitoring unit, but we used shotgun & parabolic microphones to hone in on specific birds with less training data. Parabolic mic we used to scan for noise, and then shotgun we used for high-quality recording.

After the data collection is complete, you go through and delete areas of extended silence, not all silence though, because the space between each bird’s chirp can indicate a different kind of call. Spectrograms were the dataform of choice as you can see above. Within each spectrogram there can be several birds, and so you need to box the frequency signature that is associated with each specifc bird. The model distingushes between birds by the frequency signature of their call! Such a cool and non-invasive way to log instances of this biological class.

This was an exceptionally fun project that I completed in a small team during a 2-week course called “Research in Robotic Design for Agriculture” (January 2025). The object of the course was to make modifications that optimize a FarmBot Genesis in a super short time frame. My team used Tensorflow to create a computer vision model that would use a pi-camera to distinguish between crops and weeds in real time so that the FarmBot would promptly remove non-desired plants.

FarmBot ↑↑ + RPi Cam Demo ↑↑