Author: Justin Salamon

Following on the heels of the PLOS ONE article, the second BirdVox publication will be presented at the ICASSP 2017 conference:

Fusing Shallow and Deep Learning for Bioacoustic Bird Species Classification
J. Salamon, J. P. Bello, A. Farnsworth and S. Kelling
In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, USA, March 2017.
[PDF][Copyright]

Abstract:

Automated classification of organisms to species based on their vocalizations would contribute tremendously to abilities to monitor biodiversity, with a wide range of applications in the field of ecology. In particular, automated classification of migrating birds’ flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we explore state-of-the-art classification techniques for large-vocabulary bird species classification from flight calls. In particular, we contrast a “shallow learning” approach based on unsupervised dictionary learning with a deep convolutional neural network combined with data augmentation. We show that the two models perform comparably on a dataset of 5428 flight calls spanning 43 different species, with both significantly outperforming an MFCC baseline. Finally, we show that by combining the models using a simple late-fusion approach we can further improve the results, obtaining a state-of-the-art classification accuracy of 0.96.

The first study to come out of the BirdVox project has just been published in PLOS ONE:

Towards the Automatic Classification of Avian Flight Calls for Bioacoustic Monitoring
J. Salamon , J. P. Bello, A. Farnsworth, M. Robbins, S. Keen, H. Klinck and Steve Kelling
PLOS ONE 11(11): e0166866, 2016. doi: 10.1371/journal.pone.0166866.
[PLOS ONE][PDF][BibTeX]

Abstract:

Automatic classification of animal vocalizations has great potential to enhance the monitoring of species movements and behaviors. This is particularly true for monitoring nocturnal bird migration, where automated classification of migrants’ flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we investigate the automatic classification of bird species from flight calls, and in particular the relationship between two different problem formulations commonly found in the literature: classifying a short clip containing one of a fixed set of known species (N-class problem) and the continuous monitoring problem, the latter of which is relevant to migration monitoring. We implemented a state-of-the-art audio classification model based on unsupervised feature learning and evaluated it on three novel datasets, one for studying the N-class problem including over 5000 flight calls from 43 different species, and two realistic datasets for studying the monitoring scenario comprising hundreds of thousands of audio clips that were compiled by means of remote acoustic sensors deployed in the field during two migration seasons. We show that the model achieves high accuracy when classifying a clip to one of N known species, even for a large number of species. In contrast, the model does not perform as well in the continuous monitoring case. Through a detailed error analysis (that included full expert review of false positives and negatives) we show the model is confounded by varying background noise conditions and previously unseen vocalizations. We also show that the model needs to be parameterized and benchmarked differently for the continuous monitoring scenario. Finally, we show that despite the reduced performance, given the right conditions the model can still characterize the migration pattern of a specific species. The paper concludes with directions for future research.

BirdVox has been awarded a $1.5 million Big Data program grant, awarded to the project, BirdVox: Automatic Bird Species Identification from Flight Calls, conducted jointly by NYU and the Cornell Lab of Ornithology (CLO), who lead the project.

Further information is provided in the NYU press release.

Collecting reliable, real-time data on the migratory patterns of birds can help foster more effective conservation practices, and – when correlated with other data – provide insight into important environmental phenomena. Scientists at CLO currently rely on information from weather surveillance radar, as well as reporting data from over 400,000 active birdwatchers, one of the largest and longest-standing citizen science networks in existence. However, there are important gaps in this information since radar imaging cannot differentiate between species, and most birds migrate at night, unobserved by citizen scientists. The combination of acoustic sensing and machine listening in this project addresses these shortcomings, providing valuable species-specific data that can help biologists complete the bird migration puzzle.

The Music Technology program of New York University is accepting applications for at least 4 fully-funded PhD fellowships to start in Fall 2017. Fellowships are for 4 years including full tuition remission, health insurance and a yearly stipend. Accepted candidates will join the Music and Audio Research Laboratory (MARL), a multidisciplinary team of scholars and practitioners working at the intersection of sound, music, science and technology, and will work on a variety of projects including recently-funded initiatives such as the NYU Holodeck, SONYC and BirdVox.

For further details please see the call for applications.

On Friday June 24th the popular Science Friday radio show featured a segment about the BirdVox project. The segment included a live interview with Juan Pablo Bello, as well as sound bites from Andrew Farnsworth and Justin Salamon.

You can listen to the segment here.

UPDATE: PRI has published a follow-up article about BirdVox.

From right to left: Science Friday director Charles Bergquist and BirdVox researchers Juan Pablo Bello, Andrew Farnsworth and Justin Salamon.

Applications are invited for a postdoctoral associate position to join our project. The focus of the position will be on the development of machine listening solutions to the analysis of bird flight calls in environmental sound streams. We’re looking for someone with expertise in audio signal processing and machine learning to explore novel data-driven solutions, particularly the use of recurrent and deep feature learning approaches for this task. The successful candidate will join a research collaboration between the Cornell Lab of Ornithology, where the associate will be affiliated, and New York University’s Music and Audio Research Laboratory where the associate will be hosted. This is an annual term appointment with the possibility of renewal, with a starting salary of US$70k. The position is based in New York City.

The Birdvox team is concerned by the lack of diversity in our field, and would like to strongly encourage applications from women and others from communities traditionally underrepresented in computer science and engineering research.

Please send questions and applications to Juan Pablo Bello. Candidates should submit a single pdf file containing a letter of intent, full CV including a list of publications, and names/email addresses of 3 references. Review of applications will start immediately and will continue until the position is filled. We are hoping for the associate to start in Fall 2016.

Together with Gaël Richard, Tuomas Virtanen, Nobutaka Ono and Hervé Glotin, Juan Pablo Bello is guest editing a special issue of the prestigious IEEE/ACM Transactions on Audio, Speech and Language Processing. The topic of the issue is sound scene and event analysis for indoor and outdoor environments, including applications in bio-acoustics. The call for papers is available from here. Manuscripts are due July 1st, 2016.

The Birdvox team is proud to announce that Juan Pablo Bello and Steve Kelling have been awarded a Google Faculty Award for the proposal entitled “BirdVox: Automatic Bird Species Identification from Flight Calls”. The $53,450 award, in the “machine perception” category, is to support our research on how to use machine listening solutions for bird flight call detection and classification in environmental sound streams. You can find the google announcement here and a list of recipients by subject area here.

Justin Salamon will present the BirdVox team’s latest work on flight call classification at the research workshop Listening In The Wild 2015: Animal and machine audition in multisource environments on Friday, August 28th.

The workshop, hosted by Queen Mary University of London’s C4DM, will bring together researchers in engineering disciplines (machine listening, signal processing, computer science) and biological disciplines (bioacoustics, ecology, perception and cognition), to discuss complementary perspectives on making sense of natural and everyday sound.

For further details and registration see: http://litw2015.eventbrite.co.uk/