Identifying Bird Species From Audio Recordings

Bird identification from audio is the most successful applied bioacoustics task there is, and it works by a slightly absurd trick: convert sound into a picture and hand it to an image classifier. The trick works for a reason, and understanding that reason tells you exactly when it stops working.

The spectrogram-as-image move

A bird vocalisation is a pattern in time and frequency — a rise, a trill, a pair of notes at a fixed interval, a buzz with harmonics at particular spacings. Rendered as a mel spectrogram, that pattern becomes a localised two-dimensional shape, and shape recognition in two dimensions is precisely what convolutional networks are good at. The inductive bias transfers because the biases match: translation invariance along the time axis is desirable (the call means the same thing wherever it starts in the window), and limited translation invariance along the frequency axis is desirable too, since individual birds vary in pitch.

Note that the frequency-axis invariance is only partly true, and this is where the picture analogy leaks. A shape shifted up an octave in an image is the same object; in a spectrogram it may be a different species. Networks handle this by having enough depth to encode absolute frequency position, and it is one of the reasons audio convolutional models are not simply image models with a different input.

What the input actually looks like

BirdNET is the reference system, and the BirdNET-Analyzer model documentation publishes its geometry. V2.4, released in June 2023, works at 48 kHz on three-second segments and computes two mel spectrograms rather than one: a low-frequency channel covering 0–3000 Hz and a high-frequency channel covering 500–15000 Hz. Those feed an EfficientNetB0-like backbone with a 1024-dimension embedding, and the classifier covers 6,522 classes including 11 non-event classes. The documentation gives 0.826 GFLOPs per segment and 50.5 MB as FP32.

The dual-channel split is the interesting design decision. A single mel bank spanning 0–15 kHz would allocate most of its resolution to the low end where the mel scale is dense, and most passerine song sits between 2 and 8 kHz. Splitting the range lets the low channel resolve the fundamental structure of doves, owls and grouse while the high channel keeps fine resolution where warblers and kinglets live. One transform cannot do both without more bands and more compute.

Three seconds is also a decision with consequences. It is long enough to contain a complete phrase for most species and short enough that a detection localises the event usefully. It is too short for species whose diagnostic feature is the structure of a whole song lasting ten seconds, and those species are systematically harder.

Why the dawn chorus breaks it

At dawn, ten species sing at once. In the spectrogram this is additive: the energy of every simultaneous vocalisation lands in the same time–frequency plane, and there is no separation step. Three specific things then go wrong.

• Occlusion in frequency. Two species with overlapping frequency ranges produce bins containing the sum of both. The convolutional filters that learned a clean shape now see that shape with extra energy laid over it, which is not a small perturbation — it changes the local contrast the filters respond to.
• Confidence dilution. A three-second segment with five species produces five moderate scores rather than one high one. Under a fixed minimum-confidence threshold, all five can fall below it, so a busy segment yields fewer detections than a quiet one containing a single bird. Detection rate is not a proxy for activity.
• Harmonic collision. One species’ harmonics can land where another species’ fundamental sits. This produces genuine, systematic confusions between particular pairs rather than random noise, which means your error structure is species-dependent and cannot be summarised by one accuracy figure.

The structural mitigation is the same one described in multi-label audio tagging: independent sigmoid outputs, so multiple species can be reported from one segment without competing. Overlapping segments help too — sliding the three-second window with a one-second hop gives each vocalisation several chances to appear in a segment where it happens to be the loudest thing present.

Location and date are features too

BirdNET ships a separate species range model, updated in January 2024 to incorporate eBird distribution data, which predicts which species are plausible at a given latitude, longitude and week of year. Applied as a filter or a prior over the classifier’s outputs, this removes a large class of errors cheaply: an acoustically similar confusion between two species is trivially resolved if one of them has never been recorded on that continent.

Verifying a detection

A confidence score from any of these models is a monotone ranking signal, not a calibrated probability, and it should not be reported as one. The workable protocol is to pick a threshold, then draw a random sample of detections at and just above that threshold and have someone listen to them, which gives you a precision estimate with a confidence interval for your recorder, your site and your season. Repeat per species, because precision at a fixed threshold varies enormously across the 6,000-odd classes.

Recall is harder and requires annotated audio: someone listens to a sample of raw recording and marks everything audible, and you count what the model missed. It is expensive, it is the only way to know, and skipping it is how monitoring programmes end up reporting a decline that is actually a change in background noise. The cost of running this at scale, in both storage and compute, is worked through in bioacoustic monitoring of wildlife.

Finally, check the licence before building anything commercial. The BirdNET documentation states the source code is MIT while the models are licensed CC BY-NC-SA 4.0 — a non-commercial share-alike term that is a real constraint on a product, and one that is easy to miss because the repository’s headline licence is the permissive one.