Detecting Applause, Laughter and Crowd Reaction in Recordings

Applause and laughter are trivially distinguishable to a person and routinely confused by a classifier. The acoustic reason is specific, and understanding it tells you exactly which segments will be wrong.

Two very different signals

Applause is a dense stochastic impulse train. Each clap is a short broadband transient with no harmonic structure, and a room of people clapping produces hundreds per second at random phase. Summing many uncorrelated impulses approaches noise, so applause has a flat-ish spectral envelope, a high spectral flatness measure, no detectable pitch, and a temporal fine structure that is dense and aperiodic. Its onset is fast and its offset is gradual as people stop at different times.

Laughter is voiced speech. It has a fundamental frequency and a full harmonic series, formant structure imposed by the vocal tract, and — the distinguishing feature — a strong syllabic rhythm. Laughter bouts consist of repeated call units at a rate in the region of four to five per second, which is close to the syllable rate of speech and produces a clear peak in the modulation spectrum around that frequency.

So a feature set that separates them well is not exotic: spectral flatness, harmonic-to-noise ratio or a pitch-strength measure, and the energy in the 3–8 Hz band of the amplitude modulation spectrum. Applause is flat, unpitched, and modulated only weakly and broadly. Laughter is peaked, pitched, and modulated sharply in the syllabic band. A learned model finds the same distinctions; naming them tells you when they will fail.

Where the two collapse into each other

They fail in four identifiable situations, and every one of them is a case where the distinguishing property is genuinely absent from the signal.

• Sparse applause. Three people clapping is not a dense impulse train — it is three impulses, individually resolvable, with large gaps. Its statistics look nothing like the full-house case the model learned, and the impulses can read as consonantal onsets.
• Crowd roar laughter. A large audience laughing together sums many uncorrelated voices at different pitches, and the harmonic structure of each is destroyed by the superposition. The result is broadband, unpitched noise with a fast onset — which is the description of applause.
• Simultaneity. Laughter that turns into applause passes through a region where both are present, and a single-label classifier must pick one. A multi-label formulation with independent per-class sigmoids is the correct design here; a softmax over mutually exclusive classes builds the error in at the architecture level.
• Post-production. Broadcast audio is compressed, limited and often has an added laugh track at a fixed level. Dynamic range compression raises the noise floor between claps and flattens the modulation depth that distinguishes the two, and it is applied precisely to the loud crowd segments you care about.

A worked confusion matrix

Take a balanced two-class evaluation of 1,000 one-second segments: 600 applause, 400 laughter. The counts below are a worked illustration, not a measurement, chosen to show what the metrics do with a realistic asymmetric confusion.

                      predicted
                  applause  laughter   total
  actual applause      540        60      600
  actual laughter       90       310      400
  ------------------------------------------
  predicted total      630       370    1,000

  accuracy = (540 + 310) / 1,000 = 0.850

  applause: precision = 540/630 = 0.857
            recall    = 540/600 = 0.900
            F1        = 0.878

  laughter: precision = 310/370 = 0.838
            recall    = 310/400 = 0.775
            F1        = 0.805

  macro-F1 = (0.878 + 0.805) / 2 = 0.842

The asymmetry is the informative part. Laughter recall (0.775) is well below applause recall (0.900) because 90 laughter segments were called applause — the crowd-roar case above, where the harmonic structure is gone. Meanwhile applause precision suffers from the same 90 errors arriving as false positives. One physical failure mode shows up as a recall problem in one class and a precision problem in the other, which is why reading a single macro-F1 tells you nothing about what to fix.

What happens on a real timeline

Now run the same model over an hour of broadcast. Assume 3,600 one-second segments composed of 72 seconds of laughter (2%), 108 seconds of applause (3%), and 3,420 seconds of everything else — speech, music, silence, room tone.

Apply the rates from the evaluation. Laughter recall of 0.775 finds 56 of the 72. Applause misclassified as laughter at 60/600 = 10% adds 11 false positives. And the third category, which was not in the evaluation at all, contributes at whatever rate it does — say 1%, which is optimistic for a model that never saw speech during evaluation:

  true positives            0.775 * 72     =  56
  FP from applause          0.10  * 108    =  11
  FP from speech/music      0.01  * 3,420  =  34
  ------------------------------------------------
  precision = 56 / (56 + 11 + 34) = 56/101 = 0.554

Precision falls from 0.838 to 0.554, and the largest single source of error — 34 of the 45 false positives — comes from a class that was never evaluated. The applause confusion the evaluation was designed around contributes a third as much, because applause is 3% of a broadcast and was 60% of the test set.

That is a different failure from the prevalence problem in a home sound detector, where the classes are right and the base rate is wrong. Here the base rate is also wrong, but the deeper issue is that the evaluation omitted the class that dominates deployment. The fix is a background or “none of the above” class populated from real material at real proportions, and an evaluation set that is a sample of the timeline rather than a balanced collection of examples. Balanced test sets are for comparing models; timeline samples are for predicting behaviour, and only one of those is what a product needs.

Segments, events and boundaries

One more layer sits above the classifier and it changes the numbers again. Most uses of crowd reaction detection are not per-second: they want the moments — where the audience laughed, so a chapter marker or a highlight can go there.

That means merging consecutive positive segments into events with a gap tolerance, and evaluating events against reference events with an onset collar. Both parameters are consequential. A one-second merge gap splits a laugh that dips briefly into two events; a five-second gap merges two separate reactions. Report the gap and the collar alongside any event-level score, because the score is not interpretable without them.

The final practical note is on class definition. “Crowd reaction” as a single class covering laughter, applause, cheering and booing is easier to detect than any of its members and is sufficient for chapter marking, since all four mark the same kind of moment. Splitting them is only worth the precision cost if a downstream consumer treats them differently. Deciding that first saves building a four-way classifier whose confusions nobody was going to act on — and if the reason for splitting is to infer how the audience felt, the limits on inferring affect from vocal signal apply here with full force.