Video Content Moderation: What Gets Checked Frame by Frame

Nobody classifies every frame of an uploaded video. The pipeline picks some frames and ignores the rest, and the rate at which it picks is the single decision that sets both the compute bill and what gets through. Almost every other choice in a moderation pipeline is downstream of it.

A video is four separate things to check

Before any sampling question, it is worth being clear about what is being checked, because a pipeline that only looks at pixels misses most of what policy teams actually care about. A single upload carries four largely independent signals.

• The image track. Frames, sampled and passed to classifiers or a vision model. This is the expensive one and the rest of this page is mostly about it.
• The audio track. Transcribed, then run through the same text classifiers the rest of the product uses. Audio is cheap relative to video and covers a category of violation the image track cannot see at all, which makes it the highest-value channel per unit of compute.
• Text rendered into the picture. Burned-in captions, overlaid slogans, a sign held up to the camera. This is not in the transcript and it is not something a general image classifier reads reliably, so it needs its own pass — see reading on-screen text in video frames.
• Everything outside the media. Title, description, thumbnail choice, uploader history, and hash matches against known material. The thumbnail matters more than its size suggests: it is the one frame the uploader chose, and it is shown to everyone whether or not they play the video.

The frame count nobody wants to say out loud

Take a single one-hour upload at 30 frames per second. That is 3,600 seconds of footage and 108,000 frames. If each frame goes through one image classifier, that is 108,000 inference calls for one video. At any realistic upload volume this is not a budget question, it is an impossibility, so the pipeline samples.

one hour at 30 fps        3600 s x 30       = 108,000 frames
sampled at 1 fps         3600 s x 1        =   3,600 frames   (3.3% of them)
sampled every 5 s        3600 / 5          =     720 frames   (0.67%)
sampled every 10 s       3600 / 10         =     360 frames   (0.33%)

Those four numbers are the whole trade-off in one block. Going from one frame a second to one frame every ten seconds is a tenfold reduction in classifier cost for the same video. It is also a tenfold increase in the length of footage that can pass without a single frame of it being looked at, and that second sentence is the one that gets skipped when the decision is made on a cost spreadsheet.

Decoding is a separate cost from classification and it does not shrink when you sample. A compressed stream is a sequence of keyframes and predicted frames, so to produce the frame at 00:04:37 you generally have to decode forward from the preceding keyframe. Seeking to sampled timestamps in a long file can therefore cost more wall-clock time than decoding the file straight through once and discarding what you do not need. If sampling is sparse, align the sample points to keyframes and the decode collapses to a keyframe-only pass.

What a sampling rate actually costs you

Uniform sampling has a clean coverage property, and it is worth stating precisely because the intuition is usually optimistic. Suppose the material you need to catch is present for a continuous stretch of D seconds, and you sample one frame every S seconds at fixed offsets. A sample lands inside that stretch if and only if the stretch spans one of the sample points. With the stretch starting at a time uniformly distributed relative to the sampling grid, the probability that at least one sample lands inside it is min(1, D / S).

sampling every 1 s
  a 5 s segment    D/S = 5/1  = 1.00   always sampled, ~5 frames of it
  a 1 s segment    D/S = 1/1  = 1.00   always sampled, exactly 1 frame
  a 0.4 s flash    D/S = 0.4  = 0.40   caught 2 times in 5

sampling every 10 s
  a 5 s segment    D/S = 5/10 = 0.50   caught half the time
  a 0.4 s flash    D/S = 0.04 = 0.04   caught 1 time in 25

Two consequences follow. The first is that short insertions are the adversarial case by construction — a single offending frame spliced into an otherwise clean video has a probability of detection equal to your sampling rate divided by the frame rate, which at one frame per second in 30 fps footage is about 3%. That is not a model quality problem and no better classifier fixes it.

The second is that if your policy requires two independent flagged frames before it acts — a common way to suppress false positives — the coverage figure above is the probability of getting one, and the probability of getting two from a short segment is far lower. A two-frame rule applied on top of one-frame-per-second sampling effectively raises the minimum duration you can catch to two seconds. Decide the sampling rate and the confirmation rule together, because they multiply.

Sampling on content instead of on the clock

Uniform sampling spends its budget in proportion to duration, which is the wrong denominator. A ten-minute static shot of a lecture slide gets 600 samples of the same image; a rapid-cut sequence with forty different scenes gets forty seconds of coverage each. Content-adaptive sampling fixes the allocation without raising the total.

• Sample per shot. Run shot boundary detection first and take a fixed number of frames from each shot. The budget then follows visual variety rather than the clock, and a long static interview stops consuming the same budget as an action sequence.
• Drop near-identical frames. A perceptual hash per candidate frame, discarding anything within a small Hamming distance of the last kept frame, removes redundancy before it reaches the classifier. The arithmetic is worked out in frame deduplication before sending to a model.
• Keep a floor. Content-adaptive sampling has one failure mode and it is severe: a video engineered to look like a single static shot gets almost no samples. Always keep an unconditional minimum rate — one frame every N seconds regardless of what the shot detector says — so the worst case is bounded by the clock rather than by an attacker’s editing choices.
• Escalate rather than decide. A cheap first-pass classifier at a low threshold on sparse samples, followed by a dense re-sample of any region that scored near the boundary, spends most of the budget on the small fraction of footage that is ambiguous. This is the structure that makes a real pipeline affordable.

Precision at review volume

The last piece of arithmetic is the one that decides whether a pipeline is operable, and it has nothing to do with model quality in the abstract. Assume, purely as a worked figure, a service taking 100,000 one-hour uploads a day and sampling at one frame per second. That is 360 million classifier calls a day. A false-positive rate of 0.1% — which would be an unremarkable number on most classification tasks — produces 360,000 spurious flags in a day.

No human review team absorbs that. This is why production moderation aggregates before it flags rather than after: a video is escalated on a count of flagged frames, or on flagged frames clustered in time, or on a score aggregated across the whole upload, and the per-frame classifier output is treated as evidence rather than as a decision. It is also why the threshold on the aggregate is tuned against review capacity and not only against a validation set — a threshold that produces more work than the team can do is a threshold that produces unreviewed queues, which is worse than a stricter one.