Sports Video Analytics: Tracking Players and Events With AI

“Distance covered” sounds like the easy statistic: track the player, add up the steps. Do exactly that on broadcast footage and you will report a stationary substitute as having run eighty metres. The arithmetic showing why is short, and it explains most of the disagreements between commercial tracking providers.

The pipeline, end to end

Five stages, each of which can be swapped independently.

• Detection. A person detector per frame, plus a ball detector, which is a genuinely hard small-object problem — a football occupies a handful of pixels in a wide broadcast shot and disappears entirely against a crowd.
• Tracking. Association across frames into trajectories with stable identities, exactly as described under tracking with identity. Sports is the adversarial case for appearance-based association: eleven players in identical kit, frequent occlusion, and a camera that pans faster than the players move.
• Team and role assignment. Cluster the appearance embeddings into two kits plus goalkeepers plus officials. Colour clustering handles most of it; the referee in a third kit and a goalkeeper in a fourth are the reason a fixed two-cluster assumption fails.
• Camera calibration. Map image coordinates onto pitch coordinates. Without this every measurement is in pixels and means nothing, because a pixel near the far touchline covers several times the ground of a pixel in the foreground.
• Event detection. Classify or spot passes, shots, tackles and set pieces on the timeline. SoccerNet, released by Giancola and colleagues in 2018, is the standard public benchmark for this — their paper is on arXiv and the project site hosts the current tasks.

Pixels to pitch coordinates

A football pitch is planar, and the mapping between two planes under perspective projection is a homography: a 3×3 matrix with eight degrees of freedom, determined by four point correspondences in general position. The correspondences come free from the pitch markings, whose real-world geometry is fixed by the Laws of the Game — the IFAB recommends 105 by 68 metres for international matches, and the penalty area, centre circle and goal area dimensions are all specified. The IFAB publishes the field-of-play law.

A point is mapped by multiplying the homogeneous image coordinate by the matrix and dividing through by the third component. Two properties follow from that division and both matter. Position error is amplified non-uniformly: the same one-pixel detection error corresponds to a few centimetres near the camera and a substantial fraction of a metre at the far touchline. And the homography must be re-estimated for every frame of a moving broadcast camera, because pan, tilt and zoom all change it; estimating it once from a wide shot and reusing it is a common and serious mistake.

The practical difficulty is that broadcast shots frequently contain too few visible line features to constrain eight parameters — a tight shot on the halfway line may show one straight line and nothing else. Systems handle this by propagating the previous frame’s homography through estimated camera motion and re-anchoring whenever enough features reappear, which means calibration quality varies through the match and so does every measurement derived from it.

Distance covered, worked

Take one play: twelve seconds of broadcast at 25 frames per second, so 300 tracked positions for the player of interest, each in metres on the pitch plane. The obvious estimator sums the Euclidean distance between consecutive positions.

d_total = sum over t of  sqrt( (x_t - x_{t-1})^2 + (y_t - y_{t-1})^2 )

12 s at 25 fps  ->  300 samples, 299 intervals
a player averaging 4.2 m/s covers 12 x 4.2 = 50.4 m of true path

Now add the measurement error that is unavoidably present. Suppose the combined detection and homography error puts each reported position off by an independent Gaussian with a standard deviation of 0.15 metres per axis — an optimistic figure for broadcast footage. Consider the extreme case that isolates the effect completely: a player standing perfectly still for those twelve seconds.

Why the naive total is nearly double

per-axis position error:            sigma   = 0.15 m
error on a DIFFERENCE of two positions:
                                   sigma_d = 0.15 x sqrt(2) = 0.2121 m per axis

the magnitude of a 2-D zero-mean Gaussian is Rayleigh distributed,
with mean  sigma_d x sqrt(pi/2):

  0.2121 x 1.2533 = 0.2659 m expected "step" per frame, from noise alone

299 intervals x 0.2659 m = 79.5 m

reported distance for a player who did not move: ~80 m in 12 s

Every term there is elementary probability, not a measurement. The consequences are exact and they are the reason this statistic is harder than it looks:

• The bias is always upward. Distance is a sum of magnitudes, and a magnitude cannot be negative, so noise can only add. Unlike most measurement error, it does not average out over a match; it accumulates.
• It scales linearly with sample rate. The number of intervals is frame rate times duration, so doubling the frame rate doubles the noise contribution while leaving the true path length unchanged. Sampling more often makes this estimator worse, which is deeply counter-intuitive and catches people every time.
• It scales linearly with position error. Halving sigma halves the spurious distance, so calibration quality translates directly into the headline number.

The fix is not to sample less — that throws away real motion too — but to estimate the trajectory before measuring it. Fit a smoother to the position series, a Kalman smoother with a plausible acceleration model or a Savitzky–Golay filter over a window of a few hundred milliseconds, and compute the path length of the smoothed curve. A velocity threshold that discards steps below a few tenths of a metre per second is the cruder version and introduces a downward bias on genuinely slow movement, so it trades one error for another.

This is why two providers report different distances for the same player in the same match, and neither is lying: they chose different smoothing, different sample rates and different speed thresholds. The same applies to derived statistics — “high-speed running distance” and “sprint count” depend on speed thresholds that are a convention rather than a standard, and a threshold moved by half a metre per second changes a sprint count substantially. Never compare totals across providers without the method.

Events, and what the numbers do not say

Event detection sits on top of the tracking output and is usually scored as action spotting: predict a timestamp and a class, and count it correct if it falls within a tolerance of the true one. That tolerance is doing a lot of work in any published figure — a system that is excellent at five seconds may be poor at one — so read the tolerance before the score, exactly as you would read the IoU threshold in moment localization.

Two structural limitations to state plainly. Broadcast footage shows only what the director chose to show, so off-ball movement — which is most of what players do — is invisible for long stretches, and any total computed from broadcast is an interpolation over those gaps. Dedicated multi-camera installations exist precisely because of this, and they bring their own synchronisation problem. And an automatically detected event is a classifier output with a false positive rate, so a season aggregate built from it inherits that rate without any indication in the number. Report confidence-weighted counts, or validate against hand-annotated matches, before anyone makes a selection decision from a leaderboard your pipeline produced.