Skip to content
Multigrid

Classical ML in production

The working practice of machine learning on tables: label definitions, leakage, calibration, thresholds and causal claims, worked with numbers.

Most of the money made from machine learning is still made on tables. Someone is deciding which transaction to hold, how many units to order, which customer is worth a discount and which loan to decline, and the model doing it is almost always a gradient-boosted ensemble over a few dozen columns. That work predates the current wave of language models and has been largely untouched by it.

These twenty pages are about the parts of that work that decide whether the model earns anything, which are rarely the parts about the model. Defining the label correctly, keeping the future out of the training set, turning a score into a decision at the right threshold, knowing whether a 0.8 means eighty per cent, and being honest about what an observational dataset can support are all harder than fitting, and all of them are where deployed projects quietly fail.

The theory sits one level up, in the machine learning cluster, and these pages link to it rather than restating it. What is here is the practice, with the arithmetic done on the page.

Tabular Data Is Still Where Most ML Money Is

The task types that make up production machine learning on tables, and the three structural reasons gradient-boosted trees still win them.

9 min read

Gradient Boosting, Explained by Building Three Trees

Three boosting rounds worked by hand on six rows, with the residuals and the squared error after each tree, then the generalisation to any loss.

11 min read

Random Forests and Bagging: Where the Variance Goes

Why averaging trees reduces error, with the ambiguity decomposition worked on five predictions and the correlation floor that explains feature subsampling.

10 min read

Time Series Forecasting: Baselines Before Models

The four baselines every forecast must beat, MASE worked over two weeks of data, and the backtest that stops a model looking better than it is.

11 min read

Anomaly Detection in Production Data

Four detection methods ordered by the alert volume each one produces, derived from its own threshold arithmetic, with the code for the one that survives contact.

11 min read

Churn Prediction End to End

Defining the churn label with an observation window, a blackout and a label window, the five leakage traps inside it, and the pipeline that survives them.

12 min read

Demand Forecasting for Inventory: The Loss Is Not Symmetric

Why forecasting the mean is the wrong target for an inventory decision, with the critical fractile derived and the ordering decision worked to the pound.

11 min read

Credit and Risk Scoring Under Regulation

How a scoring model is built when a decline has to be explained, covering reason codes, monotonic constraints, proxy variables and population stability.

12 min read

Uplift Modelling: Who Would Have Bought Anyway

The four-quadrant framing of treatment effects, and the arithmetic showing an accurate churn model driving a campaign to a loss of £191,000.

11 min read

A/B Testing Machine Learning Systems

Sample size derived from first principles, the interference that makes marketplace tests invalid, novelty effects, and committing to the metric before the split.

12 min read

Feature Stores and the Problem They Solve

Train-serve skew shown with one feature computed two ways, the point-in-time join that prevents it, and an honest account of when a feature store is overkill.

10 min read

Model Drift and When to Retrain

Data drift and concept drift are different failures with different monitors, and label delay decides how long you cannot tell which one you have.

11 min read

Data Leakage: The Most Common Silent Failure

Six leakage patterns, each with the exact line of code or join that causes it, the fix, and the tests that catch leakage before production does.

12 min read

Calibration: Making 0.8 Mean 80%

A reliability table from ten buckets of predictions, the expected calibration error summed row by row, and both standard corrections worked with numbers and code.

11 min read

Choosing a Decision Threshold With Real Costs

A four-cell cost matrix in, an optimal threshold out, derived and then verified against total cost over 10,000 scored transactions.

10 min read

Survival Analysis for Product Questions

Kaplan-Meier worked on eight customers, showing how dropping censored rows understates median lifetime by a quarter, and where Cox models belong.

11 min read

Causal Inference When All You Have Is Observational Data

Confounders, colliders and the backdoor criterion, with a reversal worked in full and a template for what an observational estimate may honestly claim.

12 min read

Clustering That Produces Segments People Use

k-means through HDBSCAN, with silhouette worked, the stability test that decides whether the clusters are real, and the business check that decides whether they matter.

11 min read

Machine Learning With 400 Rows

How many features 400 rows can support, how wide the confidence interval on your accuracy really is, and the point at which a rule beats a model.

10 min read

Putting a Classical Model and an LLM in One System

Three arrangements that work — router, feature extractor and verifier — with the cost arithmetic that decides whether the router is worth building.

11 min read

Other topics