Threshold Tuner

Drag a threshold across your own scores and watch precision, recall and expected cost move with it.

Scores and labels, one per lineA score from your classifier, then whether the example should have been flagged. Any score range works — the slider adapts.

Threshold — flag everything at or above 0.5000Data runs from 0.0400 to 0.9800.

β for Fβ1 balances. 2 weights recall; 0.5 weights precision.

Cost of a false positiveAny unit you like — minutes, dollars, apologies.

Cost of a false negative

At a threshold of 0.5000

55.6% precision · 83.3% recall

10 caught, 2 missed, 8 false alarms out of 34 examples.

Precision (blue), recall (orange) and Fβ (green) as the threshold moves left to right. The vertical line is where your slider sits.

True positives / false positives: 10 / 8
False negatives / true negatives: 2 / 14
F1.00 at this threshold: 0.667
Threshold with the highest Fβ: 0.5500 (Fβ 0.690)
Threshold with the lowest expected cost: 0.3400 (cost 12.0)
Cost at your threshold: 18.0
ROC AUC — threshold-free ranking quality: 0.818
Average precision (area under the PR curve): 0.711

Every line parsed as a score and a label.

What this checked: this recomputes the four cells of the confusion matrix at every distinct score in your data, and reports precision, recall, Fβ and expected cost at each. AUC is the rank-based (Mann–Whitney) form with ties averaged; average precision is the step-wise sum, not the trapezoid. It names the line and column of every row it could not parse. It does not know whether your scores are calibrated probabilities, whether this sample resembles production, or whether the costs you typed are the real ones — and a threshold picked on the same data you evaluated on is optimistic.

What this assumes: an example is flagged when its score is greater than or equal to the threshold, so a higher score means "more likely positive". If your score is a distance, where lower means more similar, negate it before pasting. Ties at the threshold all fall on the flagged side. AUC and average precision are undefined when the data contains only one class, and are shown as "—" rather than as a number.

Everything on this page runs in your browser. Nothing you paste is uploaded, logged, or put in the URL — only the settings above the input are, so a configured tool can be linked to.

The threshold is a product decision wearing a lab coat

Precision and recall trade against each other along a single dial, and no amount of model work moves both at once — that is what the curve above shows. Which point you choose is not a modelling question. It is a question about what a false positive costs you compared with a false negative, and it is answerable only by the person who has to handle both. A spam filter and a cancer screen sit at opposite ends of the same curve for reasons that have nothing to do with the classifier.

That is why the cost fields are here. Fill them in with anything on a common scale — support minutes, refunds, hours of review — and the "lowest expected cost" row picks the threshold your own numbers imply. It will frequently disagree with the best-F1 threshold, and when it does, the cost answer is the right one. F1 is only the correct objective when a miss and a false alarm hurt you equally, which is almost never true.

Two traps

First, a threshold tuned on your test set is fitted to your test set. Split the data, tune on one half, report on the other, or expect the production numbers to come in worse. Second, the AUC row is deliberately separate from everything above it: it summarises the ranking regardless of threshold, so it is the number to watch when comparing models and the number to ignore when choosing an operating point.

Threshold Tuner

The threshold is a product decision wearing a lab coat

Two traps

The reasoning behind this

Related tools