Defect Detection in Manufacturing With Computer Vision

A defect detector has one job that is not “be accurate”: it has to sit at an operating point where the cost of what it misses plus the cost of what it wrongly rejects is as low as you can make it. Those two costs differ by orders of magnitude on most lines, and the threshold that minimises their sum is nowhere near the one that maximises accuracy.

The imbalance that breaks the usual metrics

Take a line running 20,000 units per shift with a defect rate of 0.4%. That is 80 defective units and 19,920 good ones. A model that predicts “good” for every single unit scores 99.6% accuracy and catches nothing. Accuracy is not merely a weak metric here; it is actively misleading, and any dashboard that leads with it should be changed before anything else is.

What to report instead is recall on defects (of the 80, how many were flagged), false-reject rate on good units (of the 19,920, how many were stopped), and the two costs those imply. Precision is worth watching but is not the operational quantity, because it mixes both populations into one number and hides which one moved. The general relationship between these metrics holds here; what is specific to a production line is the extreme asymmetry of what each error costs.

The cost of an escape, worked

Every figure below is an assumption, stated so you can substitute your own. The arithmetic is the part worth keeping; the inputs are yours to measure from your own scrap, warranty and re-inspection records.

ASSUMPTIONS (substitute your own)
  units per shift                  20,000
  defect rate                       0.4%   -> 80 defective, 19,920 good
  cost of one escape (field)         900   currency units
  cost of one false reject            12   currency units (manual re-inspect)

OPERATING POINT A  (high precision)
  recall on defects                 0.95   -> 76 caught, 4 escape
  false-reject rate on good units   0.5%   -> 19,920 * 0.005 = 99.6 -> 100

  escape cost         4 * 900  =  3,600
  false-reject cost 100 *  12  =  1,200
  total per shift              =  4,800

OPERATING POINT B  (threshold lowered)
  recall on defects                 0.99   -> 79.2 caught, 0.8 escape
  false-reject rate on good units   2.0%   -> 19,920 * 0.02  = 398.4 -> 398

  escape cost       0.8 * 900  =    720
  false-reject cost 398 *  12  =  4,776
  total per shift              =  5,496

On these inputs, point A wins — the fourfold increase in false rejects costs more than the 3.2 escapes it prevents. Change one assumption and the answer flips. Raise the escape cost to 3,000, which is unremarkable for anything safety-related or anything that triggers a recall, and point A costs 12,000 + 1,200 = 13,200 against point B’s 2,400 + 4,776 = 7,176. Point B wins by a wide margin.

That is the whole argument for doing this arithmetic before choosing a threshold instead of after. The break-even is where the marginal escape cost equals the marginal false-reject cost, and moving between A and B costs 298 extra false rejects to prevent 3.2 escapes — a ratio of 93 to 1. So point B is correct exactly when one escape costs more than 93 false rejects. With a false reject at 12, that is an escape cost above roughly 1,120. Nothing about the model is involved in that calculation; it is a property of your business that you can compute before any model exists.

Moving the operating point

Moving between A and B does not mean retraining. It means changing the score threshold at which a unit is flagged, which slides you along the model’s existing precision-recall curve. Two practical points follow. First, produce that curve on a held-out set that reflects the real defect rate — a validation set balanced 50/50 for training convenience gives a precision curve that is wildly optimistic relative to a 0.4% line. Second, the threshold is a number in a config file and should be treated as one: versioned, reviewable, and changeable without a model deployment.

The scores you are thresholding also need to mean something stable across time, which raw neural network outputs do not — see why a classifier’s 0.9 is not a 90% chance. A calibrated score lets you write the threshold in terms of the cost ratio you just derived rather than as a magic constant somebody tuned in a meeting.

Supervised detection or anomaly detection

There are two ways to build this and the choice is decided by whether your defect set is closed. Supervised detection needs labelled examples of each defect type; it is more accurate on those types and it is blind to a mode it never saw. That blindness is the real risk, because new defect modes are exactly what a process change produces, and the model will pass them silently at a high confidence.

Anomaly detection inverts the problem: train only on good units, model what normal looks like, and flag deviation. It needs no defect labels, which matters enormously when defects are rare enough that collecting 200 examples of one takes a year. It also produces a per-pixel or per-patch score rather than a class, so an operator sees where the model objected. The cost is a higher false-reject rate and no defect taxonomy — it tells you something is wrong, not what. The camera-side anomaly detection setup covers that path; the general framing is in anomaly detection.

Most working lines end up with both: an anomaly model as the safety net for unknown modes, and supervised classifiers for the handful of known modes where the specific label drives a specific corrective action.

The physical setup is most of the accuracy

The largest single accuracy gain available on a defect line is usually not in the model. A defect is visible or it is not, and that is decided by the optics. Coaxial or dark-field illumination makes a surface scratch that is invisible under diffuse light appear as a bright line; a polarising filter suppresses specular glare that otherwise saturates the sensor exactly where the defect is; a telecentric lens removes the perspective change that makes the same part look different at the two ends of the field of view.

Resolution has to be checked against the defect, not the part. If the smallest defect you must catch is 0.2 mm and the field of view is 300 mm across, a 4,096-pixel-wide sensor gives 0.073 mm per pixel, so that defect spans under three pixels — near the limit at which any detector can separate it from sensor noise. The usual working rule is to want the smallest feature covered by several pixels rather than one, and if the arithmetic says you do not have that, no model will fix it.

The related failure is that the setup drifts. A lamp ages, a lens collects dust, a fixture is knocked, and the images move away from the distribution the model was trained on while every unit test still passes. Monitoring mean image brightness and a focus measure per shift catches this long before the defect escape rate does, and it is the specific form of the gap between validation accuracy and production accuracy that vision on a factory floor is most prone to.