The bias is audited. The drift isn't.

In March, a surface-inspection vision system at a Tier-1 exterior-parts supplier clears its bias audit. Confusion matrices per defect class. False-reject rates balanced across shifts and part families. A 40-page report, three signatures. In October the paint supplier quietly changes the flattening agent in the clearcoat. Sheen drops half a gloss unit – invisible to the eye, decisive to the pixels. The model, validated on the old sheen, stops seeing an entire defect class: fine orange peel folds into statistics it never learned. Runs of defective tailgates ship to the OEM. Nothing broke. The audit on the shelf is still green. Nobody re-audited, because the change request mentioned gloss units, not models.

This week the governance industry is out in force – bias-detection checklists from TechTarget, a moral-urgency push from MIT Sloan, a new enterprise governance platform from AvePoint. All of it answers how you evaluate a model at a point in time. Almost none of it answers the practitioner problem: the plant will invalidate that evaluation within two quarters, and it will do so silently.

What a bias audit actually measures

A bias audit is a comparison between a frozen model and a frozen dataset. Strip the ceremony away and that is all it is. Both artefacts are historical by the time the report reaches the quality director's desk. The dataset is last year's production. The model has often been retrained since. The line has moved on regardless. The audit certifies the model against a benchmark; it says nothing about the materials, suppliers and settings the model will actually run on. That is receiving-inspection logic. You inspected one incoming lot and certified the supplier in perpetuity.

My ethical-hacking instincts apply directly here. Anything validated once starts decaying the moment validation ends. A penetration test is stale by the next patch cycle. A model report is stale by the next lot change. A point estimate is not a control.

Why the plant guarantees drift

Every 4M change – man, machine, material, method – moves the data distribution under any model watching the process. A new coil supplier changes surface topography; a different resin batch changes how flow marks photograph. Swap the hall lighting to LEDs to cut the energy bill and you have re-baselined every camera on the line. Cut cycle time by 8% and the blur on your weld images is new physics. We accept physical drift as normal. The torque wrench gets quarterly calibration, the CMM gets interim checks, an engineering change forces requalification of the process. Software gets a pass, because it shows no wear.

We calibrate the torque wrench every quarter and trust the neural network forever, because software shows no wear and we mistake silence for stability.

When I built a greenfield QA/QC department for 900-plus people, calibration cadence went in on day one. Every gauge on an interval. Seeded-defect masters pushed through the vision cells and the manual stations alike – a station that could not find the seeded defect was out of service until it could. Somewhere between the gauge and the model behind it, we forgot our own logic. Why does the camera get a schedule and the classifier get none?

Controls that survive drift

What I would wire into any plant running models in the loop, in ascending order of cost:

• Golden-sample cadence. Seeded-defect masters run at fixed intervals through every automated station. A miss is a failed measurement-system analysis: stop release, contain, investigate. Masters age and get consumed – budget their replacement like gauge blocks.
• Redundant models with a consensus threshold. Building MultiPS – 63+ models running in parallel with consensus synthesis – taught me that model disagreement is a measurement signal, not noise. On a line, run three models over the same image: agreement is the measurement, divergence past a threshold is the alarm. Two gauges disagree, you check the gauges before you argue about the part. Same reflex.
• Revalidation wired into change management. Every 4M change request lists the models in its blast radius. No requalification, no release – the PPAP gate we already force onto tooling and gauges, applied to software.
• Attribute agreement on human overrides. Chart override rates and run attribute agreement analysis monthly. Inspectors notice drift before dashboards do. A rising override rate is the cheapest early-warning system you own.

Nobody budgets this layer. A bias audit is a project – one invoice, one deliverable, one slide for the board. Drift control is operating expense: masters, line time, triple inference. A golden-sample programme on a single line runs perhaps €25,000–€40,000 a year. Running three models instead of one costs fractions of a cent per part. Set that against one Class-A surface escape: sorting, containment and revalidation of a shipped week will consume five years of that budget. And the commercial direction is hardening. Electronics makers are signing AI production contracts with fees tied to results; zero-defects programmes are being sold on escape rates. In that world, silent drift is not a quality nuisance. It is an unbudgeted liability.

A bias audit is receiving inspection – one lot, one gate, one snapshot. Drift control is SPC for models: a time series, control limits, a reaction plan, requalification triggered by change management instead of a customer complaint. The October escape in my opening did not begin in October. It began in March, the day a point-in-time report was filed and called control. Put the model on a calibration schedule like every other gauge you own. The plant will keep changing underneath it whether or not anyone signs for the change.