Three announcements in one week. P&G is scaling AI-based inspection across its operations. Siemens already runs AI-powered inspection at global scale. And AWS packages the whole thing as a purchasable toolchain – cameras, models, plumbing, invoice. The camera is officially a line item. Which means the failures that matter have moved: out of the model, into the words. I have seen one surface flaw logged as a scratch at one plant, a tool mark at a second and a cosmetic nonconformity at a third. Three severities. Three reaction plans. One model expected to serve all of them.
The confusion matrix is an audit you never authorised
When a single trained model meets twenty plants, its misclassification report doubles as a map of organisational disagreement. The industry reads that report as a model defect. It is usually a taxonomy defect. The pilot passed because it inherited one plant's definitions – one dialect, one lighting booth, one inspector trainer who taught everyone what a scratch looks like. The rollout inherits every dialect at once. When plant B's tool-mark cluster keeps landing in plant A's scratch row, the model is not failing. It is reporting, with terrible patience, that your plants have never agreed.
I lived this on the unglamorous side. Leading quality initiatives across a 2,000+ workforce spread over multiple sites, the hardest work was never the audits or the 8Ds – it was getting several plants to call the same weld spatter the same thing, so an escalation call could last fifteen minutes instead of an hour. Harmonising defect codes and escalation language was the project underneath every other project. Nobody funded it as one.
An inspection model does not create disagreement between your plants. It photographs it.
The defect code is load-bearing, not labelling
Quality people treat defect codes as labelling. They are structural. PFMEA severity keys off the defect type; detection ratings assume it. Containment triggers fire on specific codes. Escape-rate trends are code trends. Let the code vary by site and your escape data becomes noise wearing a dashboard.
A VDA 6.3 auditor taught me this the expensive way. Sampling nonconformance records across two sites, he pulled the same physical defect filed under three different codes and asked the room one question nobody could answer: which escape trend should I believe? We closed the finding in six weeks. The trend stayed unreliable for a year, until the codes behind it were harmonised.
QRQC works at speed for one reason: everyone escalates in the same language, so the five whys start at the part instead of at the dictionary. A model feeding that loop inherits the same obligation. Feed it a code nobody downstream agrees on and you have installed a translator inside a process whose entire value is the absence of one.
One dictionary, no committee war
The fix is unglamorous and cheap next to the alternative. Freeze a top-50 defect code set before rollout, not after the first confusion matrix. Fifty codes cover the Pareto at almost any plant I have audited, and capping the list changes the meeting. Committees fight over vocabulary because vocabulary feels like identity. Give them fifty bars and they fight over the Pareto instead, which is a fight worth having.
Then treat inter-plant disagreement as a measurable. Run an attribute agreement study – an MSA for humans. Same 30 parts to each site's inspectors; compute the disagreement, publish it, re-rate quarterly. You would reject a gauge with 60% attribute agreement. Unharmonised inspector networks sit roughly there, and that is the hard ceiling on any model trained on their labels. Fix drift through reaction-plan discipline, not re-labelling marathons: when a code drifts, the reaction plan – not the label – protects the customer while you argue.
On cost, I can be specific, because I have paid both bills. I built the defect taxonomy from zero at a greenfield plant of 900+ employees: codes, severities and reaction plans defined before the first part shipped. Weeks of argument with engineering and the customer, upfront, in one room – the cheapest quality money that plant ever spent. Retrofitting a common taxonomy onto sites that have argued in their own dialect for a decade is slower and meaner. Figure ten to twelve weeks of cross-site engineering time, call it €80,000–120,000 with travel and retraining, before anyone retrains a model on labels the sites finally share. Greenfield is cheaper. Retrofit still beats the alternative: one serious customer escape burns more than that in containment, sorting and credibility – and the customer remembers. I would sign the retrofit invoice without flinching.
Key takeaways
- Freeze a top-50 defect code set before the multi-site rollout – a pilot's definitions never survive contact with a second plant's dialect.
- Run an attribute agreement study across your own inspectors, quarterly. Human disagreement between sites is the mathematical ceiling on any model you deploy.
- Wire every code to a severity, a detection assumption and a containment trigger – drift then gets caught by reaction-plan discipline, not by re-labelling marathons.
- Budget the taxonomy as part of the AI programme. A retrofit costs ten to twelve weeks and €80,000–120,000; a single customer escape costs more, and unlike the retrofit, it buys you nothing.
The hyperscalers will commoditise the eye. Siemens, AWS, whoever ships next month – each release sharper and cheaper than the last, and none of it will settle a scratch-versus-tool-mark argument between two plants that have disagreed since before the model existed. The dictionary stays yours, because nobody can sell it to you. Plants that agree on their words will compound returns from every model they deploy. Plants that don't will keep buying sharper vision for the same blind spots. The model scales. The vocabulary has to be built – by hand, in a room, by people who had no reason to agree until now. Book the room before the rollout, not after the confusion matrix.