The AI decides use-as-is. It learned that from you.

Here is an uncomfortable observation from two decades of non-conformance reviews: the most expensive signature in a plant is the one nobody remembers making. Use-as-is. Not scrap, not rework – use-as-is, the disposition that ships the problem to the customer and closes the file with a clean little label.

I have sat in those reviews. Friday, 16:40, the line is bleeding €40,000 an hour, and there is a cosmetic deviation on a stamped bracket. Someone reasonable says the sentence that ends all debate: it will never see load in service. Perhaps true. The stamp comes down. No concession note to the customer, because nobody wants to wake the customer.

Now multiply that scene by ten years and a dozen production lines. That residue – thousands of quiet, undocumented judgements – is what just went into your new AI disposition assistant. And it learned everything.

The model learns the label, not the truth

Strip away the vendor slides and an AI triage tool is a pattern-matcher fed your disposition history. If 78% of cosmetic findings on a given part family were closed use-as-is, that number becomes the model's prior before its first inference. It cannot distinguish the deviations your best engineer examined for forty minutes from the ones a tired shift leader waved through in ninety seconds. The labels look identical. To the mathematics, they are identical.

I deploy these systems; I do not train foundation models – and the hard engineering was never the models. An orchestration platform I designed runs 63+ models in parallel with consensus synthesis, and the nastiest problem by far was deciding what counts as ground truth. In quality data, ground truth is whatever the last signer believed on the day they signed.

A neural network has never had a production manager leaning over its desk at 16:40 on a Friday. It does not need the experience. You already encoded the pressure into the data.

The model does not learn what you should have decided. It learns what you actually signed – on the days you were right, and on the days you were in a hurry.

Why use-as-is is the worst thing to teach it

Scrap errors are honest. The cost lands the same week, on your own P&L, and somebody answers for it. Use-as-is errors fly. The part leaves, the file closes, and the consequence – a warranty claim eighteen months later, a field return, in aerospace occasionally worse – never reconnects with the training record. Statisticians call this censoring. It deserves a stronger word.

So the model trains on a dataset where failures of caution are visible and failures of leniency are invisible. It becomes confidently generous in exactly the disposition class that hurts most. That is survivorship bias with a login and a password.

Audit the data before you audit the model

The fix is unglamorous and entirely within reach of a competent quality organisation:

• Measure concession rates per engineer, part family and shift. If one engineer's use-as-is rate sits at triple the peer median, you have found the training-data problem before touching a GPU.
• Capture rationale at the moment of decision. When I stood up the QA function at a 900-person greenfield plant, a deviation log with mandatory reasoning was among the first five systems I built – no AI anywhere; you simply cannot improve what you refuse to see.
• Measure the human override rate. If reviewers approve 96% of AI recommendations, your human-in-the-loop is a signature farm. Sudden agreement is an alarm, not a success metric.
• Reconnect consequences to records. Weld warranty returns and field data back into the disposition database by part number, so the model meets the parts you got away with – and the ones you did not.

None of this is new doctrine. At SNOP, the largest share of a 70% defect-cost reduction came from forcing discipline into deviations – QRQC within 24 hours, A3 for the repeats, no silent concessions. The tools were analog. The principle is identical: the label must mean something before anything, human or model, is allowed to learn from it.

The standards agree, incidentally. AS9100 and IATF 16949 both demand objective evidence and traceable disposition. An AI recommendation that cannot be reconstructed fails the audit on its own, whatever its accuracy in testing – and the auditor would be right.

Red-team your own history

My second trade is ethical hacking – CEH, thirty-plus security certifications, a name on a telecom's bug-bounty hall of fame for a clickjacking disclosure. The mindset transfers directly. Poisoned training data is not always an attack. Frequently it is a tired engineer and a stopping line. So red-team the disposition model the way you would red-team an authentication flow: feed it your borderline cases, your archived field returns, the deviations you survived. If it inherits the Friday-afternoon signature, you have found a genuine vulnerability – and it is not in the code.

Here is the uncomfortable version of progress. An AI reviewer does not add risk to a weak disposition process; it industrialises the risk you already carry, at machine speed, without the hesitation that occasionally saved you. Clean the labels. Capture the rationale. Watch the override rate. Red-team the priors. Then let it read your non-conformances.

Your AI will be the most honest auditor you have ever hired. It will report, with flawless confidence, exactly what you taught it.