Gartner told CIOs this week that AI vendors are not enterprise-ready. Any incoming inspector I have worked with could have written that on one line of a whiteboard. In a plant we have a word for material from an unqualified source: suspect stock. It gets yellow-tagged, moved to quarantine and physically blocked from the line until somebody proves what it is. This month that same unqualified source is scheduling your production, drafting your maintenance instructions and answering your customers – and it was waved in on a demo and a term sheet.
ServiceNow reported enterprise AI investment up 119%, against 22% of organisations holding any governance process at all. Read it like a plant manager: over three quarters of your AI supply base arrives with no inspection, no qualification and no reaction plan. If a fastener survey showed those numbers, you would stop the line and start containment calls before the morning meeting ended.
Incoming inspection for models
Two decades of supplier-quality work under IATF 16949 and VDA 6.3 – at SNOP, at WITTE Automotive, across FOREAST clients like ArcelorMittal – taught me one rule that transfers to AI without translation: you never qualify a part, you qualify the process that makes it. A beautiful sample proves nothing. A stable, monitored, capable process proves everything. Software deserves the same disrespect.
So treat every model as purchased material. Incoming inspection for a model asks what data it was built on, which evaluation suite runs against every release, what regression testing looks like, what the incident history is, and what the system does when it does not know. I carry a CEH and enough bug-bounty scar tissue to treat vendor claims as marketing until tested. When a vendor says the model is enterprise grade, I ask for the failure modes and the escape rate. The ones who cannot answer were never going to pass an audit.
That logic is why I built MultiPS as a multi-model platform instead of a wrapper around one vendor: 63+ models running in parallel with consensus synthesis, because no single model on the market has earned incoming-release status. Run the same request across the fleet and compare. Divergence is the defect signal – the nonconformance reaches you before your customer finds it. Deploying one model with no second source is single-sourcing a critical part from a supplier who reserves the right to change it nightly.
The Friday update is an unannounced engineering change
Your fastener supplier needs written approval to move a hole 0.2 mm, change plating chemistry or swap a sub-supplier, and a PPAP resubmission rides with it. Your AI vendor silently rewrites the engine mid-production on a Friday afternoon. On Monday your work instructions, quotations and supplier emails read slightly differently, and nothing in your records says why. In automotive supplier quality an unnotified change is a violation even when the part still fits.
I once stopped a release because a supplier had switched to a cheaper coating without notification. The parts passed every dimensional check; nothing on the print forbade the chemistry change. We caught it in a routine audit – the surface looked wrong under the light – and froze everything. Two truckloads had already shipped. Sorting, re-plating, containment and the customer conversation cost the better part of €60,000. The change was not the expensive part. Not knowing about it was.
An AI vendor without a change-notification clause is not a partner – it is an unannounced engineering change with a sales team.
The contract fix is short. Pin the model version in production. Define every update as an engineering change: notification with lead time, your written approval, revalidation of critical use cases before switch-over, and the right to refuse. When a vendor says the model improves continuously and you receive improvements automatically, translate it honestly: they will change your production process whenever they like and tell you never.
A control plane is not a control
The market heard the panic and monetised it. Boomi shipped an AI governance control plane; Concentrix bought CastleHill to chase the same budget line. Dashboards will now show you how much ungoverned AI you have. A real control plan has a characteristic, a specification, a reaction plan and a named owner. Governance software without owners, triggers and reaction plans is a smoke detector in a building with no doors.
What works for model incidents is what works for every incident on a shop floor: QRQC. Get to the actual prompt log, not the vendor's summary. Contain the affected use cases. Root cause within 48 hours – model, retrieval, prompt, or the pipeline change nobody announced. Verify the countermeasure and check it holds. Then put acceptance criteria on model outputs the way a control plan treats a weld: sampling plan, tolerance on error rate, control limits. If the error rate drifts, you react now, not at the quarterly review.
Governance sits at 22% because companies bought the software and skipped the operating discipline. A committee is not a gate. Procurement is – the last one management still respects, because it holds the pen on the order.
Key takeaways
- Treat every AI model as purchased material – qualify the vendor's pipeline, evaluation suite and incident history, never the demo.
- Pin model versions in the contract; define every update as an engineering change requiring notification, written approval and revalidation.
- Run QRQC on model incidents and set acceptance criteria on outputs: specification, sampling, reaction plan, named owner.
- Demand a software PPAP: capability evidence, change-notification clause, rollback plan, audit rights. Half the vendor list disappearing means the screen works.
So demand a software PPAP. Capability evidence: evaluation results, regression tests and failure rates against your use cases, not a reference call. A change-notification clause with approval rights. A rollback plan with pinned versions and a tested way back. Audit rights, including the right to test claims rather than read them – the instinct that put me on a bug-bounty hall of fame belongs in supplier contracts too. Then run every AI vendor, current and prospective, through the same review your fastener supplier faces quarterly. Expect half the list to disappear. That is not a procurement failure; it is the screen finally switched on. Use the gate.