A Saturday Essay From The Head Of An AI Vendor
On Saturday 10 October, Satya Nadella published an essay on his personal blog titled 'Models as Insider Risks in the Super Intelligence Era'. TechCrunch reported it as a post on X and described it as a call to step back and assess the trust architecture of AI. According to TechCrunch, it followed reports of AI companies seeming to lose control of their models and a plan from Anthropic chief executive Dario Amodei for more cautious development.
The author runs a major AI vendor and wrote that responsibility cannot be outsourced to model providers. His central line is that organisations need to separate the supply of intelligence from the authority over it. He proposes surrounding non-deterministic models with deterministic system design, human controls and reliable operating procedures and treating frontier models, closed or open-weight, like insider risks. The comparison does not assume malice: any capable actor with access to important systems can make mistakes or be compromised.
Seven Principles, Seven Questions For A Vendor
The essay lists seven principles. Each one converts into a question you can put to an agent vendor and a line you can ask for in the contract.
| Principle | Question for the vendor | What to ask for |
|---|---|---|
| Model diversity | Can one model both do the work and verify it? | No single model as the sole dependency or its own checker |
| Observe everything | Does every meaningful action leave readable, tamper-proof evidence? | Action logs a human can read and the customer can export |
| Verifiability | Is the whole system tested against failures, attacks and edge cases? | Test results beyond successful tasks |
| Independent controls | Who decides what the model can access and do? | Permissions set by the customer, not vendor defaults |
| Independent auditability | Is validation independent of the intelligence being validated? | Evidence that does not rest on the model's own report |
| Containment | Can an authorised person pause or stop the model mid-task? | A documented stop control and who may use it |
| Incident disclosure | Will you tell affected customers what failed and which controls failed? | A notification duty with a deadline |
The containment and observation principles carry the essay's sharpest wording: assume a model is compromised and contain it from the start, like an emergency brake. Make every meaningful action leave tamper-proof, human-readable evidence.
Where The Checklist Points Back At Microsoft
The same vendor is building agent products. According to heise, Windows 11 is set to get an agentic desktop. In mid-September Microsoft presented a draft code of conduct for its MAI models under which they should never resist being interrupted or shut down. Nadella's list is therefore a test Microsoft's own products can be put through, and the answers will show how far its practice has followed the essay.
Microsoft is also not the only vendor in the market, and nothing in the essay is specific to it. The seven questions work the same on any agent product, which is the reason to write them down now, before the next procurement round.
Which Questions You Can Answer Today
Sort the seven questions into two columns: those you can answer for your current agents from documentation and those you cannot. The second column is your list for the vendor. Decide before the meeting which missing answer would stop the purchase. Containment and observation are the easiest to test in a demo, because you can ask for a mid-task stop and a sample log.
The essay does not mention the EU AI Act, so whether a deployment falls under its human oversight duties is a question for your counsel, not something this checklist settles. What it gives you is a shared vocabulary for the conversation, written by the head of an AI vendor.
Read next: Copilot Got a New Name. Your Invoice Got a New Shape. | The Patch That Closed One Hole and Opened Another
