Microsoft’s chief executive is calling for an emergency brake on AI. He also wants users to begin with the premise that the model you're using has already gone wrong.
Satya Nadella set out this view in a lengthy post on X covering the risks of highly advanced AI models and how they should be handled. At the heart of his argument: AI can no longer be treated as a “set of nested black boxes” whose advice and actions we simply take or leave.
The brake pedal is the headline
Anyone who has tracked the industry’s safety conversation will recognize most of the points on Nadella’s list. He calls for fast disclosure of incidents, independent audits, verifiable data and containment of the models. It’s on containment that he pushes further than some of his peers.
“We must assume a model is compromised and contain it from the start. Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task. More advanced models will require more advanced containment technologies that we need to standardize on,” Nadella said.
Hearing this openly from the leader of one of AI’s largest backers stands out. Beginning from “compromised” treats failure as the baseline, which runs counter to the way these products are typically sold.
What he wants instead of black boxes
Nadella is pushing for a more transparent arrangement in which models can be both contained and observed, and in which they produce “tamper-proof human readable evidence.” In plain terms, someone should be able to review afterward what a model did and the reasoning behind it, rather than relying on the model’s own account.
The word choice undercuts the message
There is one problem. Throughout the post, Nadella refers to AI as “super intelligence.”
That term clashes with an appeal for transparency and restraint. When the goal is to get people to view these systems as fallible tools that need a kill switch, overstating what they are undermines the case. Embrace the emergency brake and drop the branding.