Satya Nadella said on Saturday that advanced AI should ship with containment, independent controls and an emergency brake that humans can pull.
What they're saying: "We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient," Nadella wrote.
- "Treating frontier closed and open weight models like insider risks is a way to build such a system," he said.
Zoom in: Nadella laid out principles of observability: model diversity, a human-readable footprint of a model's actions, continuous system testing, independent controls and auditability, containment, and incident disclosure.
The big picture: He joins Bill Gates, Anthropic's Dario Amodei, OpenAI's Sam Altman and Elon Musk in warning that safety practice is lagging capability. A researcher who quit Anthropic last month accused that company and OpenAI of gambling with our lives.
- An Anthropic alignment lead put the odds that the technology could kill all humans within the next decade at greater than 10%.
The other side: President Donald Trump has repeatedly dismissed AI extinction risk and stressed staying ahead of China, and this month created an AI Force led by Director of National Intelligence Jay Clayton.
Why it matters: Microsoft runs much of the infrastructure frontier models train and serve on, so a design standard it pushes tends to become the default for enterprise buyers. Nadella inverts the usual pitch: "It will be the one that enables us to trust the model the least."
