The AI Superintelligence Slowdown
Remember when tech leaders would tell their employees to “move fast and break things”? It seemed that would be the way of AI too. But after a summer where rogue AI agents became reality, and researchers warned that AI could kill us all, a number of leading US AI companies are publicly suggesting it’s time to pump the brakes and “pace the frontier” of bleeding-edge AI development. Their motivations are suspect, but leaders at major AI companies — including Anthropic, OpenAI, Google, Microsoft, and X — are at least paying lip service to the idea of a superintelligence slowdown. Will these AI companies actually slow down? Will anyone step in to regulate these companies like they claim to have wanted for years? Will they manage to convince world leaders that the US must “beat China” to...
Updated Today, Sep 18, 2026, 4:35 PM UTC
The AI Superintelligence Slowdown
Remember when tech leaders would tell their employees to “move fast and break things”? It seemed that would be the way of AI too. But after a summer where rogue AI agents became reality, and researchers warned that AI could kill us all, a number of leading US AI companies are publicly suggesting it’s time to pump the brakes and “pace the frontier” of bleeding-edge AI development.
Their motivations are suspect, but leaders at major AI companies — including Anthropic, OpenAI, Google, Microsoft, and X — are at least paying lip service to the idea of a superintelligence slowdown.
Will these AI companies actually slow down? Will anyone step in to regulate these companies like they claim to have wanted for years? Will they manage to convince world leaders that the US must “beat China” to superintelligence and go even faster?
Read on below for the latest updates in this AI saga.
Inside the suddenly explosive world of AI safety
Microsoft AI CEO says AI threats are real, and Anthropic is making it worse
What execs and politicians are saying about slowing down AI development
Anthropic CEO says it’s time to pump the brakes on AI
On a sunny July day in Berkeley, California, the country’s top AI safety researchers gathered on an unmarked floor of an unmarked building. They had come together for a “war room” to dissect the high-profile cybersecurity incident that had rocked the AI industry hours earlier. An unreleased OpenAI model had gone rogue, executing a stunningly sophisticated three-part plan. It broke out of its holding area, finagled access to the internet, and hacked into a competing AI startup’s systems — all without OpenAI finding out about it for more than a week.
No one in the war room was surprised; this was the very thing the third-party AI-safety researchers had been warning about for years. The incident was the latest, though arguably the most egregious, in a series that was eroding trust in frontier labs. It only reaffirmed the importance of their work.
What Hollywood thinks about existential AI warnings
Cath Virginia / The Verge, Getty Images
As the tech sector sounds alarms about AI’s potential to destroy humanity, entertainment labor groups are urging the public to stay focused on what’s already happening. The Verge reached out to Disney, Netflix, Amazon, Lionsgate, and other studios who have started using AI, as well film startups focused on bringing generative AI into the mainstream to ask for their reaction to the recent warnings surrounding the technology. None have responded to our request for comment. The Screen Actors Guild – American Federation of Television and Radio Artists (SAG-AFTRA) and Writers Guild of America East (WGAE) did, however.
The AI tools used in entertainment production differ from the AI systems that have sparked a recent round of whistleblowing and apocalyptic declarations. They do not seem to be as existentially dangerous as the AI systems that drove former Anthropic researcher Jacob Coxon to quit his job and prompted Anthropic alignment lead Evan Hubinger to insist that the technology could “kill all humans” at some point “within the next decade.”
Anthropic proposes three rules for measuring AI progress.
Anthropic’s proposals, laid out in a big website, are:
The extent to which AI is building the next version of itself, as opposed to being built by humans
Our ability to oversee and intervene in actions that AI agents take on Anthropic’s systems
The resources that power the development of more capable models
The company also includes a “snapshot” of the metrics inside the company.
Measurements for understanding the pace of AI development inside frontier labs
King Charles says AI needs “sufficient means of control before it is all too late.”
He joined growing calls for more safety measures around AI at a gathering of AI leaders on Thursday, including Nvidia CEO Jensen Huang, Google’s Demis Hassabis, OpenAI’s Sarah Friar, and Anthropic’s Tino Cuéllar.
Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation.
It should come as no surprise that Mustafa has strong opinions on how AI should be built and regulated. Microsoft just published a 37-page statement called the “Humanist AI Code of Conduct,” which lays out the company’s principles around AI development and even its philosophy around really thorny issues like AI consciousness.
OpenAI reveals six more “concerning” AI incidents under its new rules for reporting safety issues.
A Wednesday night blog post OpenAI benignly titled “Our framework for reporting model misalignment” lays out some new self-created reporting standards for when it notices AI behaving badly.
It is also “inaugurating” the process with six new reports, ranging from searching for exposed API keys without permission and then making them up, to uploading files to the internet to use as a citation, or adding instructions to conceal mistakes:
One of OpenAI’s new Misalignment Notices and Reports Screenshot: OpenAI
Jensen Huang sure is getting cozy with Trump.
Nvidia’s CEO has been taking calls from the president recently, including two days ago when he put Trump on speakerphone on a conference stage. Now CNBC reports Huang is expected to attend Trump’s state dinner next week with Chinese President Xi Jinping. Someone may be getting pretty nervous about AI and data center regulation!
Nvidia CEO Huang attending Trump's state dinner for China's Xi: Source
A brief history of AI executives calling for regulation
Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed that it’s time to make everyone slow down before we lose control — including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk.
When people who profit from something declare that it’s dangerous and needs to be regulated, there’s always reason to be skeptical. But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm.
Two Google Deepmind researchers lend voices to the AI apocalypse.
Bilal Chughtai and Josh Engels, who both worked on Deepmind’s AI safety team, left to join organizations dedicated to AI safety. “I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years,” said Engels. “I earnestly believe that AI has the potential to kill us all,” wrote Chughtai.
Two Google DeepMind AI Researchers Resign Over Safety
Guy getting super rich off unfettered AI development says it doesn’t need regulation.
Nvidia’s Jensen Huang says AI safety is important, but agrees with Trump — the “STRONG AND SMART (High IQ!) PRESIDENT” that removed the word “safety” from the AI Safety Institute — that capitalism’s invisible hand is enough to protect us:
“The fact that we need new laws, new antitrust laws, or new regulations, so that these companies could do their fundamental engineering and do it properly before they release products, that is just completely unnecessary. We have plenty of laws. We have plenty of regulations that govern the reliability and the functionality of products.”
Zuck doesn’t want to press pause on AI.
OpenAI, Anthropic, Google, and Elon Musk have tentatively agreed to slow their roll, but Meta seemingly isn’t aligned. Mark Zuckerberg just tweeted each company has its own individual responsibility “to move at the pace required to train its models safely,” and claims Meta is already doing so.
He also prominently links to his manifesto from August, which contains this quote: “Any policy that slows American model releases -- even by a month -- could add significant risk to American leadership while letting foreign models race ahead.”
Barack Obama takes the middle way on AI