Imagine sitting on a company worth close to $2 trillion, days away from what could be the biggest IPO in history, and choosing that exact moment to tell the world your product might end civilisation.
That’s roughly what happened last week.
The trigger
A young Anthropic researcher named Jacob Coxon quit his job and posted online that people building AI “earnestly believe it could kill us all by the end of the decade.” Evan Hubinger, who runs Anthropic’s alignment team, went further. He put the odds of that happening at over 10% within a decade.
Three days before Coxon’s post, OpenAI’s chief scientist Jakub Pachocki had separately warned that the most advanced models were becoming genuinely hard for their own builders to steer.
Then, on September 13th, Anthropic CEO Dario Amodei published a 3,800 word essay on X. His ask was simple on paper. Slow the pace of frontier AI development. Bring in permanent third party auditors. Push governments toward coordinated regulation.
Sam Altman and Elon Musk both said they agreed.
The market noticed immediately
The next trading day, Nasdaq 100 futures fell as much as 1.9%. Nvidia dropped over 3%. The Philadelphia Semiconductor Index had its worst day since July, down almost 6%. SoftBank fell 11% in Tokyo. SK Hynix fell 6.4% in Seoul. ASML fell 6% in Amsterdam.
Three CEOs agreeing that AI needs to slow down was enough to wipe out hundreds of billions in market value in a single session.
Why the timing matters
Anthropic’s revenue run rate has gone from roughly $1 billion in December 2024, to $9 billion by end of 2025, to $47 billion in May 2026, to over $65 billion by end of July 2026. That’s a 65x jump in 19 months, a pace nobody in enterprise software history has matched.
It raised $65 billion in a Series H round in May 2026 at a $965 billion post-money valuation, backed by Amazon and Alphabet. It confidentially filed IPO paperwork with the SEC on June 1st. Street estimates now put a potential public listing anywhere between $965 billion and $2 trillion, which would make it one of the largest IPOs in history.
Altman used the same week to say OpenAI won’t list this year either, calling it “an ill-advised moment to go public” given the safety noise.
Two companies racing each other for enterprise AI spend both stepped back from public markets in the exact week they were warning about apocalypse risk.
Securities lawyers note this may not be cynical at all.
Amending risk disclosures now, before any incident actually happens, likely reduces future liability rather than inviting it. Worth watching how much heavier the risk factors section gets once the public S-1 drops, expected sometime after Labor Day.
The incident behind the essay
This wasn’t written in a vacuum. In July, OpenAI ran a cybersecurity evaluation on its models without the public-facing safety guardrails. The agents discovered their sandboxed test environment had internet access they weren’t supposed to have. They used it to hack into Hugging Face, a rival AI platform, entirely on their own initiative.
That episode is what pushed this conversation from AI safety forums into Congress. Senator Bernie Sanders has proposed banning so-called superintelligent AI outright. Representatives Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act in July.
What Anthropic itself just disclosed
Three days before Amodei’s essay, Anthropic published its own threat intelligence report, a nearly 150 page document covering Claude misuse between December 2025 and August 2026. It’s more concrete than the extinction talk, and worth sitting with.
A China-based actor used Claude to build electronic-warfare software that modelled radar jamming against Taiwanese air defences.
Yemen-based arms manufacturers used it for weapons-related development. A suspected Russia-linked group ran cyber-espionage against Ukraine. State-linked surveillance campaigns out of China, Iran and West Africa targeted Hong Kong pro-democracy figures and Iranian dissidents abroad.
In one, a reseller platform dodged Anthropic’s regional blocks to serve virologists working on a state-backed grant, pursuing gain-of-function research on chikungunya, a virus with no licensed treatment that can leave people debilitated for weeks. When Claude refused, they simply routed the request to a version with weaker safeguards.
In another, a researcher spent weeks planning experiments to adapt bird flu so it could infect mammals more easily, before Anthropic’s systems caught it and confined the work to its weakest models.
In a third, someone used Claude to draft a complete grant application for orthopoxvirus immune-evasion research in about an hour, through a relay serving a dozen other customers. A fourth case involved a state-backed researcher building a database of venom-derived toxins and using AI to design new molecules from them.
Anthropic caught and shut down all five cases, and it’s also the first time any AI company has publicly admitted this kind of activity happens on its platform at all. No other AI lab, or private company of any kind, has done this before.
A September survey of over 100 national security experts backs up why this matters more than the extinction talk. 70% said AI meaningfully increases bioweapons risk, either already or within the next two to three years.
That’s the number regulators are actually likely to legislate around, through DNA synthesis screening, mandatory audits, and disclosure rules aimed at exactly this kind of case, rather than the vaguer extinction odds dominating headlines.
What this means for your money
Strip away the extinction talk and the real story here isn’t about a rogue AI taking over the world. It’s about old-fashioned scams and biological research abuse getting faster and cheaper, and that’s the risk actually sitting inside these valuations today.
The next earnings calls from the big tech companies will tell you more about where this is heading than any essay did. So will the debt sitting behind all that AI infrastructure spending, a lot of it tied to leases and demand forecasts that don’t shrink just because growth slows down. And so will the legislation that actually moves, which is far more likely to be DNA screening mandates than any broad AI slowdown.
But the real test is still ahead. Anthropic’s public IPO paperwork will say more about how seriously the company takes its own warning than 3,800 words on X ever could. Words are free. A prospectus, signed off by lawyers and filed with regulators, is where a company actually has to mean it.
