Something peculiar is happening in the artificial-intelligence industry. Three researchers connected with Anthropic have publicly announced that the technology they are helping to develop may destroy humanity. Jacob Coxon, who recently resigned after working at both OpenAI and Anthropic, says that “the people building AI earnestly believe that it could kill us all by the end of the decade”. Evan Hubinger, who leads alignment science at Anthropic, agrees and places his own estimate of human extinction from AI within the next decade at greater than ten per cent. Samuel Marks, another Anthropic researcher, says that concern increases with seniority inside the industry. These are extraordinary claims, and Axios, Reuters, AP and most of the technology press have understandably treated them as such.

Perhaps they are right. Nobody can prove that they are not. But before surrendering control of one of the most important technologies since the personal computer to a regulatory structure designed in Washington and advised by the very corporations presently dominating the industry, we might ask a less exciting question. Who benefits from the panic?

The timing is interesting. The American artificial-intelligence industry has spent the past several years constructing a business model around scarcity. Frontier models require colossal investment, vast clusters of expensive processors, power stations, data centres and armies of highly paid researchers. These costs provide the justification for equally colossal valuations. Anthropic is now spoken of in terms that would have sounded absurd only a few years ago; OpenAI has become one of the most important private corporations on earth. The economic premise is that only a small number of institutions possessing extraordinary concentrations of capital can build genuinely capable artificial intelligence.

China has spent the past two years making that premise increasingly uncomfortable. DeepSeek and other Chinese laboratories have not proved that American frontier models are worthless. Claims that they deliver “99 per cent of Claude for one hundredth of the price”, repeated in some of the more excited commentary, should be treated cautiously. Current independent comparisons still find meaningful capability differences: one recent September comparison had Claude Opus leading DeepSeek V4-Pro on a coding benchmark by more than fifteen percentage points. But the price difference is nevertheless substantial. Depending on the models and pricing tiers compared, DeepSeek can cost roughly four to thirteen times less than Claude for output.

That is not a trivial development. It means the moat around the American laboratories may be shallower than their investors assumed. Worse from their point of view, open-weight models can be downloaded, modified and in some cases run locally. Once the weights are available, the owner of the model no longer controls every inference, cannot meter every token and cannot necessarily withdraw the product from a customer who has offended somebody important.

This changes the political meaning of the AI safety debate. OpenAI is no longer merely asking governments to understand artificial intelligence. On 9 September it called explicitly for “mandatory national AI safety requirements” and said it wanted Congress to establish “mandatory, capability-based national AI safety regulation”. citeturn975328search3 That language may be motivated entirely by sincere concern. Indeed, I suspect much of it is. The mistake is to assume that sincerity eliminates interest.

Industries have always discovered that regulation can simultaneously restrain them and protect them. The large incumbent complains about the cost of compliance, then notices that the same compliance costs are fatal to the fellow trying to enter the market with one-hundredth of his capital. Banks survive banking regulations that prevent new banks from appearing. Pharmaceutical corporations curse regulatory expense while knowing that the expense protects existing patent portfolios and established distribution systems. Defence contractors do not particularly enjoy procurement bureaucracy, but neither do they lose sleep over the difficulty encountered by two engineers in a garage attempting to compete for a Pentagon contract.

Artificial intelligence is unusually vulnerable to this process because almost any restriction can be presented as existential necessity. Licensing becomes necessary because the model might escape. Independent safety evaluation becomes necessary because the model might deceive its creators. Compute thresholds become necessary because sufficiently powerful models might design biological weapons. Restrictions on model weights become imaginable because once a model is distributed nobody can guarantee what somebody in Shanghai, Manchester or Nebraska will do with it. Every ordinary protectionist mechanism can therefore be recast as an emergency measure to prevent human extinction. This is what makes the current panic politically dangerous.

The danger does not require Jacob Coxon to be lying. I see no particular reason to think he is. Axios reports that he left Anthropic shortly before his equity vested, hardly the obvious behaviour of somebody trying to inflate his personal stake in the company. Hubinger may likewise believe every word he says. Moral panics are rarely manufactured from nothing by a room full of conspirators who agree on a script. They usually begin with a real concern, attract sincere believers, acquire institutional patrons and eventually become useful to people whose motives have very little to do with whatever frightened everyone in the first place.

Climate policy, terrorism legislation, financial regulation and public-health emergency powers have all demonstrated versions of the same mechanism. The original danger can be real while the political machinery built around it develops purposes of its own. AI may be dangerous. That does not make the proposed regulators harmless.

There is already evidence that money is flowing into the production of public arguments about AI risk. Physicist Sabine Hossenfelder recently published a video entitled “I Was Offered Money to Tell You AI Will Kill Us”. Her actual argument is more balanced than the title used by some commentators: she says there is money available both for apocalyptic claims and for optimistic claims about AI, and that creators are being paid to advance competing narratives. That is less dramatic than proving a centrally directed propaganda operation. It is also more believable. A new ecosystem of foundations, safety institutes, advocacy organisations, consultants and academic programmes now has an obvious professional interest in establishing that AI safety is one of the defining political questions of the age.

This is how permanent establishments are created. First comes a risk. Then an expert class forms around the risk. The experts naturally conclude that more expertise is required. Governments discover a new field requiring supervision. Corporations appoint regulatory departments. Universities establish centres. Think tanks publish papers. Journalists develop specialist beats. Within a few years, thousands of intelligent and perfectly respectable people derive their salaries, status or institutional importance from ensuring that the problem requiring their expertise does not disappear. There need be no conspiracy because the incentives do the organising.

The economic competition from China makes all of this more serious. The United States has become accustomed to regulating industries in a world where regulation mainly affected American and European competitors operating under broadly similar legal systems. AI is different. Washington can impose an expensive certification regime on OpenAI. It cannot impose one on every Chinese laboratory whose model weights can be downloaded across the Internet. The temptation will therefore be not merely to regulate American development but to regulate access.

The argument will write itself. Chinese models have not passed our safety tests. Their training data cannot be inspected. Their alignment procedures do not satisfy American standards. Their weights may contain hidden vulnerabilities. Their distribution creates national-security risks. Therefore American businesses must not use them, cloud providers must not host them, chip manufacturers must prevent certain models from running, and platforms must stop distributing their weights. At that point a safety regime becomes a trade barrier.

This possibility is not imaginary, though one should be precise about who is advocating what. Anthropic has publicly rejected the accusation that it wants an outright prohibition on open-weight models. Dario Amodei wrote in July: “Anthropic has never advocated for a ban on open-weights models.” He went further, describing non-dangerous open-weight systems as “a public good”. That statement should be taken seriously.

But notice the qualification. Open weights are acceptable provided they do not possess dangerous capabilities. Who determines when that threshold has been crossed? Once government establishes the principle that some models are too capable to circulate freely, the entire political battle shifts to defining “dangerous”. The laboratories with the largest safety departments, closest relationships with government and greatest ability to comply with evaluation procedures will naturally exercise enormous influence over the definition. A Chinese model that suddenly matches an American product at one fifth of the price will not merely be a competitor. It can become a national-security problem. Perhaps sometimes it genuinely will be. That is why the problem is difficult.

The correct libertarian response cannot simply be that every warning about AI is fraudulent. Artificial intelligence plainly creates real risks. Anthropic itself documents cases in which sophisticated actors attempt to use frontier models for cyber operations and to extract capabilities from American systems. More capable models may make certain forms of computer intrusion, surveillance or weapons development easier. It would be foolish to deny this because Sam Altman has commercial interests. But we should apply the same scepticism to proposed solutions that we apply to commercial claims.

The American frontier laboratories have a fortunate combination of arguments available to them. When raising capital, they possess technologies of almost unimaginable productive value that will transform the world. When seeking regulatory protection, they possess technologies of almost unimaginable destructive power that must be confined to institutions capable of managing them responsibly. The same enormous capital requirements that once proved their technological superiority can become evidence of their suitability for regulation.

Cheap models present a threat to this structure. Local models present a still greater one. A program running on hardware somebody owns cannot easily be rationed by API access, monitored centrally, politically deplatformed or priced at whatever rate preserves a trillion-dollar valuation. This is why the distinction between AI safety and AI control must remain absolutely clear.

A civilisation in which millions of people possess intelligent software on their own machines will certainly create problems. It will also distribute an extraordinary amount of intellectual power. A civilisation in which four corporations possess that intelligence and everybody else rents access to it under rules negotiated with the federal government creates a different kind of problem. I know which arrangement frightens me more.

The Chinese competition makes the choice urgent because it is destroying the comfortable assumption that advanced AI must remain centralized. DeepSeek may not equal the best American models in every category. It does not need to. If a business can obtain eighty or ninety per cent of the useful capability for a fraction of the price, the economics change dramatically. If increasingly powerful open models can be run locally, the economics may change again. That is the moment when incumbents traditionally discover principles.

The British rail companies discovered the importance of safety when road competition appeared. Established broadcasters discovered the importance of broadcasting standards when independent media threatened their audiences. Large financial institutions discover systemic risk whenever somebody proposes allowing competitors to operate without bearing the same regulatory overhead. None of these concerns is necessarily false. Their convenience is what should attract attention. And the AI extinction story is convenient.

If ordinary technological competition were the issue, OpenAI and Anthropic would have to beat DeepSeek on capability, efficiency and price. If the issue becomes preventing the extinction of mankind, comparison shopping suddenly looks irresponsible. Cheap Chinese models cease to be bargains and become uncontrolled strategic weapons. Open weights cease to be software and become proliferation events. The inability to regulate them becomes not an argument against regulation but evidence that stronger international controls are required.

The debate has therefore acquired all the ingredients of a classical moral panic: an incomprehensible new technology, predictions of unprecedented disaster, insiders confessing that the danger is worse than the public understands, demands that something must be done immediately, and a ready-made class of experts explaining what the something should be. The only unusual feature is that the organisations most likely to benefit from restrictions are among the organisations whose warnings provide the panic with authority.

Again, this proves no conspiracy. It proves that we should be suspicious. Hubinger says there is more than a ten per cent chance AI kills everyone during the next decade. Fine. Let him explain how he derived the number. Let others challenge his assumptions. Let us examine the mechanisms by which a language model becomes an extinction event. If the evidence is compelling, I am willing to reconsider my scepticism. What I am not willing to do is move directly from an unverifiable probability estimate to a regulatory architecture that leaves the world’s most powerful AI corporations safely installed behind government barriers to entry.

The great danger of moral panics is not usually that the feared object is wholly imaginary. It is that fear suspends the normal examination of interests, evidence and consequences. Everyone becomes so desperate to be seen taking the danger seriously that asking who benefits from the proposed solution becomes indecent. We should ask anyway. If artificial intelligence really is about to kill us all, a licensing regime negotiated with Anthropic will not save us.

If it is not, such a regime may still succeed brilliantly at saving Anthropic from DeepSeek. That possibility deserves rather more attention than it is receiving.

Sources

Axios, “Anthropic insiders warn AI could kill all humans”, 9 September 2026.

Axios, “Anthropic whistleblower gave up his equity to leave the company”, 9 September 2026.

Associated Press, “Anthropic researcher resigns with warning about the dangers of AI development”, September 2026.

Reuters, “Ex-Google DeepMind researcher adds to warnings that AI could ‘kill all humans’”, 15 September 2026.

OpenAI, “The AI policy window is open. We need to act”, 9 September 2026.

Anthropic, Dario Amodei, “Our position on open-weights models”, 27 July 2026.

Anthropic, “Countering misuse of AI: September 2026”.

LLM Waves Research, “DeepSeek vs Claude: the real price gap is 4x to 13x”, 9 September 2026.

Sabine Hossenfelder, “I Was Offered Money to Tell You AI Will Kill Us”, 5 September 2026.

Mike Adams, “The Plot to Criminalize Open Source AI and Hand Tech Giants a Government-Protected AI Cartel”, Natural News, 14 September 2026. The article is the immediate stimulus for the argument above, though several of its factual and causal claims should be treated as allegations rather than established facts.