Anthropic’s Proposed AI Safety Bill Only Kicks In at 50 Deaths or $1 Billion in Damages

Anthropic’s draft bill sets a 50-death or $1 billion damage threshold before Commerce can seek a court-ordered AI shutdown

Rex Edison Avatar
Rex Edison Avatar

By

Image: Deposit Photos

Key Takeaways

Key Takeaways

  • Anthropic’s draft bill lets only the Commerce Department seek federal AI model shutdowns.
  • Federal intervention requires over 50 deaths or $1 billion in damages to trigger action.
  • Proposed federal preemption would restrict state AI oversight on disclosures and incident reporting.

Fifty deaths. One billion dollars in damages. Those are the numbers Anthropic thinks should trigger federal intervention on a dangerous frontier AI model.

The company has privately circulated a draft federal AI safety bill to U.S. lawmakers, according to Politico’s October 16, 2026 reporting. Its most consequential design choice is structural: all enforcement power would route through a single agency, the Commerce Department, while reported language would block every other federal body from independently suspending a model.

What It Takes to Trigger a Shutdown

The proposal sets a precise catastrophic-harm threshold before the government can act on a frontier AI model.

The proposal would allow intervention only when a frontier AI model presents a “foreseeable and imminent risk” of causing more than $1 billion in damages or killing or seriously injuring more than 50 people, according to Politico.

Three specific scenarios would qualify: the creation or release of a biological weapon, an AI-enabled cyberattack, or a model evading its developer’s controls.

That enumerated list is narrow by design. Any harmful AI incident falling below those thresholds, or outside those three categories, would have no clear suspension pathway under this proposal.

One Agency, One Gatekeeper

Commerce would be the only federal agency authorized to seek a model shutdown, subject to a federal judge’s approval.

Under the proposed enforcement architecture, independent third-party evaluators would register with Commerce and periodically assess frontier AI companies and their models, according to Politico. AI laboratories could pay those evaluators directly, provided the payments did not affect evaluator independence. Whether that condition is enforceable in practice is an open structural question, not a finding stated in the proposal.

If an evaluator identified an imminent enumerated risk, it would have 72 hours from completing its assessment to notify Commerce, according to Politico. Commerce would then bring the evidence to a federal judge in the U.S. District Court for the District of Columbia and seek a court order halting the model’s release or use.

The reported language would prohibit other federal departments, agencies, or officers from independently suspending an AI model’s development, deployment, or internal use. Centralizing that authority creates predictability, but it also concentrates risk. If Commerce declines to act or cannot assess a threat quickly enough, no other federal body could seek a suspension under this bill, though the available reporting does not establish that every other form of federal action would be foreclosed. That is an analytical implication of the enforcement design, not a stated finding in the proposal.

Anthropic’s Position

The company frames the draft as an extension of its Advanced AI Framework, which it has promoted to lawmakers since June.

An Anthropic spokesperson said the draft reflects the company’s Advanced AI Framework, which it has promoted to federal and state lawmakers since June. The company described the measures as sensible steps Congress should enact.

The proposal would also require frontier laboratories to establish safety frameworks, publish transparency reports for new models, protect whistleblowers, and report critical safety incidents to Commerce, according to Politico.

A Note on State Authority

Federal preemption provisions in the draft could limit state-level AI oversight in areas already covered by the bill.

The proposal would reportedly preempt some state AI laws covering public risk disclosures, third-party evaluations, and critical-incident reporting, according to Politico. That is a meaningful constraint, given that the preemption would apply in areas where states are actively pursuing their own frameworks.

Until Congress takes it up and passes it, the proposal carries no legal weight. What it does carry is a clear signal about how one major AI company thinks federal oversight should be structured: catastrophe-first thresholds, a single enforcement gatekeeper, and a judicial review requirement that adds accountability alongside the time that judicial process requires. Whether that framework is tight enough is exactly the debate you should expect on Capitol Hill.

Share this

At Gadget Review, our guides, reviews, and news are driven by thorough human expertise and use our Trust Rating system and the True Score. AI assists in refining our editorial process, ensuring that every article is engaging, clear and succinct. See how we write our content here →