Anthropic has drafted a bill that would let a judge halt an AI model if more than 50 people could die

The draft uses paid evaluators and a 72-hour notice, and Anthropic is the only named backer.

Abstract EMRGNG cover image for a story about Anthropic

50 people, $1 billion, 72 hours. Those are the three numbers in a draft AI safety bill that Anthropic has circulated, TechCrunch reported on 20 October, following Politico’s earlier report on 16 October.

The bill would let the government block a model’s release where there is a foreseeable, imminent risk of more than 50 deaths or more than $1 billion in property damage. The risk would have to come from a biological weapon, a model evading its developer’s controls, or a cyberattack.

Independent evaluators registered with the Commerce Department would assess models, and labs could pay them, provided payment does not depend on the result. If they find an imminent catastrophic risk, they have 72 hours to notify Commerce, which must then ask a Washington, D.C., judge to halt the release. Agencies would be barred from stopping distribution themselves.

The thresholds resemble those in California’s SB 53, which is already law. An Anthropic spokesperson confirmed the draft and said it reflects the company’s Advanced AI Framework, which it has pushed to lawmakers since June.

TechCrunch names no opponents of this draft. It notes that labs generally resist mandatory oversight, and that Nvidia’s Jensen Huang has argued the industry needs no new AI laws, though not in response to this bill.

This is a draft being shared, not legislation, and whether Congress will take it up is unknown. It is also unclear how paid evaluators would stay independent, and TechCrunch calls the listed risks a little far afield from today’s incidents, such as agentic hacking.

Read more here.

More from EMRGNG