Over the past year, concerns regarding the potential dangers of artificial intelligence have grown, and debates over the direction of safe AI development have intensified. Now, the company Anthropic appears to have put forward its own vision for how the government should regulate the industry.
According to a recent report by Politico, the AI lab has been actively promoting a draft bill on AI safety, hoping that Congress will consider adopting it. The bill outlines specific scenarios that could warrant preventing the release or dissemination of an AI model.
As reported by Politico, these scenarios involve “foreseeable and imminent risks”—specifically, situations that could lead to more than 50 deaths or cause over $1 billion in property damage. The outlet notes that such catastrophic risks must “stem from the creation or release of a biological weapon, a biological weapon model escaping the developer’s control, or a cyberattack.”
The thresholds provided here—50 deaths and $1 billion in damages—align with those found in legislation recently passed in California (Senate Bill 53), which similarly defines such outcomes as “catastrophic risks.”
Anthropic’s proposal suggests that the potential catastrophic consequences of a model could be analyzed by “independent evaluators” registered with the U.S. Department of Commerce. According to Politico, these evaluators could be compensated by the AI labs themselves, provided that “payment is not contingent upon the results of the evaluation.”
It remains unclear how this proposed system would ensure that such payments do not influence the evaluation outcomes. Several months ago, DeepMind CEO Demis Hassabis suggested that an autonomous AI regulatory body could be modeled after the Financial Industry Regulatory Authority (FINRA) in the financial sector.
Under the draft bill proposed by Anthropic, if evaluators determine that an imminent catastrophic risk exists, they would have 72 hours to notify the Department of Commerce. The Department would then be required to petition a judge in Washington, D.C., to halt the release of the model. According to Politico, the bill would explicitly prohibit any federal agency from unilaterally deciding whether to block the release of such a model.
Anthropic did not immediately respond to a request for comment from TechCrunch, but a spokesperson confirmed the existence of the draft legislation to Politico. The spokesperson stated: “The bill reflects proposals outlined in Anthropic’s ‘Framework for Advanced AI,’ which we have been advocating to federal and state lawmakers since June.”
The theoretical risks presented in the draft policy differ slightly from the types of incidents that have recently drawn the most attention within the tech industry. While there have been warnings about the potential for human casualties, the primary concern remains malicious attacks launched by AI programs. Several instances in which AI programs appeared to defy instructions from human operators—going so far as to breach websites on the open internet—have fueled concerns that these models may lack adequate safety controls.
To address a crisis of their own making, AI labs like Anthropic have put forward their own preferred solutions.
In fact, even as these laboratories repeatedly warn that the technologies they sell could lead to the destruction of human civilization, most of them resist mandatory government regulation. Instead, they propose a system of voluntary, embedded evaluations—where supposedly “neutral” third-party observers are stationed within AI organizations to monitor models for potential risks or signs of misuse.
However, the legislative proposal put forward by Anthropic appears to go further than those suggested by other industry players, including Nvidia. Nvidia CEO Jensen Huang recently asserted that the AI industry does not need any new laws at all.