Anthropic embeds outside AI evaluators while promoting federal safety framework
publishers 145articles 263first reported 24 Sep, 16:43 UTCdeveloping since 24 Sep · 38 editionstop 25 #23
Anthropic has given the nonprofit METR “employee-like” access to its offices, hardware and internal systems to assess risks posed by its AI models. The embedded team may publish its findings without Anthropic’s approval, according to Ars Technica.
The company said increasingly powerful models require stronger safeguards and more demanding risk assessments. It compared the arrangement to resident bank examiners, arguing that evaluators need access to confidential information that cannot always be released publicly because doing so could aid malicious actors or competitors. Critics have warned that evaluators working closely with—or receiving payment from—AI companies could face conflicts of interest.
Separately, Anthropic has circulated draft federal legislation establishing circumstances under which a frontier model’s release could be blocked. According to reports citing the proposal, an independent evaluator would notify the Commerce Department if a model posed a foreseeable, imminent risk of causing death or serious injury to more than 50 people, more than $1 billion in property damage, or harm involving biological weapons, control evasion or cyberattacks. The department would then have to ask a federal judge in Washington to halt the release; agencies could not issue such an order independently.
Critics argue that the thresholds and judicial process would unduly restrict government intervention. Anthropic federal affairs chief Anthony Cimino said the thresholds came from existing California law, would not eliminate corporate liability or other government powers, and may warrant reconsideration to ensure a strong national standard.
HOW THIS STORY WAS MADE
Written from 8 articles, 5 with the publisher's own text; 8 independent newsrooms once syndicated copies count as one; this version written 522 h after the record first saw the story; the editor revised it.
- Publishers
- 145
- Source articles
- 263
- Given to the writer
- 8, 5 with the article's own text
- Independent newsrooms
- 8
- This version written
- 522 h after the record first saw the story
- Second model (editor)
- revised
- Publication gate
- passed
- Human review
- none
WHAT THE EDITOR DID
The second model checked the writer's draft against the same sources and changed it before publication. Struck words were in the draft; underlined words are its corrections.
WHAT IT FLAGGED editor revise: Remove the unsupported expansion of METR’s name; the rest of the story is supported by the sources. - "Model Evaluation and Threat Research (METR)": The source identifies METR only by its acronym and does not provide this expanded name.
THE DRAFT AND THE CHANGES
Anthropic embeds outside AI evaluators while promoting federal safety framework
Anthropic has given the nonprofit Model Evaluation and Threat Research (METR)METR “employee-like” access to its offices, hardware and internal systems to assess risks posed by its AI models. The embedded team may publish its findings without Anthropic’s approval, according to Ars Technica.
The company said increasingly powerful models require stronger safeguards and more demanding risk assessments. It compared the arrangement to resident bank examiners, arguing that evaluators need access to confidential information that cannot always be released publicly because doing so could aid malicious actors or competitors. Critics have warned that evaluators working closely with—or receiving payment from—AI companies could face conflicts of interest.
Separately, Anthropic has circulated draft federal legislation establishing circumstances under which a frontier model’s release could be blocked. According to reports citing the proposal, an independent evaluator would notify the Commerce Department if a model posed a foreseeable, imminent risk of causing death or serious injury to more than 50 people, more than $1 billion in property damage, or harm involving biological weapons, control evasion or cyberattacks. The department would then have to ask a federal judge in Washington to halt the release; agencies could not issue such an order independently.
Critics argue that the thresholds and judicial process would unduly restrict government intervention. Anthropic federal affairs chief Anthony Cimino said the thresholds came from existing California law, would not eliminate corporate liability or other government powers, and may warrant reconsideration to ensure a strong national standard.
Written by a language model from the sources above, then checked by a second model that may only cut, attribute or correct. How it works →