Anthropic gives METR evaluators employee-like access and publishing rights
Anthropic says it has started giving employee-like access to a team from METR, a nonprofit that evaluates frontier AI models. Six METR staff have company hardware, office space and access to internal tools roughly on par with Anthropic's own risk assessment teams, outside a few categories such as customer data and model architecture details. METR will publish its analysis of Anthropic's risk levels, incidents or practices; Anthropic cannot veto publication but can require narrow redactions, and METR can say publicly whether it thinks those were justified. Anthropic is also working with SecureBio and Accenture as evaluators under more limited access, is in early talks with others including Hugging Face, and describes three tiers of evaluator access.
Sorted take: this is a promise of outside scrutiny, not a result yet; the useful signal will be what METR actually publishes.
Why it matters: If you pick AI vendors for regulated or sensitive work, independent evaluator reports can become evidence for your own risk reviews. Watch for METR's first published assessment and ask other vendors what outside review they allow.
Anthropic ↗AI-assisted, reviewed by Sorted AI
▲ 0
0 comments