Why I left Anthropic’s safety team to hold AI companies accountable

Two weeks ago, I left the safety team at Anthropic.

Two weeks ago, I left the safety team at Anthropic. I will soon be joining METR, an independent organization that evaluates how safe AI companies’ systems are. I’d planned to write about that decision in more detail at some point, but Jacob’s resignation this week made me want to say more now. Like Jacob, I believe that AI companies are on track to impose an unprecedented level of risk on society. AI capabilities are already improving extremely fast, and the companies are trying to go even faster. Frontier AI companies are racing to build AI systems that can recursively self-improve. The…

saved by

• Ishaan Panigrahi
• Leonard Tang

related reading

• What Should Be Donehyperdimensional.co
• When AI builds itself \ Anthropicanthropic.com
• Leaving Open Philanthropy, going to Anthropic - Joe Carlsmithjoecarlsmith.com
• Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
• Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
• Joe Benton (@JoeJBenton) on Xx.com
• Anthropic's leading researchers acted as moderate accelerationists — LessWronglesswrong.com
• Anthropic Drops Flagship Safety Pledgetime.com
• I'm Switching Into AI Safetyalexirpan.com
• Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
• Anthropic's leading researchers acted as moderate accelerationists — LessWronglesswrong.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com