Why I'm Not a Security Doomer - by Miles Brundage

Preventing theft of AI systems is hard. But sometimes it's worth it, and protecting model weights specifically is a good target.

Why I'm Not a Security Doomer Preventing theft of AI systems is hard. But sometimes it's worth it, and protecting model weights specifically is a good target. Miles Brundage Mar 11, 2025 27 6 Share I have recently – in a somewhat joke-y way – started using the term “security doomer.” By security doomer, I mean people who think that AI systems can’t be protected from theft by sophisticated state attackers, or at least that protecting them would slow down AI research so much that it isn’t worth the effort. This is a reasonable perspective, since securing anything against sophisticated attackers

related reading

• The End-State Fallacy: Where Is AI Security Headed?endstatefallacy.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Cybersecurity Looks Like Proof of Work Nowdbreunig.com
• AI Safety Seems Hard to Measurecold-takes.com
• Vulnerability Research Is Cooked - Quarrelsomesockpuppet.org
• Ten AI safety projects I'd like people to work onthirdthing.ai
• A basic systems architecture for AI agents that do autonomous research — LessWronglesswrong.com
• Fields that I reference when thinking about AI takeover prevention — LessWronglesswrong.com
• Your Solution Doesn't Know Your Problem Existsblog.atlascomputing.org
• Using Dangerous AI, But Safely?youtube.com
• The case for ensuring that powerful AIs are controlledblog.redwoodresearch.org
• Simulated Users & Sad LLMs1a3orn.com