The Myth of unsafe Open Source AI
florianbrand.com · 1,146 words · saved by 1 readers
Thoughts about open and closed AI
The Myth of unsafe Open Source AI The Myth of unsafe Open Source AI Researchers, often working at closed labs, argue that open models are inherently unsafe because you cannot control them after their release, and because they can be fine-tuned for any malicious use case. While this argument is theoretically true, it also assumes that closed models are safer, that their guardrails work, and that their providers take measures against misuse. Therefore, I went ahead and researched the misuse of both open and closed models and compiled a list. For this, I looked into third-party reports examining
saved by
related reading
- Building Auto Mode for Open Models — Benjamin Andersonbenanderson.work
- AI safety advocates: it's time to be more self-reflective about our personal usage of frontier AIconnorsscratchpad.substack.com
- Countering misuse of AI: September 2026 / Anthropicanthropic.com
- An alignment assessment of recent cybersecurity incidentsanthropic.com
- A Safe Path to Open Weights - Thinking Machines Labthinkingmachines.ai
- AI safety is not a model propertysubstack.com
- Why open-weight models without guardrails are a AI safety risk : NPRnpr.org
- Investigating three real-world incidents in our cybersecurity evaluations \ Anthropicanthropic.com
- Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face — LessWronglesswrong.com
- AI models can be dangerous before public deploymentmetr.org
- [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
- The End-State Fallacy: Where Is AI Security Headed?endstatefallacy.com