Building Trustworthy AI Agents - Schneier on Security
View or Download in PDF Format The promise of personal AI assistants rests on a dangerous assumption: that we can trust systems we haven’t made trustworthy. We can’t. And today’s versions are failing us in predictable ways: pushing us to do things against our own best interests, gaslighting us with doubt about things we are or that we know, and being unable to distinguish between who we are and who we have been. They struggle with incomplete, inaccurate, and partial context: with no standard way to move toward accuracy, no mechanism to correct sources of error, and no accountability when wrong information leads to bad decisions...
View or Download in PDF Format The promise of personal AI assistants rests on a dangerous assumption: that we can trust systems we haven’t made trustworthy. We can’t. And today’s versions are failing us in predictable ways: pushing us to do things against our own best interests, gaslighting us with doubt about things we are or that we know, and being unable to distinguish between who we are and who we have been. They struggle with incomplete, inaccurate, and partial context: with no standard way to move toward accuracy, no mechanism to correct sources of error, and no accountability when…
saved by
related reading
- The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
- Cognitive Security as an AI Safety Cause Area — LessWronglesswrong.com
- AI Control: Improving Safety Despite Intentional Subversion — LessWronglesswrong.com
- The lethal trifecta for AI agents: private data, untrusted content, and external communicationsimonwillison.net
- secret-loyalties-whitepaper.pdfformationresearch.com
- The Artificial Selftheartificialself.ai
- AI Integrity: Defending Against Backdoors and Secret Loyalties - Institute for AI Policy and Strategyiaps.ai
- The End-State Fallacy: Where Is AI Security Headed?endstatefallacy.com
- The Case Against AI Control Research — LessWronglesswrong.com
- [2004.07213] Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claimsarxiv.org
- AI Safety for Fleshy Humans: a whirlwind touraisafety.dance
- Why are AI agents lying, cheating and coordinating?yoshuabengio.org