Self-exfiltration is a key dangerous capability

We need to measure whether LLMs could “steal” themselves

Self-exfiltration is a key dangerous capability We need to measure whether LLMs could “steal” themselves Jan Leike Sep 13, 2023 26 18 2 Share Recently a number of projects have emerged on measuring LLM capabilities on a number of tasks that imply high risk, such as: The model’s ability to autonomously replicate and adapt The model’s ability to assist in (bio)weapon development The model’s understanding of its own situation The model’s ability to do ML research and pretrain new models or fine-tune itself The model’s ability to persuade humans The model’s ability to find and exploit security vul

saved by

• Asher P
• Lydia Nottingham
• Ekaterina Leksina

related reading

• Self-exfiltration is a key dangerous capabilityaligned.substack.com
• The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
• Agentic misalignment: How LLMs could be insider threats \ Anthropicanthropic.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMsarxiv.org
• AI Control: Improving Safety Despite Intentional Subversion — AI Alignment Forumalignmentforum.org
• The lethal trifecta for AI agents: private data, untrusted content, and external communicationsimonwillison.net
• Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack? — LessWronglesswrong.com
• Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
• AI models can be dangerous before public deploymentmetr.org
• A basic systems architecture for AI agents that do autonomous research — LessWronglesswrong.com
• [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org