What is input/output filtering in AI safety? - by Sarah
Large Language Models (LLMs) can sometimes produce dangerous or harmful outputs, such as instructions for dangerous activities (like building weapons), inappropriate or offensive content, or misinformation.
Large Language Models (LLMs) can sometimes produce dangerous or harmful outputs, such as instructions for dangerous activities (like building weapons), inappropriate or offensive content, or misinformation. Developers implement measures during training to try and stop this from occurring. These include Reinforcement Learning From Human Feedback (where models are trained on human preference data), supervised fine-tuning (where they are trained to imitate harmless behaviour) and input data filtering (where potentially harmful data is removed from the training set). However, these methods are…
saved by
related reading
- [2312.06942] AI Control: Improving Safety Despite Intentional Subversionarxiv.org
- [2507.11473] Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetyarxiv.org
- 2312.06674arxiv.org
- Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
- What We’ve Learned From A Year of Building with LLMs – Applied LLMsapplied-llms.org
- RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentarxiv.org
- The lethal trifecta for AI agents: private data, untrusted content, and external communicationsimonwillison.net
- [2401.05566] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingarxiv.org
- The fragile foundations of CoT monitoring | Christopher Pottsweb.stanford.edu
- [2512.11949] Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorsarxiv.org
- Refusal in LLMs is mediated by a single direction — LessWronglesswrong.com
- Legilimens: Practical and Unified Content Moderation forLarge Language Model Servicesarxiv.org