Should We Train Against (CoT) Monitors? — LessWrong
The question I actually try to answer in this post is a broader one (that doesn't work as well as a title): Should we incorporate proxies for desired…
x Should We Train Against (CoT) Monitors? — LessWrong Aether AI Frontpage 50 Should We Train Against (CoT) Monitors? by RohanS 23rd Apr 2026 39 min read 7 50 The question I actually try to answer in this post is a broader one (that doesn't work as well as a title): Should we incorporate proxies for desired behavior into LLM alignment training? Epistemic status: My best guess. I tentatively claim that we should be more open to incorporating proxies for desired behavior into LLM training, but I try to clarify the spectrum of possible answers beyond just 'yes' and 'no,' and I try to present and s
saved by
related reading
- Alignment Faking Mitigationsalignment.anthropic.com
- We need 3rd party Training-Run Assessments — LessWronglesswrong.com
- Simulators — LessWronglesswrong.com
- Deep Deceptiveness — LessWronglesswrong.com
- The case for ensuring that powerful AIs are controlled — LessWronglesswrong.com
- The Most Forbidden Technique — LessWronglesswrong.com
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
- Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
- Simulated Users & Sad LLMs1a3orn.com
- How far does alignment midtraining generalize?alignment.openai.com
- Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationarxiv.org