Why study alignment interventions on pre-RL checkpoints? — LessWrong
This is a dual post that lays out our current research project where we compare pre-RL-training methods on their ability to prevent models from ‘prot…
x Why study alignment interventions on pre-RL checkpoints? — LessWrong AI Frontpage 64 Why study alignment interventions on pre-RL checkpoints? by Edward James Young , Puria , Cam 8th Jul 2026 7 min read 1 64 This is a dual post that lays out our current research project where we compare pre-RL-training methods on their ability to prevent models from ‘ proto-training gaming ,’ which we predict is selected for over the course of production RL post-training. In this post, we outline what we mean by pre-RL ‘alignment checkpoints’, give our reasons for focussing on these stages of training, and su
saved by
related reading
- [2601.10160] Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentarxiv.org
- How hard is it to inoculate against misalignment generalization? — LessWronglesswrong.com
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment — LessWronglesswrong.com
- How far does alignment midtraining generalize?alignment.openai.com
- Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignmentalignmentpretraining.ai
- Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
- Should We Train Against (CoT) Monitors? — LessWronglesswrong.com
- Alignment Faking Mitigationsalignment.anthropic.com
- Metagaming matters for training, evaluation, and oversightalignment.openai.com
- Defining alignment research — LessWronglesswrong.com
- Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org