“Alignment Engineering” vs. “Misalignment Science” — LessWrong
lesswrong.com · 9,946 words · saved by 12 readers
There has been much discussion recently around whether a large portion of alignment research is net negative. Without endorsing or refuting them, the…
11 min read • • • • • • 422 5th Oct 2026 422 There has been much discussion recently around whether a large portion of alignment research is net negative. Without endorsing or refuting them, the basic arguments here are: Prosaic alignment of models is becoming a bottleneck for capabilities. Therefore improving the prosaic alignment of models enables faster capabilities advances, which bring us closer to RSI. It is unlikely these prosaic alignment methods remain sufficient during the RSI loop, and so this work brings us closer to doom. Furthermore, dealing with these more…
saved by
- Yixiong Hao
- Lydia Nottingham
- Andrew Wu
- Gloria Ma
- Espen Slettnes
- Jaeho Lee
- Andrea Chen
- Ishaan Panigrahi
- Bhagyesh Kumar
- Micah Hees
- David Garsten
- Mark Stanley
related reading
- Current AIs seem pretty misaligned to me — LessWronglesswrong.com
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- Alignment is not solved but it increasingly looks solvablealigned.substack.com
- Readings on the nature of alignment researchcasparoesterheld.com
- Defining alignment research — LessWronglesswrong.com
- I think alignment work is more promising than control work — LessWronglesswrong.com
- The Artificiality of Alignmentjoinreboot.org
- Explaining AI Alignment as an NLPer and Why I am Working on Itruiqizhong.substack.com
- Model Organisms of Misalignment: The Case for a New Pillar of Alignment Research — AI Alignment Forumalignmentforum.org
- Recommendations for Technical AI Safety Research Directionsalignment.anthropic.com
- Automated alignment is harder than you thinkarxiv.org
- Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com