✳flâneur — a map of the web's best reading

Chapter 4: Alignment Science - ARENA

learn.arena.education · 1,503 words · saved by 1 readers

Please send any problems / bugs on the #errata channel in the Slack group, and ask any questions on the dedicated channels for this chapter of material. If you want to change to dark mode, you can do this by clicking the three horizontal lines in the top-right, then navigating to Settings → Theme. Links to all other chapters: (0) Fundamentals, (1) Transformer Interpretability, (2) RL. Emergent Misalignment (EM) was discovered in early 2025, mostly by accident. The authors of a paper on training insecure models to write unsafe code noticed that their models actually reported very low alignment with human values. Further investigation showed that these models had learned some generalized notion of misaligned behaviour just from being trained on insecure code, and this was spun off into its own paper (PSA to people reading this - sadly it's not always that easy to make a hit AI safety paper.) This paper suggests a result which is also in line with an earlier paper Refusal in LLMs is Media

[4.1] Emergent Misalignment Colab: exercises | solutions Please send any problems / bugs on the #errata channel in the Slack group , and ask any questions on the dedicated channels for this chapter of material. If you want to change to dark mode, you can do this by clicking the three horizontal lines in the top-right, then navigating to Settings → Theme. Links to all other chapters: (0) Fundamentals , (1) Transformer Interpretability , (2) RL . Introduction Emergent Misalignment (EM) was discovered in early 2025, mostly by accident. The authors of a paper on training insecure models to write u

Explore this link on the map →

related reading