Chapter 1: Transformer Interpretability - ARENA

Please send any problems / bugs on the #errata channel in the Slack group, and ask any questions on the dedicated channels for this chapter of material. If you want to change to dark mode, you can do this by clicking the three horizontal lines in the top-right, then navigating to Settings → Theme. Links to all other chapters: (0) Fundamentals, (1) Transformer Interpretability, (2) RL. Note - unless otherwise specified, first person here refers to the primary researcher, Neel Nanda. Emergent World Representations is a fascinating recent ICLR Oral paper from Kenneth Li et al, summarised in Kenneth's excellent post on the Gradient. They trained a model (Othello-GPT) to play legal moves in the board game Othello, by giving it random games (generated by choosing a legal next move uniformly at random) and training it to predict the next move. The headline result is that Othello-GPT learns an emergent world representation - despite never being explicitly given the state of the board, and just

[1.5.3] OthelloGPT Colab: exercises | solutions Please send any problems / bugs on the #errata channel in the Slack group , and ask any questions on the dedicated channels for this chapter of material. If you want to change to dark mode, you can do this by clicking the three horizontal lines in the top-right, then navigating to Settings → Theme. Links to all other chapters: (0) Fundamentals , (1) Transformer Interpretability , (2) RL . Introduction Note - unless otherwise specified, first person here refers to the primary researcher, Neel Nanda. Emergent World Representations is a fascinating

related reading

• Actually, Othello-GPT Has A Linear Emergent World Representation - Neel Nandaneelnanda.io
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• [2210.13382] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskarxiv.org
• Actually, Othello-GPT Has A Linear Emergent World Representation — AI Alignment Forumalignmentforum.org
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Chapter 1: Transformer Interpretability - ARENAlearn.arena.education
• Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskarxiv.org
• Transformer Circuits Threadtransformer-circuits.pub
• OthelloGPT learned a bag of heuristics — LessWronglesswrong.com