Leela Interpretability

One of our early experiments was to do activation patching. We patch a small part of Leela's activations from the forward pass of a corrupted version of a puzzle into the forward pass on the original puzzle board state. Measuring the effect on the final output tells us how important that part of Leela's activations was.

Evidence of Learned Look-Ahead in a Chess-Playing Neural Network 1UC Berkeley, 2Independent NeurIPS 2024 Choose A Chessboard State Do neural networks learn to implement algorithms involving look-ahead or search in the wild? Or do they only ever learn simple heuristics? We investigate this question for Leela Chess Zero, arguably the strongest existing chess-playing network. We find intriguing evidence of learned look-ahead in a single forward pass. This website showcases some of our results, see our paper for much more. Setup We consider chess puzzles such as the following (you can…

related reading

• Transformer Progress | Leela Chess Zerolczero.org
• Grandmaster-Level Chess Without Searcharxiv.org
• Actually, Othello-GPT Has A Linear Emergent World Representation - Neel Nandaneelnanda.io
• Chess-GPT’s Internal World Model | Adam Karvonenadamkarvonen.github.io
• OthelloGPT learned a bag of heuristics — LessWronglesswrong.com
• Jane Street Blog - Can you reverse engineer our neural network?blog.janestreet.com
• Attribution Patching: Activation Patching At Industrial Scale - Neel Nandaneelnanda.io
• Can activation verbalizers surface an internal chain of thought? — LessWronglesswrong.com
• [2210.13382] Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskarxiv.org
• LLM Chess: Benchmarking Reasoning and Instruction-Following in LLMs through Chessarxiv.org
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• [2512.11949] Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorsarxiv.org