A tale of three theories: sparsity, frustration, and statistical field theory — LessWrong

This post is an informal preliminary writeup of a project that I've been working on with friends and collaborators. Some of the theory was developed…

x A tale of three theories: sparsity, frustration, and statistical field theory — LessWrong AI Frontpage 63 A tale of three theories: sparsity, frustration, and statistical field theory by Dmitry Vaintrob 25th Jan 2026 22 min read 0 63 This post is an informal preliminary writeup of a project that I've been working on with friends and collaborators. Some of the theory was developed jointly with Zohar Ringel, and we hope to write a more formal paper on it this year. Experiments are joint with Lucas Teixeira (and also an extensive use of llm assistants). This work is part of the research agenda

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• Statistical Mechanics of Deep Learningganguli-gang.stanford.edu
• [2608.27540] Towards a mathematical theory of superpositionarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Interpretability Dreamstransformer-circuits.pub
• On neural scaling and the quanta hypothesisericjmichaud.com
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• [2410.12101] The Persian Rug: solving toy models of superposition using large-scale symmetriesarxiv.org
• Statistical Physics for Ambitious Interpretability: A Workshop Retrospective — LessWronglesswrong.com
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com