A tale of three theories: sparsity, frustration, and statistical field theory — LessWrong

This post is an informal preliminary writeup of a project that I've been working on with friends and collaborators. Some of the theory was developed…

x A tale of three theories: sparsity, frustration, and statistical field theory — LessWrong AI Frontpage 63 A tale of three theories: sparsity, frustration, and statistical field theory by Dmitry Vaintrob 25th Jan 2026 22 min read 0 63 This post is an informal preliminary writeup of a project that I've been working on with friends and collaborators. Some of the theory was developed jointly with Zohar Ringel, and we hope to write a more formal paper on it this year. Experiments are joint with Lucas Teixeira (and also an extensive use of llm assistants). This work is part of the research agenda

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• Maybe I was too harsh on deep learning theory (three days ago) — LessWronglesswrong.com
• The generalization phase diagram — LessWronglesswrong.com
• What if not Circuits? — LessWronglesswrong.com
• [2604.21691] There Will Be a Scientific Theory of Deep Learningarxiv.org
• [2608.27540] Towards a mathematical theory of superpositionarxiv.org
• Renormalization Redux: QFT Techniques for AI Interpretability — LessWronglesswrong.com
• A Theory of Deep Learning | Elements of a Vector Spaceelonlit.com
• Statistical Physics for Ambitious Interpretability: A Workshop Retrospective — LessWronglesswrong.com
• Weight-Sparse Circuits May Be Interpretable Yet Unfaithful — LessWronglesswrong.com
• Elon Litman | Elements of a Vector Spaceelonlit.com
• [Interim research report] Taking features out of superposition with sparse autoencoders — LessWronglesswrong.com