✳flâneur — a map of the web's best reading

Transformer Progress | Leela Chess Zero

lczero.org · 1,984 words · saved by 1 readers

Recently, the transformer architecture has dominated domains as diverse as vision and natural language processing. Over the past two years, the Lc0 team has been trying to answer the following question: What chess-specific enhancements can be made to the transformer architecture? Leela’s nets have historically struggled with long-range dependencies, failing to recognize positional and tactical ideas involving squares that are far away from each other, such as multiloaded pieces. This was because Leela’s models used a convolution-based architecture, like those of the DeepMind AlphaZero project on which Lc0 is based. Effectively, each square iteratively analyzes the information at each adjacent square and uses that information to refine its representation. The main drawback of this approach is the small receptive fields of the convolution filters. For the a1 square to “learn” about what piece is on h8, the information must make at least 7 trips from square to square. Our strongest transf

Transformer Progress | Leela Chess Zero 2024-02-28 Transformer Progress ⚠️ Update (2026): This architecture was accepted as a conference paper at ICLR 2026 — Chessformer: A Unified Architecture for Chess Modeling . The Leela-CF model in the paper is the same network referred to as BT4 on this website. This blog post and the earlier preprint Mastering Chess with a Transformer Model should be treated as superseded by the ICLR 2026 paper; please cite that version instead. Recently, the transformer architecture has dominated domains as diverse as vision and natural language processing. Over the pa

Explore this link on the map →

saved by

related reading