yifanzhang-pro/recurrent-looped-tranformer: Recurrent Looped Transformer ·
Recurrent Looped Transformer (RLT) carries latent computation across every prompt and response token. A causal encoder builds global key–value memory; a recurrent decoder combines that memory with sliding-window attention (SWA) and feedback from its previous final hidden state. Author: Yifan Zhang Date: September 12, 2026 [English Paper] [中文论文] [Project Website] “Infinite depth” means an extensible temporal computation path as the sequence grows, not infinite computation within one token. Reasoning improvements, hardware speedups, and RL scaling are research goals rather than measured results in this report. For token 𝑥 𝑡 , let 𝑒 𝑡 be its causal encoder representation and 𝑀 ≤ 𝑡 the encoder-derived global KV memory. The complete decoder state includes both the recurrent output and layerwise SWA KV: The concrete configuration uses 48 encoder layers + 48 decoder layers, with compatible attention and FFN weights shared across stages. Decoder blocks additionally perform encoder-m