flâneur

Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion

arxiv.org · 7,709 words · saved by 1 readers

We introduce Orthrus, a simple and efficient dual-architecture framework that unifies the exact generation fidelity of autoregressive Large Language Models (LLMs) with the high-speed parallel token generation of diffusion models. The sequential nature of standard autoregressive decoding represents a fundamental bottleneck for high-throughput inference. While diffusion language models attempt to break this barrier via parallel generation, they suffer from significant performance degradation, high training costs, and a lack of rigorous convergence guarantees. Orthrus resolves this dichotomy natively. Designed to seamlessly integrate into existing Transformers, the framework augments a frozen LLM with a lightweight, trainable module to create a parallel diffusion view alongside the standard autoregressive view. In this unified system, both views attend to the exact same high-fidelity Key-Value (KV) cache; the autoregressive head executes context pre-filling to construct accurate KV repres

Orthrus: Memory-Efficient Parallel Token Generation via Dual-View Diffusion Chien Van Nguyen University of Oregon &Chaitra Hegde Google DeepMind &Van Cuong Pham University of Oregon Ryan A. Rossi Adobe Research &Franck Dernoncourt Adobe Research &Thien Huu Nguyen University of Oregon Email: chienn@uoregon.edu Abstract We introduce Orthrus , a simple and efficient dual-architecture framework that unifies the exact generation fidelity of autoregressive Large Language Models (LLMs) with the high-speed parallel token generation of diffusion models. The sequential nature of standard autoregressive

saved by

related reading