Alex L. Zhang

Alex Zhang, Sep 26, 2026. Some short musings on the shape of language models, e.g. what it means to design a language model around a harness, and not the other way around. Since the release of ChatGPT, I’ve observed that the input / output shape of language models has roughly remained static. A reasonable guess is that users would always prefer to use the best models available, and frontier mode

Alex Zhang, Sep 26, 2026. Some short musings on the shape of language models, e.g. what it means to design a language model around a harness, and not the other way around. Since the release of ChatGPT, I’ve observed that the input / output shape of language models has roughly remained static. A reasonable guess is that users would always prefer to use the best models available, and frontier model labs would rather not deviate from the model shape that has proven to work, because it is very expensive to bet on alternatives. As a result, all work on designing language agents has been around…

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Language Models will be Scaffoldsalexzhang13.github.io
• Recursive Language Models | Alex L. Zhangalexzhang13.github.io
• Recursive Language Models — Alex Zhang, MIT PhDyoutube.com
• Alex L. Zhangalexzhang13.github.io
• Language model harnesses are compositional generalizersalexzhang13.github.io
• A History of Large Language Modelsgregorygundersen.com
• RLMs are Compositional Generalizers with Alex Zhang from MITyoutube.com
• Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
• To Understand Language is to Understand Generalization | Eric Jangevjang.com
• Large Language Models Reading List | Sebastian Raschka, PhDsebastianraschka.com
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org