Language Models will be Scaffolds | Alex L. Zhang
alexzhang13.github.io · 673 words · saved by 2 readers
Alex Zhang's Website.
Alex Zhang, Feb 25, 2026. I have been somewhat convinced since before I started my PhD that the language models we interact with in the (near) future will be what we call scaffolds today. For the earlier half of this decade, it was generally believed that any work other than improving the base “neural” language model (i.e. an end-to-end neural network model like Opus 4.6 or Qwen3-8B) was a contrarian take on Sutton’s Bitter Lesson that would ultimately fall victim to scale. And this belief was genuinely a good bet since the invention of the Transformer in 2017: religiously following this…
saved by
related reading
- Alex L. Zhangalexzhang13.github.io
- Transformer Circuits Threadtransformer-circuits.pub
- machine learning imindslice.substack.com
- Language Models, World Models, and Human Model-Buildinglingo.csail.mit.edu
- Composer2.pdfcursor.com
- GenAI Handbookgenai-handbook.github.io
- alex zhang (@a1zhang) on Xx.com
- Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
- My picture of the present in AI — LessWronglesswrong.com
- Scaling: The State of Play in AIoneusefulthing.org
- Automated Alignment Researchers: Using large language models to scale scalable oversight \ Anthropicanthropic.com
- 2308.03958arxiv.org