Transformer²: Self-Adaptive LLMs

Transformer²: Self-Adaptive LLMs

--> Summary Adaptation is one of the most remarkable phenomena in nature. From the way an octopus can change their skin color to blend into its surroundings, to how the human brain rewires itself after an injury , allowing individuals to recover lost functions and adapt to new ways of thinking or moving. Living organisms exhibit adaptability that allows life to flourish in diverse and ever-changing environments. In the field of AI, the concept of adaptation holds a similar allure. Imagine a machine learning system that could adjust its own weights dynamically to thrive in unfamiliar settings,

saved by

• Winnie Xu

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Self-Adapting Language Modelsarxiv.org
• [2506.06105] Text-to-LoRA: Instant Transformer Adaptionarxiv.org
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
• Can LLMs Be Computers?percepta.ai
• [2604.16812] Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2605.12484] Learning, Fast and Slow: Towards LLMs That Adapt Continuallyarxiv.org
• A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencealphaxiv.org
• AI Revolution - Transformers and Large Language Models (LLMs)blog.eladgil.com
• Introspection Adapters: Training LLMs to Report Their Learned Behaviorsarxiv.org
• [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org