Cross-Model Activation Generalizability Isn't Strong (Yet) — LessWrong

TL;DR * Tested activation similarities across different LLM families (Llama, Gemma, Qwen, Pythia) at small scale (1~3B) * CKA Similarity : Cross-ar…

x Cross-Model Activation Generalizability Isn't Strong (Yet) — LessWrong Interpretability (ML & AI) AI Frontpage 6 Cross-Model Activation Generalizability Isn't Strong (Yet) by J Lee 6th Apr 2026 17 min read 0 6 TL;DR Tested activation similarities across different LLM families (Llama, Gemma, Qwen, Pythia) at small scale (1~3B) CKA Similarity : Cross-architectural activation similarity is statistically real, but weak. Within-family activations are much stronger (4~9x) Linear Transferability : Trained linear bridges for linear activation transfers for binary classification and next token predic

related reading

• Transformer Circuits Threadtransformer-circuits.pub
• How do LLMs generalize when we do training that is intuitively compatible with two off-distribution behaviors? — LessWronglesswrong.com
• LLM Architecture Gallery | Sebastian Raschka, PhDsebastianraschka.com
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Backdoors have universal representations across large language models — LessWronglesswrong.com
• Insights on Crosscoder Model Diffingtransformer-circuits.pub
• I am worried about near-term non-LLM AI developments — LessWronglesswrong.com
• Interpretability Infrastructure at Frontier Scale: Harvesting Activations from a Trillion-Parameter Modelgoodfire.ai
• Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers — LessWronglesswrong.com
• [2512.15674] Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainersarxiv.org
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainersalignment.anthropic.com