Mor Geva on X: "What's in a neuron? 💫 (an atypically long, almost personal post) Neurons in LMs have always been a fascinating object to study. I've been studying them since 2020, viewing them as key-value memory cells, analyzing what they capture in vocabulary space, and how they compose https://t.co/179ELFnEw3" / X

What's in a neuron? 💫 (an atypically long, almost personal post) Neurons in LMs have always been a fascinating object to study. I've been studying them since 2020, viewing them as key-value memory cells, analyzing what they capture in vocabulary space, and how they compose https://t.co/179ELFnEw3

@megamor2: What's in a neuron? (an atypically long, almost personal post) Neurons in LMs have always been a fascinating object to study. I've been studying them since 2020, viewing them as key-value memory cells, analyzing what they capture in vocabulary space, and how they compose together to form features. https:// arxiv.org/abs/2012.14913 https:// arxiv.org/abs/2203.14680 https:// arxiv.org/abs/2501.08319 https:// arxiv.org/abs/2506.10920 But many neurons still remain opaque! They do many things. Our recent work led by @AsafAvrahamy tackles this challenge by decomposing neuron wei

saved by

• Ishaan Panigrahi

related reading

• How LLMs Actually Work | 0xkato0xkato.xyz
• Toy Models of Superpositiontransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• Zoom In: An Introduction to Circuitsdistill.pub
• Language models can explain neurons in language modelsopenaipublic.blob.core.windows.net
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• Neuronpedianeuronpedia.org
• Multimodal Neurons in Artificial Neural Networksdistill.pub
• The World Inside Neural Networksgoodfire.ai
• https://io0.github.ioio0.github.io
• Fact Finding: Attempting to Reverse-Engineer Factual Recall on the Neuron Level (Post 1) — AI Alignment Forumalignmentforum.org
• how neural networks think at scalemarmik.xyz