how neural networks think at scale

Neural networks like Large Language Models (LLMs) first convert a sequence of input tokens into an 𝑛 n dimensional vector by applying a linear transformation. This 𝑛 − n− dimensional vector, called an embedding, then goes through a series of computations or layers like Attention and Multi Layer Perceptrons (MLP) and then is converted back into some output token by applying another linear transformation. These layers perform various linear and sometimes non-linear transformations on the embedding. The intermediate representations formed in the network at the end of each layer are called the hidden/ latent vectors or activations of the network. The embedding passes through a series of transformations at each layer that extract or encode different kinds of information about the data in the activations. Mechanistic Interpretability (Mech Interp) aims at understanding these activations to reverse engineer how each component of the neural network cause it to produce a specific output. S

how neural networks think at scale Marmik Chaudhari , Nishkal Hundia Introduction Neural networks like Large Language Models (LLMs) first convert a sequence of input tokens into an n n n dimensional vector by applying a linear transformation. This n − n- n − dimensional vector, called an embedding , then goes through a series of computations or layers like Attention and Multi Layer Perceptrons (MLP) and then is converted back into some output token by applying another linear transformation. These layers perform various linear and sometimes non-linear transformations on the embedding. The inter

related reading

• Toy Models of Superpositiontransformer-circuits.pub
• Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
• What Would Non-Linear Features Actually Look Like? — Liv Gortonlivgorton.com
• The Building Blocks of Interpretabilitydistill.pub
• Softmax Linear Unitstransformer-circuits.pub
• Lucius Bushnaq's Shortform — LessWronglesswrong.com
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Activation space interpretability may be doomed — LessWronglesswrong.com
• The World Inside Neural Networksgoodfire.ai
• SAE feature geometry is outside the superposition hypothesis — LessWronglesswrong.com
• Transformer Circuits Threadtransformer-circuits.pub
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io