Tracing Model Outputs to the Training Data \ Anthropic

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

Alignment Research Tracing Model Outputs to the Training Data Aug 8, 2023 As large language models become more powerful and their risks become clearer, there is increasing value to figuring out what makes them tick. In our previous work , we have found that large language models change along many personality and behavioral dimensions as a function of both scale and the amount of fine-tuning. Understanding these changes requires seeing how models work, for instance to determine if a model’s outputs rely on memorization or more sophisticated processing. Understanding the inner workings of langua

related reading

• Tracing the thoughts of a large language model \ Anthropicanthropic.com
• On the Biology of a Large Language Modeltransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• Automated Alignment Researchers: Using large language models to scale scalable oversight \ Anthropicanthropic.com
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Transformer Circuits Threadtransformer-circuits.pub
• In-context Learning and Induction Headstransformer-circuits.pub
• A “diff” tool for AI: Finding behavioral differences in new models \ Anthropicanthropic.com
• Emergent introspective awareness in large language models \ Anthropicanthropic.com
• Core views on AI safety: When, why, what, and how \ Anthropicanthropic.com
• Mapping the Mind of a Large Language Model \ Anthropicanthropic.com
• Influence functions - why, what and how — LessWronglesswrong.com