Anthropic on Twitter: "Here is another example of increasing abstraction with scale, where an AI Assistant reasoned through an AI alignment question. The top influential sequence for the 810M model shares a short phrase with the query, while the one for the 52B model is more thematically related. https://t.co/DkDNWgAhr4" / X
To view keyboard shortcuts, press question mark View keyboard shortcuts Messages Home Explore 2 Notifications Messages Lists Bookmarks Communities Verified Profile More Post Aleksi Maunu (visiting Bay Area) @Lekksuu Post See new Tweets Conversation Anthropic @AnthropicAI · Aug 8 Identifying the most influential training sequences revealed that generalization patterns become much more sophisticated and abstract with scale. For example, here are the most influential sequences for 810 million and 52 billion parameter models for a math word problem: ALT 1.9 x 1 14 100 16.6K Anthropic @AnthropicAI Here is another example of increasing abstraction with scale, where an AI Assistant reasoned through an AI alignment question. The top influential sequence for the 810M model shares a short phrase with the query, while the one for the 52B model is more thematically related. ALT 9:14 AM · Aug 8, 2023 · 9,666 Views 2 Reposts 1 Quote 56 Likes 3 Bookmarks 1.9 x Post your reply! Reply Anthropic
saved by
related reading
- AI #24: Week of the Podcast — LessWronglesswrong.com
- On the Biology of a Large Language Modeltransformer-circuits.pub
- Tracing the thoughts of a large language model \ Anthropicanthropic.com
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
- Automated Alignment Researchers: Using large language models to scale scalable oversight \ Anthropicanthropic.com
- The Scaling Hypothesis · Gwern.netgwern.net
- Alignment remains a hard, unsolved problem — LessWronglesswrong.com
- Tracing Model Outputs to the Training Data \ Anthropicanthropic.com
- Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub
- Scaling: The State of Play in AIoneusefulthing.org
- Mapping the Mind of a Large Language Model \ Anthropicanthropic.com
- Auditing language models for hidden objectives \ Anthropicanthropic.com