God Help Us, Let’s Try To Learn About Mechanistic Interpretability Techniques
astralcodexten.com · 4,814 words · saved by 2 readers
...
The Story So Far Mechanistic interpretability is the science of “reading an AI’s mind”. Large language models are “grown, not built”. Researchers run training data through a neural network. Eventually this creates a working AI; nobody really knows how. But a neural network is just a set of simulated neurons on a computer. The person with the computer can see the neurons, the connections between them, and which ones activate when the AI answers questions. So it seems like it should be possible to “reverse engineer” the AI. This would be scientifically useful: understand how AIs work, with…
saved by
related reading
- God Help Us, Let's Try To Understand The Paper On AI Monosemanticityastralcodexten.com
- Tracing the thoughts of a large language model \ Anthropicanthropic.com
- How To Become A Mechanistic Interpretability Researcher — AI Alignment Forumalignmentforum.org
- An Ambitious Vision for Interpretability — AI Alignment Forumalignmentforum.org
- On Optimism for Interpretabilitygoodfire.ai
- God Help Us, Let's Try To Understand The Paper On AI Monosemanticitysubstack.com
- A Pragmatic Vision for Interpretability — AI Alignment Forumalignmentforum.org
- Mapping the Mind of a Large Language Model \ Anthropicanthropic.com
- Introduction to Mechanistic Interpretability - by Sarahblog.bluedot.org
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- Neel Nanda on Mechanistic Interpretability: Progress, Limits, and Paths to Safer AI — EA Forumforum.effectivealtruism.org
- How AI Is Learning to Think in Secretnickandresen.substack.com