How might LLMs store facts | Chapter 7, Deep Learning - YouTube

2 of 7 lessons complete Follow along using the transcript. Transcript & Summary 2 of 7 lessons complete Follow along using the transcript. New New New New New New New New

Unpacking the multilayer perceptrons in a transformer, and how they may store facts Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support An equally valuable form of support is to share the videos. AI Alignment forum post from the Deepmind researchers referenced at the video's start: https://www.alignmentforum.org/posts/iGuwZTHWb6DFY3sKB/fact-finding-attempting-to-reverse-engineer-factual-recall Anthropic posts about superposition referenced near the end: https://transformer-circuits.pub/2022/toy_model/index.html…

saved by

• Manh Duc Hoang
• Nadeem S

related reading

• Deep Dive into LLMs like ChatGPTyoutube.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• Fact Finding: Attempting to Reverse-Engineer Factual Recall on the Neuron Level (Post 1) — AI Alignment Forumalignmentforum.org
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com
• How LLMs Actually Work | 0xkato0xkato.xyz
• Let's build GPT: from scratch, in code, spelled out.youtube.com
• Everything About Transformerskrupadave.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• interpreting GPT: the logit lens — LessWronglesswrong.com