✳flâneur — a map of the web's best reading
GPT in 60 Lines of NumPy | Jay Mody
jaykmody.com · 9,202 words · saved by 4 readers
Implementing a GPT model from scratch in NumPy.
GPT in 60 Lines of NumPy | Jay Mody GPT in 60 Lines of NumPy January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy . We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improvements on purpose to keep it as simple as possible while remaining complete. All the code for this blog post can be found at github.com/ja
Explore this link on the map →saved by
related reading
- microgptkarpathy.github.io
- What Is ChatGPT Doing … and Why Does It Work?-Stephen Wolfram Writingswritings.stephenwolfram.com
- gpt-4.pdfcdn.openai.com
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
- GPT-4openai.com
- The Annotated Transformernlp.seas.harvard.edu
- interpreting GPT: the logit lens — AI Alignment Forumalignmentforum.org
- GitHub - karpathy/nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs. · GitHubgithub.com
- interpreting GPT: the logit lens — LessWronglesswrong.com
- Five years of GPT progressfinbarr.ca
- GenAI Handbookgenai-handbook.github.io
- GPTQ: Accurate Post-Training Quantization for Generative Pre-Trained Transformersarxiv.org