The Transformer Family | Hacker News
Can anyone recommend some great courses or other online resources for getting up to speed on the state-of-the-art with respect to AI? Not really so much looking for an "ELI5" but more of a "you have a strong programming and very-old-school AI background, here are the steps/processes you need to know to understand modern tools". Edit: thanks for all the great replies, super helpful! https://karpathy.ai/zero-to-hero.html You can quickly get overwhelmed by the million good resources out there so I'll keep it to these three. If you have a strong CS background, they'll take you a long way: (1) Transformers from Scratch: https://peterbloem.nl/blog/transformers (2) Attention Is All You Need: https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547de... (3) Formal Algorithms for Transformers: https://arxiv.org/abs/2207.09238 And subsequent follow-ups (ROME, editing transformer arch): https://youtu.be/_NMQyOu2HTo I find the channel amazing explaining super complex topics in simple enough term
The Transformer Family | Hacker News Hacker News new | past | comments | ask | show | jobs | submit login The Transformer Family ( lilianweng.github.io ) 254 points by alexmolas on Jan 29, 2023 | hide | past | favorite | 46 comments hn_throwaway_99 on Jan 29, 2023 | next [–] Somewhat off topic. As someone who did some neural network programming in Matlab a couple decades ago, I always feel a bit dismayed that I'm able to understand so little about modern AI given the explosion in advances in the field starting in about the late 00s or so with things like convolutional neural networks and deep
saved by
related reading
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- Transformers Laid Outgoyalpramod.github.io
- Transformers from Scratche2eml.school
- Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper? | Hacker Newsnews.ycombinator.com
- The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
- The Annotated Transformernlp.seas.harvard.edu
- GenAI Handbookgenai-handbook.github.io
- Everything About Transformerskrupadave.com
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- Transformers from scratch | peterbloem.nlpeterbloem.nl
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com