Inference Engineering Master Map
inference-engineering-master-map.vercel.app · 1,988 words · saved by 1 readers
N/A
Scope: dense, decoder-only Transformers unless stated otherwise. GB = 10⁹ bytes; GiB = 2³⁰ bytes. FLOPs count work; FLOP/s measures a rate. Hardware settings are illustrative. FOUNDATION1 module available · 12 topics on the roadmap Select a topic to see its prerequisites and what it unlocks. ● Published lesson · ○ Roadmap only. 13 layers · 1 system LAYER 1Foundation Foundations Prerequisites Core question Unlocks Algorithm Tensor ops Kernel Hardware Memory Runtime Serving User latency Module 1 · Hardware & performance foundations Build the resource model behind every…
saved by
related reading
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- All About Transformer Inferencejax-ml.github.io
- All About Rooflines | How To Scale Your Modeljax-ml.github.io
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- How To Scale Your Modeljax-ml.github.io
- Making Deep Learning go Brrrr From First Principleshorace.io
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- A Guide to AI Inference Engineering - ByteByteGo Newsletterblog.bytebytego.com
- A guide to LLM inference and performancebaseten.co
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- All the Transformer Math You Need to Know | How To Scale Your Modeljax-ml.github.io
- How to Think About TPUs | How To Scale Your Modeljax-ml.github.io