mohit on X: "Day 2/30 of Inference Engineering - looked into where and why different CUDA Graph creation approaches are used, how this differs across PyTorch, JAX/TensorFlow and CUDA C++, and when stream capture vs explicit graph construction makes more sense - read about CUDA Graph https://t.co/OrNehU2cDA" / X
Day 2/30 of Inference Engineering - looked into where and why different CUDA Graph creation approaches are used, how this differs across PyTorch, JAX/TensorFlow and CUDA C++, and when stream capture vs explicit graph construction makes more sense - read about CUDA Graph
Day 2/30 of Inference Engineering - looked into where and why different CUDA Graph creation approaches are used, how this differs across PyTorch, JAX/TensorFlow and CUDA C++, and when stream capture vs explicit graph construction makes more sense - read about CUDA Graph constraints and limitations, including asynchronous/capture restrictions, static graph topology and parameters, memory requirements, multi-device considerations, and more - went through safe vs unsafe capture and explored some advanced CUDA Graph concepts like graph updates, device-side graph launch, and conditional nodes,…
saved by
related reading
- GitHub - wafer-ai/gpu-perf-engineering-resources: A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.github.com
- GitHub - gpu-mode/resource-stream: GPU programming related news and material linksgithub.com
- README | GPU Glossarymodal.com
- CUDA Programming Guide — CUDA Programming Guidedocs.nvidia.com
- PyTorch internals : ezyang's blogblog.ezyang.com
- CUDA C++ Programming Guide (Legacy) — CUDA C++ Programming Guidedocs.nvidia.com
- Together AI | The AI Native Cloudtogether.ai
- Homehao-ai-lab.github.io
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- Course on CUDA Programmingpeople.maths.ox.ac.uk
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- UW PLSE | How does torch.compile work?uwplse.org