tutorials/README.md at main · triton-inference-server/tutorials · GitHub

This repository contains tutorials and examples for Triton Inference Server - tutorials/README.md at main · triton-inference-server/tutorials

Accelerating Inference for Deep Learning Models Navigate to Part 3: Optimizing Triton Configuration Part 5: Building Model Ensembles Model acceleration is a complex nuanced topic. The viability of techniques like graph optimizations for models, pruning, knowledge distillation, quantization, and more, highly depend on the structure of the model. Each of these topics are vast fields of research in their own right and building custom tools requires massive engineering investment. Rather than having an exhaustive outline of the ecosystem, for brevity and objectivity, this discussion will be focuse

related reading

• Accelerated Inference for Large Transformer Models Using NVIDIA Triton Inference Server | NVIDIA Technical Blogdeveloper.nvidia.com
• How To Scale Your Modeljax-ml.github.io
• Making Deep Learning go Brrrr From First Principleshorace.io
• tutorials/Conceptual_Guide/Part_1-model_deployment at main · triton-inference-server/tutorials · GitHubgithub.com
• PyTorch Inference | onnxruntimeonnxruntime.ai
• tutorials/HuggingFace at main · triton-inference-server/tutorials · GitHubgithub.com
• server/docs/user_guide/model_configuration.md at main · triton-inference-server/server · GitHubgithub.com
• GitHub - triton-inference-server/backend: Common source, scripts and utilities for creating Triton backends. · GitHubgithub.com
• Unlocking the full power of NVIDIA H100 GPUs for ML inference with TensorRTbaseten.co
• Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
• The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
• Best practices to accelerate inference for large-scale production workloadstogether.ai