tutorials/README.md at main · triton-inference-server/tutorials · GitHub

This repository contains tutorials and examples for Triton Inference Server - tutorials/README.md at main · triton-inference-server/tutorials

Accelerating Inference for Deep Learning Models Navigate to Part 3: Optimizing Triton Configuration Part 5: Building Model Ensembles Model acceleration is a complex nuanced topic. The viability of techniques like graph optimizations for models, pruning, knowledge distillation, quantization, and more, highly depend on the structure of the model. Each of these topics are vast fields of research in their own right and building custom tools requires massive engineering investment. Rather than having an exhaustive outline of the ecosystem, for brevity and objectivity, this discussion will be focuse

related reading

• GitHub - triton-inference-server/backend: Common source, scripts and utilities for creating Triton backends. · GitHubgithub.com
• tutorials/Conceptual_Guide/Part_1-model_deployment at main · triton-inference-server/tutorials · GitHubgithub.com
• tutorials/HuggingFace at main · triton-inference-server/tutorials · GitHubgithub.com
• Accelerated Inference for Large Transformer Models Using NVIDIA Triton Inference Server | NVIDIA Technical Blogdeveloper.nvidia.com
• server/docs/user_guide/model_configuration.md at main · triton-inference-server/server · GitHubgithub.com
• Making Deep Learning go Brrrr From First Principleshorace.io
• The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
• PyTorch Inference | onnxruntimeonnxruntime.ai
• Transformer Inference Arithmetic | kipply's blogkipp.ly
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• GitHub - triton-lang/triton: Development repository for the Triton language and compiler · GitHubgithub.com
• Best practices to accelerate inference for large-scale production workloadstogether.ai