tutorials/HuggingFace at main · triton-inference-server/tutorials

This repository contains tutorials and examples for Triton Inference Server - tutorials/HuggingFace at main · triton-inference-server/tutorials

Deploying HuggingFace models Note : If you are new to the Triton Inference Server, it is recommended to review Part 1 of the Conceptual Guide . This tutorial assumes basic understanding about the Triton Inference Server. Related Pages HuggingFace model exporting guide: ONNX , TorchScript Developers often work with open source models. HuggingFace is a popular source of many open source models. The discussion in this guide will focus on how a user can deploy almost any model from HuggingFace with the Triton Inference Server. For this example, the ViT model available on HuggingFace is being used.

related reading

• tutorials/Conceptual_Guide/Part_1-model_deployment at main · triton-inference-server/tutorials · GitHubgithub.com
• tutorials/Conceptual_Guide/Part_4-inference_acceleration/README.md at main · triton-inference-server/tutorials · GitHubgithub.com
• Accelerated Inference for Large Transformer Models Using NVIDIA Triton Inference Server | NVIDIA Technical Blogdeveloper.nvidia.com
• GitHub - triton-inference-server/backend: Common source, scripts and utilities for creating Triton backends. · GitHubgithub.com
• Hugging Face – The AI community building the future.huggingface.co
• Llama 2 · Hugging Facehuggingface.co
• GitHub - huggingface/nanotron: Minimalistic large language model 3D-parallelism training · GitHubgithub.com
• server/docs/user_guide/model_configuration.md at main · triton-inference-server/server · GitHubgithub.com
• pytorch-image-models/timm/models/vision_transformer.py at main · huggingface/pytorch-image-models · GitHubgithub.com
• Walkthrough - nnsightnnsight.net
• CI/CD for AI model deploymentsbaseten.co
• GitHub - wafer-ai/gpu-perf-engineering-resources: A curated resource list for learning AI performance engineering, from GPU fundamentals to production inference.github.com