tutorials/HuggingFace at main · triton-inference-server/tutorials

This repository contains tutorials and examples for Triton Inference Server - tutorials/HuggingFace at main · triton-inference-server/tutorials

Deploying HuggingFace models Note : If you are new to the Triton Inference Server, it is recommended to review Part 1 of the Conceptual Guide . This tutorial assumes basic understanding about the Triton Inference Server. Related Pages HuggingFace model exporting guide: ONNX , TorchScript Developers often work with open source models. HuggingFace is a popular source of many open source models. The discussion in this guide will focus on how a user can deploy almost any model from HuggingFace with the Triton Inference Server. For this example, the ViT model available on HuggingFace is being used.

related reading

• tutorials/Conceptual_Guide/Part_1-model_deployment at main · triton-inference-server/tutorials · GitHubgithub.com
• Replicate - Run AI with an APIreplicate.com
• Accelerated Inference for Large Transformer Models Using NVIDIA Triton Inference Server | NVIDIA Technical Blogdeveloper.nvidia.com
• tutorials/Conceptual_Guide/Part_4-inference_acceleration/README.md at main · triton-inference-server/tutorials · GitHubgithub.com
• Hugging Face · GitHubgithub.com
• API Reference — TensorRT LLMnvidia.github.io
• PyTorch Inference | onnxruntimeonnxruntime.ai
• GitHub - triton-inference-server/backend: Common source, scripts and utilities for creating Triton backends. · GitHubgithub.com
• Interpretability Infrastructure at Frontier Scale: Harvesting Activations from a Trillion-Parameter Modelgoodfire.ai
• Introduction · Hugging Facehuggingface.co
• Inference Platform: Deploy AI models in production | Basetenbaseten.co
• server/docs/user_guide/model_configuration.md at main · triton-inference-server/server · GitHubgithub.com