Kimi-K2.5 Inference Benchmark - Luminal
Luminal is an AI inference compiler that compiles and optimizes AI models for GPUs and ASICs, delivering the fastest, highest throughput inference in the world.
Kimi-K2.5 Inference Benchmark - Luminal Inference Benchmark moonshotai/Kimi-K2.5 on 8xH200 Model moonshotai/Kimi-K2.5 GPU 8xH200 Architecture 1T MoE, 32B active params Scenarios What Do the Scenarios Represent? Each benchmark scenario simulates a different real-world usage pattern with distinct input and output token profiles. Chatbot 1,024 in / 256 out A typical conversational turn: short context (prior messages plus a new prompt) and a moderate-length reply. This is the lightest workload and represents most chat-style applications. Prefill is fast, so TTFT is low and decode speed is at its p
saved by
related reading
- Real-time LLM Inference on Standard Datacenter GPUs (3,000 tokens/s per request)blog.kog.ai
- Compiling Models to Megakernels - by Luminal and Joe Fiotiblog.luminal.com
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Interaction Models: A Scalable Approach to Human-AI Collaboration - Thinking Machines Labthinkingmachines.ai
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- Kimi K2.5 Tech Blog: Visual Agentic Intelligencekimi.com
- LocalScore - Local AI Benchmarklocalscore.ai
- Optimizing inference · Hugging Facehuggingface.co
- Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
- Kimi K3 Tech Blog: Open Frontier Intelligencekimi.com