Network and Storage Benchmarks for LLM Training on the Cloud | Henry Zhu

Personal website for some random tidbits I work on

AI usage has become universal. Teams everywhere are building RAG, generating embeddings, and training increasingly sophisticated agents. Most distributed LLM training guides focus on model architecture and hyperparameters while ignoring a critical bottleneck: infrastructure configuration. Network and storage choices often determine whether training takes hours or days. I ran benchmarks finetuning Gemma 3 12B and GPT-OSS-120B with different storage and network configurations using SkyPilot for infra and Nebius for GPUs. The results reveal that InfiniBand networking provides 10x faster training

saved by

• Dhruv Sheth

related reading

• Keep the Tokens Flowing: Lessons from 16 Open-Source RL Librarieshuggingface.co
• ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
• How to Rack 30 Petabytes of Storage | blogsi.inc
• LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
• Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructuresemianalysis.com
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• Look Ma, No Bubbles! Designing a Low-Latency Megakernel for Llama-1B · Hazy Researchhazyresearch.stanford.edu
• From bare metal to a 70B model: infrastructure set-up and scripts - Imbueimbue.com
• Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
• Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
• How to Rack 30 Petabytes of Storage | blogsi.inc
• Paged Attention from First Principles: A View Inside vLLM – Hamza's Bloghamzaelshafie.bearblog.dev