SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost
This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. spacing=nonfrench AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3 – 10 × lower cost than on-demand instances, but their unpredictable availability makes meeting deadlines difficult. Existing systems either rely solely on spot instances and risk deadline violations, or operate in simplified single-region settings. These approaches overlook substantial spatial and temporal heterogeneity in spot availability, lifetimes, and prices. We show that exploiting such heterogeneity to access more spot capacity is the key to reduce the job execution cost. We present SkyNomad , a multi-region scheduling system that maximizes spot usage
spacing=nonfrench SkyNomad: On Using Multi-Region Spot Instances to Minimize AI Batch Job Cost Zhifei Li ∗† , Tian Xia ∗† , Ziming Mao † , Zihan Zhou ¶ , Ethan J. Jackson † , Jamison Kerney † , Zhanghao Wu † , Pratik Mishra § , Yi Xu † , Yifan Qiao † , Scott Shenker †,⋄ , Ion Stoica † † UC Berkeley ¶ Shanghai Jiao Tong University § AMD ⋄ ICSI Abstract AI batch jobs such as model training, inference pipelines, and data analytics require substantial GPU resources and often need to finish before a deadline. Spot instances offer 3 – 10 × 3\text{--}10\times lower cost than on-demand instances, but
saved by
related reading
- Fast Inference from Transformers via Speculative Decodingarxiv.org
- IsoCompute Playbook: Optimally Scaling Sampling Compute for RL Training of LLMscompute-optimal-rl-llm-scaling.github.io
- Network and Storage Benchmarks for LLM Training on the Cloud | Henry Zhumaknee.github.io
- Parallel CPU-GPU Execution for LLM Inference on Constrained GPUsarxiv.org
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- 2502.11089arxiv.org
- [2402.12374] Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingarxiv.org
- Paged Attention from First Principles: A View Inside vLLM – Hamza's Bloghamzaelshafie.bearblog.dev
- 2410.21680arxiv.org
- All About Transformer Inferencejax-ml.github.io
- LLM Engineer's Almanac - Workloads | Modalmodal.com
- [2509.14786] Pre-training under infinite computearxiv.org