Training LLMs with AMD MI250 GPUs and MosaicML
With the release of PyTorch 2.0 and ROCm 5.4, we are excited to announce that LLM training works out of the box on AMD datacenter GPUs, with zero code changes, and at high performance (144 TFLOP/s/GPU)! We are thrilled to see promising alternative options for AI hardware, and look forward to evaluating future devices and larger clusters soon.
Training LLMs with AMD MI250 GPUs and MosaicML | Databricks Blog Skip to main content With the release of PyTorch 2.0 and ROCm 5.4, we are excited to announce that LLM training works out of the box on AMD MI250 accelerators with zero code changes and at high performance! With MosaicML, the AI community has additional hardware + software options to choose from. At MosaicML, we've searched high and low for new ML training hardware on behalf of our customers. We do this to increase compute availability (as the world is in an NVIDIA supply crunch!), expand and educate the market, and ultimately re
saved by
related reading
- LLM Inference Performance Engineering: Best Practices | Databricks Blogdatabricks.com
- Mosaic LLMs: GPT-3 quality formosaicml.com
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- ⭐️ Fast LLM Inference From Scratchandrewkchan.dev
- PiTorch: ML on Baremetal Raspberry Pis | projectsmasonjwang.com
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- Training great LLMs entirely from ground up in the wilderness as a startup - Yi Tayyitay.net
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- AI Chip Architecturesjacobpeake.com
- Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructuresemianalysis.com
- 1.5x faster MoE training with custom MXFP8 kernels · Cursorcursor.com
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io