Crash in NCCL when running with distributed GPU · Issue #13559 · google/jax
Description (could be a cluster issue, but unclear. This is on an HPC where I don't have sudo, and there are a lot of CUDA installations floating around, so I haven't ruled out that it's not some d...
Crash in NCCL when running with distributed GPU · Issue #13559 · jax-ml/jax · GitHub Skip to content You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} Uh oh! There was an error while loading. Please reload this page . jax-ml / jax Public Notifications You must be signed in to change notification settings Fork 3.7k Star 36k Crash in NCCL when running with distributed GPU #13559 New issue Copy
related reading
- How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance: a Worklogsiboehm.com
- What happens when you run a CUDA kernelfergusfinn.com
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- GitHub - jax-ml/jax: Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and moregithub.com
- Massively Scale Your Deep Learning Training with NCCL 2.4 | NVIDIA Technical Blogdeveloper.nvidia.com
- Strangely, Matrix Multiplications on GPUs Run Faster When Given "Predictable" Data! [short]thonking.ai
- Add GPU support to ggml · ggml-org/llama.cpp · Discussion #915 · GitHubgithub.com
- How to think in JAX — JAX documentationdocs.jax.dev
- GitHub - wangzyon/NVIDIA_SGEMM_PRACTICE: Step-by-step optimization of CUDA SGEMMgithub.com
- irhum.github.io - Tensor Parallelism with jax.pjitirhum.github.io
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io