Guide to using clusters
This tutorial explains how to work with the Snellius cluster for the Deep Learning course at the University of Amsterdam. Every student will receive an account to have resources for training deep neural networks and get familiar with working on a cluster. It is recommended to have listened to the presentation by the SURF team or the TA team before going through this tutorial. Further, this tutorial assumes that you are familiar with using the terminal in Linux. If not, a crash course can be found here. The following section gives a quick introduction to the Snellius cluster, and how it is build up. A detailed description of the system can be found in the SURF user guide. (Disclaimer: the following paragraph is an adapted version of the HPC user guide. Credits: SURF team) You can imagine a cluster computer as a collection of regular computers (known as nodes), tied together with network cables that are similar to the network cables in your home or office (see the figure below - credit:
Authors: Phillip Lippe, Danilo de Goede This tutorial explains how to work with the Snellius cluster for the Deep Learning course at the University of Amsterdam. Every student will receive an account to have resources for training deep neural networks and get familiar with working on a cluster. It is recommended to have listened to the presentation by the SURF team or the TA team before going through this tutorial. Further, this tutorial assumes that you are familiar with using the terminal in Linux. If not, a crash course can be found here. The Snellius cluster The following section…
saved by
related reading
- SLURM Job Submission with R, Python, Bash | Research Computing Lessonsvsoch.github.io
- Slurm Workload Manager - Quick Start User Guideslurm.schedmd.com
- From bare metal to a 70B model: infrastructure set-up and scripts - Imbueimbue.com
- Together AI | The AI Native Cloudtogether.ai
- Autoresearchgivemeanode.com
- SLURM - ACCRE Wikihelp.accre.vanderbilt.edu
- Slurm Workload Manager - Wikipediaen.wikipedia.org
- Rent GPUs | Vast.aivast.ai
- ClusterMAX™ 2.0: The Industry Standard GPU Cloud Rating Systemnewsletter.semianalysis.com
- GitHub - jacobhilton/deep_learning_curriculum: Language model alignment-focused deep learning curriculumgithub.com
- Accelerate AI & Machine Learning Workflows | NVIDIA Run:airun.ai
- CodaLab Worksheetsworksheets.codalab.org