flâneur

SLURM Job Submission

vsoch.github.io · 3,431 words · saved by 1 readers

You just finished up a really cool analysis, and you need to scale it. An HPC cluster with a job manager such as SLURM is a great way to do this! In this tutorial, we will walk through a very simple method to do this. First, let’s talk about our strategy for today. We will cover each of these steps in detail. Permalink Edit this page Ask a Question You first have some script in R or Python. It likely reads in data, processes it, and creates a result. You will need to turn this script into an executable, meaning that it accepts variable arguments. Permalink Edit this page Ask a Question R actually makes this very easy. While there are advanced input parsers, you can retrieve your script inputs with just a few lines: if I saved this in a script called “run.R” I could then execute: and input1 would be assigned to “tomato,” and “potato” and “shiabato” to input2 and input3, respectively. By the way, if you aren’t familiar with Rscript, it’s literally the R script executable. We are going to

SLURM Job Submission with R, Python, Bash | Research Computing Lessons SLURM Job Submission with R, Python, Bash In this tutorial we will write a job submission script for SLURM. If you haven’t yet, you should: be comfortable accessing the Sherlock cluster understand SLURM job submission and then move forward with this tutorial! Scenario You just finished up a really cool analysis, and you need to scale it. An HPC cluster with a job manager such as SLURM is a great way to do this! In this tutorial, we will walk through a very simple method to do this. First, let’s talk about our strategy for t

saved by

related reading