Rocky Slurm Basic Multi
About
In these examples, you'll see how to use SBATCH parameters to take advantage of multiple nodes and multiple cores on those nodes.
myscript.sh
#!/bin/bash H=`hostname -s` C=`ps -o cpuid -h -p ${BASHPID} | xargs` echo "${H} (${C})"
For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.
Multiple Nodes
slurm-test-nodes.sh
#!/bin/bash #SBATCH --job-name=test_nodes_job #SBATCH --nodes=3 #SBATCH --output=test_nodes_%j.log srun myscript.sh
Here you can see we are telling sbatch to use 3 nodes for this job.
sbatch slurm-test-nodes.sh
test_nodes_2954.log
moose1 (20) moose2 (20) rocky1 (40)
Here you can see from the output that the job used 3 nodes (moose1, moose2, and rocky1).
Multiple Tasks and Nodes
#!/bin/bash #SBATCH --job-name=test_tasksnodes_job #SBATCH --nodes=3 #SBATCH --ntasks=6 #SBATCH --output=log/test_%j.log srun myscript.sh
Here you can see that we've asked for 3 nodes and 6 total tasks.
moose1 (0) moose1 (20) moose2 (0) moose2 (20) rocky1 (0) rocky1 (40)
Here you can see that it broke the 6 tasks up amongst 3 nodes.
Many Tasks
#!/bin/bash #SBATCH --job-name=test_job # Job name #SBATCH --ntasks=100 #SBATCH --output=log/test_%j.log # Standard output and error log srun myscript.sh
Here you can see we've asked for 100 tasks. We have not asked for multiple nodes but this will be more tasks than can fit on a single node.
moose2 (6) moose2 (7) moose2 (27) moose1 (0) moose1 (1) moose1 (21) moose2 (26) moose1 (20) moose2 (8) moose2 (28) moose1 (19) moose2 (30) moose2 (14) moose2 (9) moose2 (29) moose2 (10) moose1 (28) moose1 (39) moose1 (30) moose2 (20) moose2 (34) moose2 (31) moose1 (26) moose1 (22) moose2 (25) moose2 (23) moose2 (0) moose1 (27) moose2 (3) moose1 (29) moose1 (9) moose2 (4) moose1 (34) moose1 (17) moose1 (6) moose2 (24) moose1 (7) moose2 (5) moose2 (12) moose1 (37) moose2 (1) moose2 (22) moose2 (2) moose2 (13) moose2 (33) moose2 (11) moose1 (3) moose1 (23) moose1 (11) moose1 (10) moose1 (14) moose2 (35) moose2 (18) moose1 (33) moose1 (2) moose1 (12) moose2 (17) moose2 (37) moose1 (8) moose2 (32) moose2 (21) moose1 (31) moose2 (15) moose1 (13) moose2 (16) moose2 (19) moose2 (39) moose1 (5) moose2 (38) moose2 (36) moose1 (32) moose1 (16) moose1 (25) moose1 (38) moose1 (4) moose1 (36) moose1 (15) moose1 (24) moose1 (18) moose1 (35) rocky1 (16) rocky1 (56) rocky1 (0) rocky1 (40) rocky1 (4) rocky1 (44) rocky1 (14) rocky1 (54) rocky1 (18) rocky1 (58) rocky1 (42) rocky1 (52) rocky1 (2) rocky1 (12) rocky1 (10) rocky1 (50) rocky1 (8) rocky1 (48) rocky1 (6) rocky1 (46)
Here you can see even though we did not ask for more than one node it has split the tasks up across multiple nodes.