Rocky Slurm Basic Multi - Revision history

Jondale at 21:06, 25 August 2022

2022-08-25T21:06:48Z

← Older revision		Revision as of 21:06, 25 August 2022
Line 3:		Line 3:
	= About =		= About =

	In these examples, you'll see how to use SBATCH parameters to take advantage of multiple nodes and multiple cores on those nodes.		In these examples, you'll see how to use SBATCH parameters to take advantage of multiple nodes and multiple cores on those nodes. For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.

	= myscript.sh =		'''myscript.sh'''

	<pre>		<pre>
Line 16:		Line 16:
	</pre>		</pre>

	''For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.''

	= Multiple Nodes =		= Multiple Nodes =

Jondale: /* Many Tasks */

2022-08-25T21:05:16Z

Many Tasks

← Older revision		Revision as of 21:05, 25 August 2022
Line 81:		Line 81:
	srun myscript.sh		srun myscript.sh
	</pre>		</pre>
	~~Here you can see we~~'ve asked for 100 tasks. We have not asked for multiple nodes but this will be more tasks than can fit on a single node.		''We've asked for 100 tasks. We have not asked for multiple nodes but this will be more tasks than can fit on a single node.''


	<pre>		<pre>

Jondale: /* Multiple Nodes */

2022-08-25T21:04:42Z

Multiple Nodes

← Older revision		Revision as of 21:04, 25 August 2022
Line 39:		Line 39:


	~~'''test_nodes_2954.log'''~~
	<pre>		<pre>
	moose1 (20)		moose1 (20)

Jondale: /* Multiple Tasks and Nodes */

2022-08-25T21:04:24Z

Multiple Tasks and Nodes

← Older revision		Revision as of 21:04, 25 August 2022
Line 59:		Line 59:
	srun myscript.sh		srun myscript.sh
	</pre>		</pre>
	''~~Here you can see that we~~'ve asked for 3 nodes and 6 total tasks.''		''We've asked for 3 nodes and 6 total tasks.''

	<pre>		<pre>
Line 69:		Line 69:
	rocky1 (40)		rocky1 (40)
	</pre>		</pre>
	''~~Here you can see that~~ it broke the 6 tasks up amongst 3 nodes.''		''The log file shows it broke the 6 tasks up amongst 3 nodes.''

	= Many Tasks =		= Many Tasks =

Jondale: /* Multiple Tasks and Nodes */

2022-08-25T21:03:33Z

Multiple Tasks and Nodes

← Older revision		Revision as of 21:03, 25 August 2022
Line 59:		Line 59:
	srun myscript.sh		srun myscript.sh
	</pre>		</pre>
	Here you can see that we've asked for 3 nodes and 6 total tasks.		''Here you can see that we've asked for 3 nodes and 6 total tasks.''

	<pre>		<pre>
Line 69:		Line 69:
	rocky1 (40)		rocky1 (40)
	</pre>		</pre>
	Here you can see that it broke the 6 tasks up amongst 3 nodes.		''Here you can see that it broke the 6 tasks up amongst 3 nodes.''


	= Many Tasks =		= Many Tasks =

Jondale: /* Multiple Nodes */

2022-08-25T21:03:24Z

Multiple Nodes

← Older revision		Revision as of 21:03, 25 August 2022
Line 31:		Line 31:
	srun myscript.sh		srun myscript.sh
	</pre>		</pre>
	Here you can see we are telling sbatch to use 3 nodes for this job.		''Here you can see we are telling sbatch to use 3 nodes for this job.''


Line 45:		Line 45:
	rocky1 (40)		rocky1 (40)
	</pre>		</pre>
	Here you can see from the output that the job used 3 nodes (moose1, moose2, and rocky1).		''Here you can see from the output that the job used 3 nodes (moose1, moose2, and rocky1).''


	= Multiple Tasks and Nodes =		= Multiple Tasks and Nodes =

Jondale: /* myscript.sh */

2022-08-25T21:03:12Z

myscript.sh

← Older revision		Revision as of 21:03, 25 August 2022
Line 16:		Line 16:
	</pre>		</pre>

	For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.		''For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.''


	= Multiple Nodes =		= Multiple Nodes =

Jondale at 21:02, 25 August 2022

2022-08-25T21:02:59Z

← Older revision		Revision as of 21:02, 25 August 2022
Line 47:		Line 47:
	</pre>		</pre>
	Here you can see from the output that the job used 3 nodes (moose1, moose2, and rocky1).		Here you can see from the output that the job used 3 nodes (moose1, moose2, and rocky1).


			= Multiple Tasks and Nodes =

			<pre>
			#!/bin/bash

			#SBATCH --job-name=test_tasksnodes_job
			#SBATCH --nodes=3
			#SBATCH --ntasks=6
			#SBATCH --output=log/test_%j.log

			srun myscript.sh
			</pre>
			Here you can see that we've asked for 3 nodes and 6 total tasks.

			<pre>
			moose1 (0)
			moose1 (20)
			moose2 (0)
			moose2 (20)
			rocky1 (0)
			rocky1 (40)
			</pre>
			Here you can see that it broke the 6 tasks up amongst 3 nodes.


			= Many Tasks =

			<pre>
			#!/bin/bash

			#SBATCH --job-name=test_job # Job name
			#SBATCH --ntasks=100
			#SBATCH --output=log/test_%j.log # Standard output and error log

			srun myscript.sh
			</pre>
			Here you can see we've asked for 100 tasks. We have not asked for multiple nodes but this will be more tasks than can fit on a single node.

			<pre>
			moose2 (6)
			moose2 (7)
			moose2 (27)
			moose1 (0)
			moose1 (1)
			moose1 (21)
			moose2 (26)
			moose1 (20)
			moose2 (8)
			moose2 (28)
			moose1 (19)
			moose2 (30)
			moose2 (14)
			moose2 (9)
			moose2 (29)
			moose2 (10)
			moose1 (28)
			moose1 (39)
			moose1 (30)
			moose2 (20)
			moose2 (34)
			moose2 (31)
			moose1 (26)
			moose1 (22)
			moose2 (25)
			moose2 (23)
			moose2 (0)
			moose1 (27)
			moose2 (3)
			moose1 (29)
			moose1 (9)
			moose2 (4)
			moose1 (34)
			moose1 (17)
			moose1 (6)
			moose2 (24)
			moose1 (7)
			moose2 (5)
			moose2 (12)
			moose1 (37)
			moose2 (1)
			moose2 (22)
			moose2 (2)
			moose2 (13)
			moose2 (33)
			moose2 (11)
			moose1 (3)
			moose1 (23)
			moose1 (11)
			moose1 (10)
			moose1 (14)
			moose2 (35)
			moose2 (18)
			moose1 (33)
			moose1 (2)
			moose1 (12)
			moose2 (17)
			moose2 (37)
			moose1 (8)
			moose2 (32)
			moose2 (21)
			moose1 (31)
			moose2 (15)
			moose1 (13)
			moose2 (16)
			moose2 (19)
			moose2 (39)
			moose1 (5)
			moose2 (38)
			moose2 (36)
			moose1 (32)
			moose1 (16)
			moose1 (25)
			moose1 (38)
			moose1 (4)
			moose1 (36)
			moose1 (15)
			moose1 (24)
			moose1 (18)
			moose1 (35)
			rocky1 (16)
			rocky1 (56)
			rocky1 (0)
			rocky1 (40)
			rocky1 (4)
			rocky1 (44)
			rocky1 (14)
			rocky1 (54)
			rocky1 (18)
			rocky1 (58)
			rocky1 (42)
			rocky1 (52)
			rocky1 (2)
			rocky1 (12)
			rocky1 (10)
			rocky1 (50)
			rocky1 (8)
			rocky1 (48)
			rocky1 (6)
			rocky1 (46)
			</pre>

			''Here you can see even though we did not ask for more than one node it has split the tasks up across multiple nodes.''

Jondale at 20:56, 25 August 2022

2022-08-25T20:56:24Z

← Older revision		Revision as of 20:56, 25 August 2022
Line 1:		Line 1:
			__TOC__

	= About =		= About =

Line 15:		Line 17:

	For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.		For example purposes, we'll use a simple bash script. It's only purpose is to output the hostname of the node we're on and the cpuid the script is running. This will help illustrate that our script is being run on multiple nodes and/or multiple cores on each node.


			= Multiple Nodes =

			'''slurm-test-nodes.sh'''
			<pre>
			#!/bin/bash

			#SBATCH --job-name=test_nodes_job
			#SBATCH --nodes=3
			#SBATCH --output=test_nodes_%j.log


			srun myscript.sh
			</pre>
			Here you can see we are telling sbatch to use 3 nodes for this job.


			<pre>
			sbatch slurm-test-nodes.sh
			</pre>


			'''test_nodes_2954.log'''
			<pre>
			moose1 (20)
			moose2 (20)
			rocky1 (40)
			</pre>
			Here you can see from the output that the job used 3 nodes (moose1, moose2, and rocky1).

Jondale: /* About */

2022-08-25T20:51:32Z

About

← Older revision		Revision as of 20:51, 25 August 2022
Line 1:		Line 1:
	= About =		= About =

			In these examples, you'll see how to use SBATCH parameters to take advantage of multiple nodes and multiple cores on those nodes.

	= myscript.sh =		= myscript.sh =