Difference between revisions of "Rocky User Guide"

From NIMBioS
 
(19 intermediate revisions by 2 users not shown)
Line 13: Line 13:
Once your account is created you will be able to SSH into a shell or SCP to copy files to/from Rocky.   
Once your account is created you will be able to SSH into a shell or SCP to copy files to/from Rocky.   


Rocky uses Public Key Authentication for access instead of passwords.  Please review the following pages about accessing Rocky:
Rocky uses Public Key Authentication for access instead of passwords.   
 
Please review the following pages for OS specific instructions:


{| class='wikitable'
{| class='wikitable'
Line 24: Line 26:


== Environmental Modules ==
== Environmental Modules ==
Rocky uses [https://lmod.readthedocs.io/en/latest/ Lmod] as it's environmental module system.  This allows you to set your environment up for specific development tasks.
Rocky uses [https://lmod.readthedocs.io/en/latest/ Lmod] as it's environmental module system.  This allows you to easily set your session or job's environment to support the language, libraries, and specific versions needed.
 
To learn more about using Lmod on Rocky, check out [[ Rocky_Environments | Rocky Environments]].
 
 
== Submitting a Job ==
 
Rocky uses [https://slurm.schedmd.com/documentation.html Slurm] to queue and submit jobs to the cluster's compute nodes. 
 
The nature of a compute cluster is that jobs are meant to be submitted to a queue.  Jobs are processed based on a [https://slurm.schedmd.com/classic_fair_share.html Fairshare] priority system.  Your job's priority is based on the associations you have (such as your lab) and how much compute you and they have used recently.  Sometimes your jobs might run instantly but other times you might have a wait time before they process. 
 
To learn how to set up your own jobs, check out the [[Rocky_Job_Anatomy | Anatomy of a Rocky Job]]
 
= Examples =
 
Beyond the below examples, we have also started a [https://github.com/rocky-cluster Github Repository].
 
==== Python ====
* [[ Rocky_Python_HelloWorld | Hello World ]]
* [[ Rocky_Python_Prime | Discover Prime Numbers ]]
* [[ Rocky_Python_Prime_Array | Discover Prime Numbers using a job array ]]
 
==== R ====
* [[ Rocky_R_HelloWorld | Hello World ]]
* [[ Rocky_R_Prime | Discover Prime Numbers ]]
* [[ Rocky_R_package | Install packages locally ]]


To use Lmod, you will use the module command.  For example, to see what modules are available:
==== MATLAB ====
<pre>module avail</pre>
* [[ Rocky_MATLAB_HelloWorld | Hello World ]]
* [[ Rocky_MATLAB_Prime_Array | Discover Prime Numbers using a job array ]]


To load the R environment:
<pre>module load R</pre>


To list what modules are loaded:
= Acknowledgement and Citations =
<pre>module list</pre>


If any work on Rocky is used in a research report, journal, or publication that requires citation of authors' work, please acknowledge NIMBioS. 


== Workload Management ==
Our suggested acknowledgment is as follows.
Rocky uses Slurm as it's workload manager.


* [[Rocky_Slurm | Basics of using Slurm on Rocky]]
'''[A portion of] The computation for this work was performed on the National Institute for Modeling Biological Systems (NIMBioS) computational resources at the University of Tennessee.'''
* [https://slurm.schedmd.com/ Official Slurm Documentation]

Latest revision as of 20:13, 28 February 2024

About Rocky

Rocky is a HPC cluster comprised of compute heavy nodes with 40 cores/80 threads and 512GB of ram [rocky], memory intensive nodes with 20 cores/40 threads and 768GB of RAM [moose], and a Ceph storage subsystem [quarrel]


Requesting Access

In order to gain access to Rocky you must first fill out the Rocky_Access_Form.


Logging in to Rocky

Rocky's firewall limits access to the UTK network. You will either need to be on campus or using the Campus VPN

Once your account is created you will be able to SSH into a shell or SCP to copy files to/from Rocky.

Rocky uses Public Key Authentication for access instead of passwords.

Please review the following pages for OS specific instructions:

Rocky_Access_SSH Linux or Mac
Rocky_Access_Windows Windows


Environmental Modules

Rocky uses Lmod as it's environmental module system. This allows you to easily set your session or job's environment to support the language, libraries, and specific versions needed.

To learn more about using Lmod on Rocky, check out Rocky Environments.


Submitting a Job

Rocky uses Slurm to queue and submit jobs to the cluster's compute nodes.

The nature of a compute cluster is that jobs are meant to be submitted to a queue. Jobs are processed based on a Fairshare priority system. Your job's priority is based on the associations you have (such as your lab) and how much compute you and they have used recently. Sometimes your jobs might run instantly but other times you might have a wait time before they process.

To learn how to set up your own jobs, check out the Anatomy of a Rocky Job

Examples

Beyond the below examples, we have also started a Github Repository.

Python

R

MATLAB


Acknowledgement and Citations

If any work on Rocky is used in a research report, journal, or publication that requires citation of authors' work, please acknowledge NIMBioS.

Our suggested acknowledgment is as follows.

[A portion of] The computation for this work was performed on the National Institute for Modeling Biological Systems (NIMBioS) computational resources at the University of Tennessee.