Monitoring Your Work on Compute Nodes

Monitoring means identifying and inspecting Slurm jobs.

Identifying Your Slurm Jobs on a Cluster

Video Number

Video Name and Link

Description

1

How to identify slurm jobs (on compute nodes) via the command line

How to use Slurm commands squeue and scontrol show job --details.

2

TODO: How to identify Open OnDemand-based slurm jobs

For OOD jobs, how to get the slurm job ID.

Inspecting Your Slurm Jobs on a Cluster

From the above subsection, you have the Slurm Job ID for your job of interest.

You may also need the compute node names and GPU indexes (if your job uses GPUs), which can be obtained from scontrol show job --details <slurm_job_id> in the video above.

Video Number

Video Name and Link

Description

1

TODO: How to show details of your job

How to use showjobusage.

2

TODO: How to show all jobs on a compute node

How to use shownodeusage.

3

TODO: How to use Grafana dashboards to get job details

How to use Grafana.

4

TODO: How to zero-in on your job within Grafana dashboards

How to use getjobutilurl.