StarCCM+
Specifying nodes and processes per node on Slurm systems
This is in a job which had “nodes=3” and “ntasks-per-node=2”:
[brownm12@ca028 ~]$ STARNL=$(srun hostname | sort -u | awk -v NT=$SLURM_NTASKS_PER_NODE '{printf $1":"NT","}' | sed s/.$//)
[brownm12@ca028 ~]$ echo $STARNL
ca028:2,ca099:2,ca100:2
Breakdown:
STARNL=$( ... )
set bash variable STARNL to the output of the command enclosed in parentheses:cmd1 | cmd2 | cmd3
send output ofcmd1
as input tocmd2
and the resulting output of that tocmd3
(pipes)srun hostname
runSLURM_NTASKS
instances ofhostname
command, one on each task (process) allocated to the jobsort -u
sort the input and remove duplicate linesawk -v NT=$SLURM_NTASKS_PER_NODE '{ ... }'
run an awk script on the input, additionaly set theawk
variableNT
to a valueprintf $1":"NT","
awk script to print the first column of each line of input, followed by a colon, the value stored inNT
, and then a commased s/.$//
remove the last character of the input line usingsed
’s “swap” capability
Example usage in a script
So in your job script, prior to the starccm+ command, you would have:
STARNL=$(srun hostname | sort -u | awk -v NT=$SLURM_NTASKS_PER_NODE '{printf $1":"NT","}' | sed s/.$//)
echo "Using nodelist: `echo $STARNL`"
So an example job script would have:
#!/bin/bash
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=6
module load starccm+
# Get hostlist and number of processes for each host and save in a variable:
STARNL=$(srun hostname | sort -u | awk -v NT=$SLURM_NTASKS_PER_NODE '{printf $1":"NT","}' | sed s/.$//)
# Print value of the variable for diagnostics
echo "Using nodelist: `echo $STARNL`"
starccm+ <your options and settings> -np $SLURM_NTASKS -on $STARNL <simfile>