Before we jump to a job script, here are some basic stuff. |
Node: a (blade) server that has multiple cores (It's actually a network terminology, but that's fine. Tardis has 48 cores/node.) |
PBS: Portable batch system that arranges jobs between users |
Scratch disk: temporary disk space used to save raw outputs from MPI jobs |
$> qsub job.sh # submit a job $> qstat -nas # check a job status $> qdel JOB_ID # cancel a job $> qstat -u USER_ID # check job status of USER $> pbsnodes -aSjUSER_ID # check job status of USER |
#PBS -q workq #PBS -N MyJobName #PBS -S /bin/bash #PBS -l nodes=3:ppn=48 #PBS -l walltime=6:00:00 #PBS -V #NCORE=`cat $PBS_NODEFILE | wc -l` cd $PBS_O_WORKDIR # module setting module purge module load intel20/compiler-20 module load intel20/mvapich2-2.3.4 logfile=log$(date +%Y%m%d%H%M).out NCPU=144 mpirun -machinefile $PBS_NODEFILE -np ${NCPU} ./ramses3d params.nml > ${logfile} # More example scripts can be found in /storage7/pbs_examples/*.sh |
#PBS -l select=6:ncpus=32:mpiprocs=16 |
-> Occupy 6 nodes with 32 cores, but run only 16 processes/node.
This will be useful if you want to have a large mem per core.
Altogether, this will use 32x6=192 cores. | |
Suppose that 32 cores are being used in 6 nodes. In this case, 16 cores are available in each node, which means 16x6=96 cores are free.
If you want to use all of these 96 cores, you should submit with
#PBS -l select=2:ncpus=48:mpiprocs=48 |
#PBS -l select=4:ncpus=16:mpiprocs=4:ompthreads=4 |
-> Occupy 4 nodes with 16 cores and run 4 processes/node. But each process will use 4 OMP threads, making the total number of threads 64. |
© All rights reserved. | Design by TEMPLATED and modified by Lizzie