Batch Processing

Flow of Job Execution

1. Creation of Job script
2. Job Submission
3. (If necessary) Confirmation of the job status
4. (If necessary) Job Cancellation

Job script

A job script is the same format as the shell script. A job script consists of an option area describing Slurm job submission options and a user program area describing the program to be executed. Please refer to here for environment variables that are automatically set when a job is executed.

Example of hybrid parallelism with 4 processes x 8 threads

An example of a job script is shown below, so you can see the script to keep in perspective. More detailed information for each is given in the next section.

#!/bin/bash
#============ Slurm Options ===========
#SBATCH -p gr19999b　# Specify the job queue (partition). It must be changed to the queue name which you want to submit.
#SBATCH -t 1:0:0  # Specify the elapsed time (example of specifying 1 hour)
#SBATCH --rsc p=4:t=8:c=8:m=8G　# Specify requested resources
#SBATCH -o %x.%j.out　# Specify the standard output file for the job
## %x is replaced by the job name and %j is replaced by the job ID.
#============ Shell Script ============

# (Optional) Specify set -x to keep track of the execution progress of the job script.
set -x

# Environment variables such as the number of MPI processes and OMP_NUM_THREADS are automatically set based on the value of the --rsc option.
# If necessary, it is possible to specify overwrite within the range of resources allocated by the srun command argument or environment variables.
srun ./a.out

## Notes on job scripts ##
# Lines beginning with '#' or after '#' in a line are treated as comments. Only lines beginning with #SBATCH are exceptionally recognised as slurm option specifications.
# The current directory at the job execution is automatically moved to the current directory at the job submission.
# Environment variables set at the job submission are inherited during job execution.

Sample script

A sample job script is available for your reference.

Slurm Option

Specify in the Slurm Options part of the job script followed by "#SBATCH".

Major options

Please refer to the Official Manual for other options and details on options. Also, please check Options not available when submitting jobs if necessary.

How to execute the program (important)

In order to execute a program on a computing node, you must use the srun command at the point where you execute the program of the job script whether it is a sequential program or an MPI program.

The following is a typical list of options for the srun command. Please refer to the Official Manual for other options and details on options.

Basic Options

Confirmation of available queues

To check the queue of jobs available for submission, use the spartition command.

spartition command

Displays the queue name, Rmin/Rstd/Rmax, and standard and maximum elapsed time values for jobs that can be submitted. For more information on Rmin/Rstd/Rmax

$ spartition
Partition  State   Rmin  Rstd  Rmax    DefTime     MaxTime
gr19999g   UP         0    64    64   01:00:00  1-00:00:00

Job Submission

To submit a job to the queue, use the sbatch command.

sbatch Command

$ sbatch sample.sh
Submitted batch job 20

• Enter the job script file you created followed by the end of the command.
• The execution of the job script is requested to the system and the job ID is displayed.
• If option is specified when a job is submitted, the option in the job script will be overwritten.

Message when a job is submitted

$ sbatch sample.sh
sbatch: cli_filter/accms_resource_req: convert_rsc_option: Updated the number of cores from c=10 to c=40 based on memory size request
Submitted batch job 21

The message above is that the number of cores has been changed based on the requested memory size. This is not an error and the job will be submitted.

Confirmation of the job status

To display information on submitted jobs, use the squeue or sacct command.

squeue Command

Displays information about jobs currently registered in the queue.

$ squeue
JOBID PARTITION     NAME     USER ST   TIME  NODES NODELIST(REASON)
    1  gr19999b interact   b59999  R   0:33      1 no0001

• Information is displayed only during job execution.
• Please refer to the Official Manual for options.

Notes on job execution(REASON)

If a job is waiting for execution, the reason is indicated in NODELIST(REASON). Typical examples are shown below. For other reasons, please refer to Official Manual for more information.

sacct Command

Displays information about jobs in the accounting database. Information about past jobs can also be displayed. Please refrain from repeating the command mechanically because it overloads the system.

$ sacct -X
JobID           JobName  Partition    Account  AllocCPUS      State ExitCode
------------ ---------- ---------- ---------- ---------- ---------- --------
1               test.sh   gr19999b     b59999          8  COMPLETED      0:0
2               test.sh   gr19999b     b59999          8  COMPLETED      0:0

• By default, a list of jobs executed on the day the command was issued is displayed.

Main options

For other options, please refer to the Official Manual.

Job Cancellation

To cancel a submitted job, use the scancel command.

scancel Command

$ scancel 20

• Specify the job ID as an argument.
• Please refer to the Official Manual for options.

How to cancel all submitted jobs

You can cancel all submitted jobs with an instruction below.

## How to delete by specifying a user name.
$ scancel -u b59999

## How to delete by specifying a queue name. 
$ scancel -p gr19999b

## How to delete by specifying a running status.
$ scancel -t pending