Simple PBS Directives

If you don't like the names of PBS output and error files, if you don't want your job to go to the default queue, if you have any other special requests, you can specify them by the means of qsub command line switches or PBS directives.

PBS directives are exactly the same as the qsub command line switches, but they are passed to PBS  through the script comment lines. For example, in order to change the name of the output file you can use

qsub -o file_name

or you can insert

#PBS -o file_name

in the preamble of the script. The effect will be the same.

The three letters, PBS, that mark  the directive can be altered too, with the command line switch

-C new_string

But it's better not to tinker with it, since it may lead to unnecessary confusion down the road.

To rename the error  file use the directive:

#PBS -e file_name

The default name of the job, which shows in the second column of the qstat listing, is the name of the submitted file. If you enter the job interactively, then the name evaluates to STDOUT. Interactive job submission  is not advisable, because mistakes are easy to make, so I haven't talked about it, but if you're curious, this is how it is done:

[gustav@bh1 PBS]$ qsub
hostname
date
exit 0
^D
12672.bh1.avidd.iu.edu
[gustav@bh1 PBS]$

The output files are then called STDIN.o12672 and STDIN.e12672:

[gustav@bh1 PBS]$ ls
STDIN.e12672  STDIN.o12672  bc.sh  job.sh  xterm.sh
[gustav@bh1 PBS]$ cat STDIN.o12672
bc68
Mon Sep  8 21:36:52 EST 2003
[gustav@bh1 PBS]$

You can change the name of the job with the directive

#PBS -N job_name

The submitted script is interpreted using the user's login shell by default. PBS does not read the

#!/bin/bash

line the scripts usually begin with. But this can be altered by using the directive

#PBS -S /bin/bash

You  can use also perl or any other scripting language as long as its comment character is the sharp sign, # (some Schemes allow for the sharp to be used as a comment character too). PBS will scan all the initial comment lines for PBS directives until the first executable line. Any directives embedded in the script after some executable lines will be ignored.

If you  don't specify the queue name, the job gets submitted to the default queue. The default queue at most institutions is a short job queue. So normally you have to specify the queue name for longer jobs. This is done with the directive

#PBS -q bg

Let us wrap up the -o, -e, -N, -S and -q directives into the following script.

[gustav@bh1 PBS]$ cat simple.sh
#PBS -S /bin/bash
#PBS -N host
#PBS -o host_out
#PBS -e host_err
#PBS -q bg
hostname
date
exit 0
[gustav@bh1 PBS]$ qsub simple.sh
12673.bh1.avidd.iu.edu
[gustav@bh1 PBS]$ ls
STDIN.e12672  bc.sh     host_out  simple.sh
STDIN.o12672  host_err  job.sh    xterm.sh
[gustav@bh1 PBS]$ cat host_out
bc68
Mon Sep  8 21:58:06 EST 2003
[gustav@bh1 PBS]$

Another two useful PBS directives result in e-mail being sent to you when the job starts, or terminates or is aborted. The corresponding option is -m and  it must be followed by

n

in which case no mail is sent

a

in which case mail is sent when the job gets aborted

b

in which case mail is sent when the job begins execution

e

in which case mail is sent when the job terminates execution

Option a is the default. a, b and e can be combined, e.g., abe. The corresponding PBS directive looks as follows:

#PBS -m abe

Now, who should the mail be sent to? This  you can specify by using the -M directive, e.g.,

#PBS -M gustav@indiana.edu

You can also send mail to several users, e.g.,

#PBS -M gustav@indiana.edu,stewart@iu.edu

Let us see how this works in combination with our simple job described above.

[gustav@bh1 PBS]$ cat simple.sh
#PBS -S /bin/bash
#PBS -N host
#PBS -o host_out
#PBS -e host_err
#PBS -q bg
#PBS -m abe
#PBS -M gustav@indiana.edu
hostname
date
exit 0
[gustav@bh1 PBS]$ qsub simple.sh
12866.bh1.avidd.iu.edu
[gustav@bh1 PBS]$ ls
bc.sh  host_err  host_out  job.sh  nodes.sh  simple.sh  xterm.sh
[gustav@bh1 PBS]$ cat host_out
bc88
Wed Sep 10 16:15:46 EST 2003
[gustav@bh1 PBS]$

Mail sent to gustav@indiana.edu is automatically redirected to my workstation, woodlands.tqc.iu.edu. Let us have a look then if there are any messages there sent from AVIDD. Here is the updated mail listing from my emacs session:

   1  10-Sep  escience-admin@cs.indiana [108] \
      [Escience] Computer's ability to verify proof is an illusion
   2- 10-Sep  to: qc@WOODLANDS.tqc.iu.e  [62] \
      Quantum dynamics of a single vortex
   3- 10-Sep   adm@bh1.uits.indiana.edu  [36] \
      PBS JOB 12866.bh1.avidd.iu.edu
   4- 10-Sep   adm@bh1.uits.indiana.edu  [31] \
      PBS JOB 12866.bh1.avidd.iu.edu

Two messages have arrived from bh1 indeed. The messages may arrive out of order, because this is how e-mail usually works. In this case the second message (number 4 in the emacs listing) corresponds to the job beginning its execution:

Envelope-to: gustav@woodlands.tqc.iu.edu
Delivery-date: Wed, 10 Sep 2003 16:16:19 -0500
Date: Wed, 10 Sep 2003 16:15:46 -0500
From: adm <adm@bh1.uits.indiana.edu>
To: gustav@indiana.edu
Subject: PBS JOB 12866.bh1.avidd.iu.edu

PBS Job Id: 12866.bh1.avidd.iu.edu
Job Name:   host
Begun execution

and the first message corresponds to the termination of the job:

Envelope-to: gustav@woodlands.tqc.iu.edu
Delivery-date: Wed, 10 Sep 2003 16:16:16 -0500
Date: Wed, 10 Sep 2003 16:15:46 -0500
From: adm <adm@bh1.uits.indiana.edu>
To: gustav@indiana.edu
Subject: PBS JOB 12866.bh1.avidd.iu.edu

PBS Job Id: 12866.bh1.avidd.iu.edu
Job Name:   host
Execution terminated
Exit_status=0
resources_used.cput=00:00:00
resources_used.mem=0kb
resources_used.vmem=0kb
resources_used.walltime=00:00:00

A good idea is to have the message about your job having been aborted or terminated sent to your cellular phone, if you have this kind of service. This way the message will reach you wherever you may be.