Submitting Xander Jobs to MSU's HPCC

Since Fall 2018 the Simple Linux Utility for Resource Management (SLURM) system has been used to submit computing jobs to MSU’s high performance computing cluster (HPCC). This exercise demonstrates how to submit a Xander job using SLURM.

Exercise steps are:

Create a working directory.
Copy the configuration and command scripts into the working directory.
Edit the configuration script.
Create an sbatch file, named for example submit_xander.sb.
From the working directory, enter a run command of the form:

sbatch submit_xander.sb

Copy Scripts into Working Directory

1. Create a working directory in your home directory, e.g. ~/xander_batch

mkdir ~/xander_batch

2. Move into your working directory.

cd ~/xander_batch

3. Copy xander_setenv.sh to your working directory renaming it to my_xander_setenv.sh in the process.

cp /mnt/research/rdp/public/RDPTools/Xander_assembler/bin/xander_setenv.sh ~/xander_batch/my_xander_setenv.sh

4. Copy run_xander_skel.sh to your working directory, renaming it run_xander in the process.

cp /mnt/research/rdp/public/RDPTools/Xander_assembler/bin/run_xander_skel.sh ~/xander_batch/run_xander.sh

Edit the Configuration Script File

There are two lines in the configuration script that you need to change. With the editor of your choice, edit the path variable for your working directory in the First section of my_xander_setenv.sh to the absolute path, e.g ~/xander_qsub. Edit the number of threads to use in the Third section to THREADS=4. THREADS should be 4, the number of genes you will list in the last line of the batch file (below). After your edits, these two lines should look like this:

WORKDIR=~/xander_batch
THREADS=4

To make these changes using the editor nano, begin by entering the following:

nano my_xander_setenv.sh

Use the arrow keys to move the cursor to the appropriate place in the text. Insert characters by typing. Delete characters with the delete or backspace keys. There is a menu at the bottom of the nano screen indicating keys to use to write out the changed file and exit nano. The “^” in this menu means the “Ctrl” key. When you are finished making changes, hold down the Ctrl key and type o. You will be offered the opportunity to change the file name. To keep the same name, just hit the Enter key. Then hold down the Ctrl key and type x to exit nano. You may check that your changes have been made with less:

less my_xander_setenv.sh

Do Not Edit the Run Script File

You do not need to edit run_xander.sh. The path for BASEDIR is already set correctly for running this execise on MSU’s HPCC. But do make sure that it is executable. If not make it so:

ls -l
chmod 755 run_xander.sh
ls -l

Create the batch File

Using the editor of your choice, create a file named submit_xander.sb in your working directory. This can be done with nano if you wish. The contents of the file should be:

#!/bin/bash -login

#SBATCH --time=00:30:00     ### limit of wall clock time - how long the job will run (same as -t)
#SBATCH --ntasks=1          ### number of tasks - how many tasks (nodes) you require (same as -n)
#SBATCH --nodes=1           ### number of nodes required
#SBATCH --cpus-per-task=5   ### number of CPUs (or cores) per task (same as -c)
#SBATCH --mem=2G            ### memory required per node - amount of memory (in bytes)
#SBATCH --job-name xander   ### you can give your job a name for easier identification (same as -J)
#SBATCH --mail-type=BEGIN,END
#SBATCH --mail-user=your_email_address

export OMP_NUM_THREADS=5

cd ~/xander_batch
./run_xander.sh my_xander_setenv.sh "build find search" "nifH nirK rplB nosZ"

cpus-per-task and OMP_NUM_THREADS should be one more than the number of genes, one more than the number of THREADS in xander_setenv.sh. And of course, replace “your_email_address” with your own actual one.

Submit the Job

Submit the job to the queue with the command:

sbatch submit_xander.sb

The job should take about 5 minutes to run. You can check the status of the job with:

squeue -u your_user_name