Submitting Jobs Without Qsub Scripts

When you submit a job with qsub, you would normally put a few lines of PBS directives followed by your commands in a job submission script say, job.qsub, and then type qsub job.qsub on the command line. However, if you want to submit multiple jobs in parallel, and they are only slightly different from each other, creating as many qsub scripts as there are jobs can be frustrating.  

In the example below, we used seqtk to randomly sample 1% of the reads from paired-end fastq files. We tried two different seeds (-s100 and -s500) and, to keep pairing, the seed was the same for read files 1 and 2. The way each of these four jobs is submitted is by echo-ing (echo) and piping (|) commands to qsub, without needing a script to contain them.

echo "module load seqtk/1.0; seqtk sample -s100 /mnt/research/common-data/Examples/RNASeq-Model/data/ERR315325_1.fastq 0.01 > ~/out1_r1.fq" | qsub -l walltime=00:30:00,mem=50mb -N out1_r1

echo "module load seqtk/1.0; seqtk sample -s100 /mnt/research/common-data/Examples/RNASeq-Model/data/ERR315325_2.fastq 0.01 > ~/out1_r2.fq" | qsub -l walltime=00:30:00,mem=50mb -N out1_r2

echo "module load seqtk/1.0; seqtk sample -s500 /mnt/research/common-data/Examples/RNASeq-Model/data/ERR315325_1.fastq 0.01 > ~/out2_r1.fq" | qsub -l walltime=00:30:00,mem=50mb -N out2_r1

echo "module load seqtk/1.0; seqtk sample -s500 /mnt/research/common-data/Examples/RNASeq-Model/data/ERR315325_2.fastq 0.01 > ~/out2_r2.fq" | qsub -l walltime=00:30:00,mem=50mb -N out2_r2

It should be noted that, in the above examples, each job is as simple as a one-liner command. For larger jobs, writing a qsub script in the usual way would still be preferable.

More information about Singularity and how to use it can be found on the HPCC Wiki (Singularity Introduction - MSU HPCC User Documentation).