|Description:|| High Performance Computing job farming|
|Last Edit:|| 22/02/18|
Copyright © 2014 Richard J. Edwards - See source code for GNU License Notice
See SLiMSuite Blog for further documentation. See
rje for general commands.
This module contains the generic code for farming jobs out over multiple processors and/or nodes. It is based on
rje_iridis Version 1.10, updated to the new RJE_Object structure and made more generic. The recommended tool for
actual job farming is SLiMFarmer, which inherits the rje_hpc.JobFarmer object and adds extra bells and whistles.
Generic HPC Job Farming pptions
subsleep=X : Sleep time (seconds) between cycles of subbing out jobs to hosts [
subjobs=LIST : List of subjobs to farm out to HPC cluster 
hpcmode=X : Mode to be used for farming jobs between nodes (rsh/fork) [
iolimit=X : Limit of number of IOErrors before termination [
memfree=X : Min. proportion of node memory to be free before spawning job [
keepfree=X : Number of processors to keep free on head node [
SLiMSuite and SeqSuite SeqBySeq program options
seqbyseq=T/F : Activate seqbyseq mode - assumes
basefile=X option used for output [
farm=X : Program to farm out using seqbyseq mode. 
seqin=FILE : Input sequence file to farm out [
basefile=X : Base for output files - compiled from individual run results [
outlist=LIST : List of extensions of outputs to add to basefile for output (basefile.*) 
pickhead=X : Header to extract from OutList file and used to populate AccNum to skip 
startfrom=X : Sequence ID at which to begin the SeqBySeq farming [
rjepy=T/F : Whether program is an RJE *.py script (adds log processing) [
jobini=FILE : Ini file to pass to the called program [
pypath=PATH : Path to python modules [
See also rje.py generic commandline options.
History Module Version History
# 1.0 - Initial Compilation based on rje_iridis V1.10.
# 1.1 - Disabled memory checking in Windows and OSX.
# 1.1.1 - Added output of subjob command to log as run.