SLiMSuite REST Server


Links
REST Home
EdwardsLab Homepage
EdwardsLab Blog
SLiMSuite Blog
SLiMSuite
Webservers
REST Pages
REST Status
REST Help
REST Tools
REST Alias Data
REST API
REST News
REST Sitemap

rje_hpc V1.1.1

High Performance Computing job farming

Module: rje_hpc
Description: High Performance Computing job farming
Version: 1.1.1
Last Edit: 22/02/18

Copyright © 2014 Richard J. Edwards - See source code for GNU License Notice


Imported modules: rje rje_obj rje_seq


See SLiMSuite Blog for further documentation. See rje for general commands.

Function

This module contains the generic code for farming jobs out over multiple processors and/or nodes. It is based on rje_iridis Version 1.10, updated to the new RJE_Object structure and made more generic. The recommended tool for actual job farming is SLiMFarmer, which inherits the rje_hpc.JobFarmer object and adds extra bells and whistles.

Commandline

Generic HPC Job Farming pptions

subsleep=X : Sleep time (seconds) between cycles of subbing out jobs to hosts [1]
subjobs=LIST : List of subjobs to farm out to HPC cluster []
hpcmode=X : Mode to be used for farming jobs between nodes (rsh/fork) [fork]
iolimit=X : Limit of number of IOErrors before termination [50]
memfree=X : Min. proportion of node memory to be free before spawning job [0.0]
keepfree=X : Number of processors to keep free on head node [1]

SLiMSuite and SeqSuite SeqBySeq program options

seqbyseq=T/F : Activate seqbyseq mode - assumes basefile=X option used for output [False]
farm=X : Program to farm out using seqbyseq mode. []
seqin=FILE : Input sequence file to farm out [None]
basefile=X : Base for output files - compiled from individual run results [None]
outlist=LIST : List of extensions of outputs to add to basefile for output (basefile.*) []
pickhead=X : Header to extract from OutList file and used to populate AccNum to skip []
startfrom=X : Sequence ID at which to begin the SeqBySeq farming [None]
rjepy=T/F : Whether program is an RJE *.py script (adds log processing) [True]
jobini=FILE : Ini file to pass to the called program [None]
pypath=PATH : Path to python modules ['/home/re1u06/Serpentry/']


See also rje.py generic commandline options.

History Module Version History

    # 1.0 - Initial Compilation based on rje_iridis V1.10.
    # 1.1 - Disabled memory checking in Windows and OSX.
    # 1.1.1 - Added output of subjob command to log as run.

© 2015 RJ Edwards. Contact: richard.edwards@unsw.edu.au.