public class PegasusLite extends java.lang.Object implements GridStart
1) create directory on worker node 2) fetch input data files 3) execute the job 4) transfer the output data files 5) cleanup the directoryThe following property should be set to false to disable the staging of the SLS files via the first level staging jobs
pegasus.transfer.stage.sls.file falseTo enable this implementation at runtime set the following property
pegasus.gridstart PegasusLite
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
CLASSNAME
The basename of the class that is implmenting this.
|
private PegasusBag |
mBag |
private java.util.Map<java.lang.String,java.lang.String> |
mChmodOnExecutionSiteMap
A map indexed by the execution site and value is the path to chmod on
that site.
|
private ADag |
mDAG |
protected boolean |
mEnablingPartOfAggregatedJob
An instance variable to track if enabling is happening as part of a clustered job.
|
protected boolean |
mGenerateLOF
A boolean indicating whether to generate lof files or not.
|
private Kickstart |
mKickstartGridStartImpl
Handle to kickstart GridStart implementation.
|
protected java.lang.String |
mLocalPathToPegasusLiteCommon
The local path on the submit host to pegasus-lite-common.sh
|
protected LogManager |
mLogger
The LogManager object which is used to log all the messages.
|
private java.lang.String |
mMajorVersionLevel
Stores the major version of the planner.
|
private java.lang.String |
mMinorVersionLevel
Stores the major version of the planner.
|
private java.lang.String |
mPatchVersionLevel
Stores the major version of the planner.
|
protected PlannerOptions |
mPOptions
The options passed to the planner.
|
protected PegasusProperties |
mProps
The object holding all the properties pertaining to Pegasus.
|
protected SiteStore |
mSiteStore
Handle to the site catalog store.
|
protected SLS |
mSLS
The handle to the SLS implementor
|
protected boolean |
mStageSLSFile
Boolean to track whether to stage sls file or not
|
protected java.lang.String |
mSubmitDir
The submit directory where the submit files are being generated for
the workflow.
|
private TransformationCatalog |
mTCHandle
Handle to Transformation Catalog.
|
protected boolean |
mTransferWorkerPackage
Boolean indicating whether worker package transfer is enabled or not
|
protected boolean |
mWorkerNodeExecution
A boolean indicating whether to have worker node execution or not.
|
(package private) java.util.Map<java.lang.String,java.lang.String> |
mWorkerPackageMap
A map indexed by execution site and the corresponding worker package
location in the submit directory
|
static java.lang.String |
PEGASUS_LITE_COMMON_FILE_BASENAME
The basename of the pegasus lite common shell functions file.
|
static java.lang.String |
PEGASUS_LITE_EXITCODE_SUCCESS_MESSAGE
The pegasus lite exitcode success message.
|
static java.lang.String |
SHORT_NAME
The SHORTNAME for this implementation.
|
static java.lang.String |
XBIT_DERIVATION_NS
The derivation namespace for the setXBit jobs.
|
static java.lang.String |
XBIT_DERIVATION_VERSION
The version number for the derivations for setXBit jobs.
|
static java.lang.String |
XBIT_EXECUTABLE_BASENAME
The basename of the pegasus dirmanager executable.
|
static java.lang.String |
XBIT_TRANSFORMATION
The logical name of the transformation that creates directories on the
remote execution pools.
|
static java.lang.String |
XBIT_TRANSFORMATION_NS
The transformation namespace for the setXBit jobs.
|
static java.lang.String |
XBIT_TRANSFORMATION_VERSION
The version number for the derivations for setXBit jobs.
|
mSeparator, VERSION
Constructor and Description |
---|
PegasusLite() |
Modifier and Type | Method and Description |
---|---|
private void |
associateCredentials(Job job,
java.util.Collection<FileTransfer> files)
Associates credentials with the job corresponding to the files that
are being transferred.
|
boolean |
canSetXBit()
Indicates whether the enabling mechanism can set the X bit
on the executable on the remote grid site, in addition to launching
it on the remote grid stie
|
private void |
complainForHeadNodeFileServer(java.lang.String jobname,
java.lang.String site)
Complains for a missing head node file server on a site for a job
|
private void |
construct(Job job,
java.lang.String key,
java.lang.String value)
Constructs a condor variable in the condor profile namespace
associated with the job.
|
protected java.lang.StringBuffer |
convertToTransferInputFormat(java.util.Collection<FileTransfer> files)
Convers the collection of files into an input format suitable for the
transfer executable
|
java.lang.String |
defaultPOSTScript()
Returns the SHORT_NAME for the POSTScript implementation that is used
to be as default with this GridStart implementation.
|
boolean |
enable(AggregatedJob job,
boolean isGlobusJob)
Enables a job to run on the grid.
|
boolean |
enable(Job job,
boolean isGlobusJob)
Enables a job to run on the grid by launching it directly.
|
private void |
enableForWorkerNodeExecution(Job job,
boolean isGlobusJob)
Enables jobs for worker node execution.
|
java.lang.String |
generateListofFilenamesFile(java.util.Set files,
java.lang.String basename)
Writes out the list of filenames file for the job.
|
private java.lang.String |
getDirectoryKey(Job job)
Returns the directory that is associated with the job to specify
the directory in which the job needs to run
|
protected java.lang.String |
getPathToChmodExecutable(java.lang.String site)
Returns the path to the chmod executable for a particular execution
site by looking up the transformation executable.
|
protected java.lang.String |
getSubmitHostPathToPegasusLiteCommon()
Determines the path to common shell functions file that Pegasus Lite
wrapped jobs use.
|
java.lang.String |
getVDSKeyValue()
Returns the value of the vds profile with key as Pegasus.GRIDSTART_KEY,
that would result in the loading of this particular implementation.
|
java.lang.String |
getWorkerNodeDirectory(Job job)
Returns the directory in which the job executes on the worker node.
|
void |
initialize(PegasusBag bag,
ADag dag)
Initializes the GridStart implementation.
|
private boolean |
removeDirectoryKey(Job job)
Returns a boolean indicating whether to remove remote directory
information or not from the job.
|
protected java.lang.String |
retrieveLocationForWorkerPackageFromTC(java.lang.String site)
Retrieves the location for the pegasus worker package from the TC for a site
|
protected boolean |
setXBitOnFile(java.lang.String file)
Sets the xbit on the file.
|
java.lang.String |
shortDescribe()
Returns a short textual description in the form of the name of the class.
|
protected java.lang.StringBuffer |
slurpInFile(java.lang.String directory,
java.lang.String file)
Convenience method to slurp in contents of a file into memory.
|
void |
useFullPathToGridStarts(boolean fullPath)
Setter method to control whether a full path to Gridstart should be
returned while wrapping a job or not.
|
protected java.io.File |
wrapJobWithPegasusLite(Job job,
boolean isGlobusJob)
Generates a seqexec input file for the job.
|
private PegasusBag mBag
private ADag mDAG
public static final java.lang.String CLASSNAME
public static final java.lang.String SHORT_NAME
public static final java.lang.String PEGASUS_LITE_COMMON_FILE_BASENAME
public static final java.lang.String XBIT_TRANSFORMATION
public static final java.lang.String XBIT_EXECUTABLE_BASENAME
public static final java.lang.String XBIT_TRANSFORMATION_NS
public static final java.lang.String XBIT_TRANSFORMATION_VERSION
public static final java.lang.String XBIT_DERIVATION_NS
public static final java.lang.String XBIT_DERIVATION_VERSION
public static final java.lang.String PEGASUS_LITE_EXITCODE_SUCCESS_MESSAGE
private java.lang.String mMajorVersionLevel
private java.lang.String mMinorVersionLevel
private java.lang.String mPatchVersionLevel
protected LogManager mLogger
protected PegasusProperties mProps
protected java.lang.String mSubmitDir
protected boolean mGenerateLOF
protected boolean mWorkerNodeExecution
protected SLS mSLS
protected PlannerOptions mPOptions
protected SiteStore mSiteStore
protected boolean mEnablingPartOfAggregatedJob
private Kickstart mKickstartGridStartImpl
private TransformationCatalog mTCHandle
protected boolean mStageSLSFile
protected java.lang.String mLocalPathToPegasusLiteCommon
protected boolean mTransferWorkerPackage
java.util.Map<java.lang.String,java.lang.String> mWorkerPackageMap
private java.util.Map<java.lang.String,java.lang.String> mChmodOnExecutionSiteMap
public void initialize(PegasusBag bag, ADag dag)
initialize
in interface GridStart
bag
- the bag of objects that is used for initialization.dag
- the concrete dag so far.public boolean enable(AggregatedJob job, boolean isGlobusJob)
enable
in interface GridStart
job
- the Job
object containing the job description
of the job that has to be enabled on the grid.isGlobusJob
- is true
, if the job generated a
line universe = globus
, and thus runs remotely.
Set to false
, if the job runs on the submit
host in any way.public boolean enable(Job job, boolean isGlobusJob)
enable
in interface GridStart
job
- the Job
object containing the job description
of the job that has to be enabled on the grid.isGlobusJob
- is true
, if the job generated a
line universe = globus
, and thus runs remotely.
Set to false
, if the job runs on the submit
host in any way.private void enableForWorkerNodeExecution(Job job, boolean isGlobusJob)
job
- the job to be enabled.isGlobusJob
- is true
, if the job generated a
line universe = globus
, and thus runs remotely.
Set to false
, if the job runs on the submit
host in any way.public boolean canSetXBit()
canSetXBit
in interface GridStart
public java.lang.String getVDSKeyValue()
getVDSKeyValue
in interface GridStart
org.griphyn.cPlanner.namespace.Pegasus#GRIDSTART_KEY
public java.lang.String shortDescribe()
shortDescribe
in interface GridStart
public java.lang.String defaultPOSTScript()
defaultPOSTScript
in interface GridStart
Kickstart.defaultPOSTScript()
private java.lang.String getDirectoryKey(Job job)
job
- the jobprivate boolean removeDirectoryKey(Job job)
job
- the job in question.private void construct(Job job, java.lang.String key, java.lang.String value)
job
- contains the job description.key
- the key of the profile.value
- the associated value.public java.lang.String generateListofFilenamesFile(java.util.Set files, java.lang.String basename)
files
- the list of PegasusFile
objects contains the files
whose stat information is required.basename
- the basename of the file that is to be createdpublic java.lang.String getWorkerNodeDirectory(Job job)
getWorkerNodeDirectory
in interface GridStart
job
- protected java.io.File wrapJobWithPegasusLite(Job job, boolean isGlobusJob)
1) create directory on worker node 2) fetch input data files 3) execute the job 4) transfer the output data files 5) cleanup the directory
job
- the job to be enabled.isGlobusJob
- is true
, if the job generated a
line universe = globus
, and thus runs remotely.
Set to false
, if the job runs on the submit
host in any way.protected java.lang.StringBuffer convertToTransferInputFormat(java.util.Collection<FileTransfer> files)
files
- Collection of FileTransfer
objects.protected java.lang.StringBuffer slurpInFile(java.lang.String directory, java.lang.String file) throws java.io.IOException
directory
- the directory where the file residesfile
- the file to be slurped in.java.io.IOException
protected java.lang.String getPathToChmodExecutable(java.lang.String site)
site
- the execution site.protected boolean setXBitOnFile(java.lang.String file)
file
- the file for which the xbit is to be setprotected java.lang.String getSubmitHostPathToPegasusLiteCommon()
public void useFullPathToGridStarts(boolean fullPath)
GridStart
useFullPathToGridStarts
in interface GridStart
fullPath
- if set to true, indicates that full path would be used.private void associateCredentials(Job job, java.util.Collection<FileTransfer> files)
job
- the job for which credentials need to be added.files
- the files that are being transferred.protected java.lang.String retrieveLocationForWorkerPackageFromTC(java.lang.String site)
private void complainForHeadNodeFileServer(java.lang.String jobname, java.lang.String site)
jobname
- the name of the jobsite
- the site