Home       Trees       Indices       Help   
glue
Package glue :: Module pipeline :: Class CondorDAG
[hide private]
[frames] | no frames]

Class CondorDAG

source code

A CondorDAG is a Condor Directed Acyclic Graph that describes a collection of Condor jobs and the order in which to run them. All Condor jobs in the DAG must write their Codor logs to the same file. NOTE: The log file must not be on an NFS mounted system as the Condor jobs must be able to get an exclusive file lock on the log file.

Instance Methods [hide private]
 
__init__(self, log, dax=False) source code
 
add_maxjobs_category(self, categoryName, maxJobsNum)
Add a category to this DAG called categoryName with a maxjobs of maxJobsNum.		 source code
 
add_node(self, node)
Add a CondorDAGNode to this DAG.		 source code
 
add_pfn_cache(self, pfn_list)
Add an lfn pfn and pool tuple to the pfn cache Note: input looks like ('/path/to/file','file:///path/to/file','local')		 source code
 
get_dag_file(self)
Return the path to the DAG file.		 source code
 
get_dax_file(self)
Return the path to the DAG file.		 source code
 
get_jobs(self)
Return a list containing all the jobs in the DAG		 source code
 
get_maxjobs_categories(self)
Return an array of tuples containing (categoryName,maxJobsNum)		 source code
 
get_nodes(self)
Return a list containing all the nodes in the DAG		 source code
 
get_pegasus_worker(self)
Return the path to the pegasus worker package.		 source code
 
get_pfn_cache(self)
Return the pfn cache		 source code
 
is_dax(self)
Returns true if this DAG is really a DAX		 source code
 
prepare_dax(self, grid_site=None, tmp_exec_dir='.', peg_frame_cache=None)
Sets up a pegasus script for the given dag		 source code
 
set_dag_file(self, path)
Set the name of the file into which the DAG is written.		 source code
 
set_dax_file(self, path)
Set the name of the file into which the DAG is written.		 source code
 
set_integer_node_names(self)
Use integer node names for the DAG		 source code
 
set_pegasus_worker(self, path)
Set the path of a pagsus worker package to use for the workflow.		 source code
 
write_abstract_dag(self)
Write all the nodes in the workflow to the DAX file.		 source code
 
write_concrete_dag(self)
Write all the nodes in the DAG to the DAG file.		 source code
 
write_dag(self)
Write either a dag or a dax.		 source code
 
write_maxjobs(self, fh, category)
Write the DAG entry for this category's maxjobs to the DAG file descriptor.		 source code
 
write_script(self)
Write the workflow to a script (.sh instead of .dag).		 source code
 
write_sub_files(self)
Write all the submit files used by the dag to disk.		 source code
Method Details [hide private]

__init__(self, log, dax=False)
(Constructor)

source code 
Parameters:
  • log - path to log file which must not be on an NFS mounted file system.
  • dax - Set to 1 to create an abstract DAG (a DAX)

add_maxjobs_category(self, categoryName, maxJobsNum)

source code 

Add a category to this DAG called categoryName with a maxjobs of maxJobsNum.

Parameters:
  • node - Add (categoryName,maxJobsNum) tuple to CondorDAG.__maxjobs_categories.

add_node(self, node)

source code 

Add a CondorDAGNode to this DAG. The CondorJob that the node uses is also added to the list of Condor jobs in the DAG so that a list of the submit files needed by the DAG can be maintained. Each unique CondorJob will be added once to prevent duplicate submit files being written.

Parameters:
  • node - CondorDAGNode to add to the CondorDAG.

set_dag_file(self, path)

source code 

Set the name of the file into which the DAG is written.

Parameters:
  • path - path to DAG file.

set_dax_file(self, path)

source code 

Set the name of the file into which the DAG is written.

Parameters:
  • path - path to DAG file.

set_pegasus_worker(self, path)

source code 

Set the path of a pagsus worker package to use for the workflow.

Parameters:
  • path - path to worker package.

write_maxjobs(self, fh, category)

source code 

Write the DAG entry for this category's maxjobs to the DAG file descriptor.

Parameters:
  • fh - descriptor of open DAG file.
  • category - tuple containing type of jobs to set a maxjobs limit for and the maximum number of jobs of that type to run at once.

write_script(self)

source code 

Write the workflow to a script (.sh instead of .dag).

Assuming that parents were added to the DAG before their children, dependencies should be handled correctly.

write_sub_files(self)

source code 

Write all the submit files used by the dag to disk. Each submit file is written to the file name set in the CondorJob.


   Home       Trees       Indices       Help   
glue
Generated by Epydoc 3.0.1 on Tue Dec 12 00:34:20 2017 http://epydoc.sourceforge.net