Home       Trees       Indices       Help   
glue
Package glue :: Module pipeline :: Class CondorDAGNode
[hide private]
[frames] | no frames]

Class CondorDAGNode

source code

Known Subclasses:

A CondorDAGNode represents a node in the DAG. It corresponds to a particular condor job (and so a particular submit file). If the job has variable (macro) options, they can be set here so each nodes executes with the correct options.

Instance Methods [hide private]
 
__init__(self, job) source code
 
__repr__(self) source code
 
add_checkpoint_file(self, filename)
Add filename as a checkpoint file for this DAG node		 source code
 
add_checkpoint_macro(self, filename) source code
 
add_file_arg(self, filename)
Add a variable (or macro) file name argument to the condor job.		 source code
 
add_file_opt(self, opt, filename, file_is_output_file=False)
Add a variable (macro) option for this node.		 source code
 
add_input_file(self, filename)
Add filename as a necessary input file for this DAG node.		 source code
 
add_input_macro(self, filename)
Add a variable (macro) for storing the input files associated with this node.		 source code
 
add_io_macro(self, io, filename)
Add a variable (macro) for storing the input/output files associated with this node.		 source code
 
add_macro(self, name, value)
Add a variable (macro) for this node.		 source code
 
add_output_file(self, filename)
Add filename as a output file for this DAG node.		 source code
 
add_output_macro(self, filename)
Add a variable (macro) for storing the output files associated with this node.		 source code
 
add_parent(self, node)
Add a parent to this node.		 source code
 
add_pegasus_profile(self, namespace, key, value)
Add a Pegasus profile to this job which will be written to the dax as <profile namespace="NAMESPACE" key="KEY">VALUE</profile> This can be used to add classads to particular jobs in the DAX		 source code
 
add_post_script_arg(self, arg)
Adds an argument to the post script that is executed before the DAG node is run.		 source code
 
add_pre_script_arg(self, arg)
Adds an argument to the pre script that is executed before the DAG node is run.		 source code
 
add_var_arg(self, arg, quote=False)
Add a variable (or macro) argument to the condor job.		 source code
 
add_var_condor_cmd(self, command, value)
Add a variable (macro) condor command for this node.		 source code
 
add_var_opt(self, opt, value, short=False)
Add a variable (macro) option for this node.		 source code
 
finalize(self)
The finalize method of a node is called before the node is finally added to the DAG and can be overridden to do any last minute clean up (such as setting extra command line arguments)		 source code
 
get_args(self)
Return the arguments for this node.		 source code
 
get_category(self)
Get the category for this node in the DAG.		 source code
 
get_checkpoint_files(self)
Return a list of checkpoint files for this DAG node and its job.		 source code
 
get_cmd_line(self)
Return the full command line that will be used when this node is run by DAGman.		 source code
 
get_cmd_tuple_list(self)
Return a list of tuples containg the command line arguments		 source code
 
get_dax_collapse(self)
Get the DAX collapse key for this node		 source code
 
get_grid_start(self)
Return the grid starter that pegasus will use.		 source code
 
get_input_files(self)
Return list of input files for this DAG node and its job.		 source code
 
get_name(self)
Get the name for this node in the DAG.		 source code
 
get_opts(self)
Return the opts for this node.		 source code
 
get_output_files(self)
Return list of output files for this DAG node and its job.		 source code
 
get_pegasus_profile(self)
Return the pegasus profile dictionary for this node.		 source code
 
get_post_script(self)
returns the name of the post script that is executed before the DAG node is run.		 source code
 
get_post_script_arg(self)
Returns and array of arguments to the post script that is executed before the DAG node is run.		 source code
 
get_priority(self)
Get the priority for this node in the DAG.		 source code
 
get_retry(self)
Return the number of times that this node in the DAG should retry.		 source code
 
get_vds_group(self)
Returns the VDS group key for this node		 source code
 
job(self)
Return the CondorJob that this node is associated with.		 source code
 
set_category(self, category)
Set the category for this node in the DAG.		 source code
 
set_dax_collapse(self, collapse)
Set the DAX collapse key for this node		 source code
 
set_grid_start(self, gridstart)
Set the grid starter that pegasus will use.		 source code
 
set_log_file(self, log)
Set the Condor log file to be used by this CondorJob.		 source code
 
set_name(self, name)
Set the name for this node in the DAG.		 source code
 
set_post_script(self, script)
Sets the name of the post script that is executed before the DAG node is run.		 source code
 
set_pre_script(self, script)
Sets the name of the pre script that is executed before the DAG node is run.		 source code
 
set_priority(self, priority)
Set the priority for this node in the DAG.		 source code
 
set_retry(self, retry)
Set the number of times that this node in the DAG should retry.		 source code
 
set_vds_group(self, group)
Set the name of the VDS group key when generating a DAX		 source code
 
write_category(self, fh)
Write the DAG entry for this node's category to the DAG file descriptor.		 source code
 
write_input_files(self, fh)
Write as a comment into the DAG file the list of input files for this DAG node.		 source code
 
write_job(self, fh)
Write the DAG entry for this node's job to the DAG file descriptor.		 source code
 
write_output_files(self, fh)
Write as a comment into the DAG file the list of output files for this DAG node.		 source code
 
write_parents(self, fh)
Write the parent/child relations for this job to the DAG file descriptor.		 source code
 
write_post_script(self, fh)
Write the post script for the job, if there is one		 source code
 
write_pre_script(self, fh)
Write the pre script for the job, if there is one		 source code
 
write_priority(self, fh)
Write the DAG entry for this node's priority to the DAG file descriptor.		 source code
 
write_vars(self, fh)
Write the variable (macro) options and arguments to the DAG file descriptor.		 source code
Method Details [hide private]

__init__(self, job)
(Constructor)

source code 
Parameters:
  • job - the CondorJob that this node corresponds to.

add_checkpoint_file(self, filename)

source code 

Add filename as a checkpoint file for this DAG node

Parameters:
  • filename - checkpoint filename to add

add_file_arg(self, filename)

source code 

Add a variable (or macro) file name argument to the condor job. The argument is added to the submit file and a different value of the argument can be set for each node in the DAG. The file name is also added to the list of input files for the DAX.

Parameters:
  • filename - name of option to add.

add_file_opt(self, opt, filename, file_is_output_file=False)

source code 

Add a variable (macro) option for this node. If the option specified does not exist in the CondorJob, it is added so the submit file will be correct when written. The value of the option is also added to the list of input files for the DAX.

Parameters:
  • opt - option name.
  • value - value of the option for this node in the DAG.
  • file_is_output_file - A boolean if the file will be an output file instead of an input file. The default is to have it be an input.

add_input_file(self, filename)

source code 

Add filename as a necessary input file for this DAG node.

Parameters:
  • filename - input filename to add

add_input_macro(self, filename)

source code 

Add a variable (macro) for storing the input files associated with this node.

Parameters:
  • filename - filename of input file

add_io_macro(self, io, filename)

source code 

Add a variable (macro) for storing the input/output files associated with this node.

Parameters:
  • io - macroinput or macrooutput
  • filename - filename of input/output file

add_macro(self, name, value)

source code 

Add a variable (macro) for this node. This can be different for each node in the DAG, even if they use the same CondorJob. Within the CondorJob, the value of the macro can be referenced as '$(name)' -- for instance, to define a unique output or error file for each node.

Parameters:
  • name - macro name.
  • value - value of the macro for this node in the DAG

add_output_file(self, filename)

source code 

Add filename as a output file for this DAG node.

Parameters:
  • filename - output filename to add

add_output_macro(self, filename)

source code 

Add a variable (macro) for storing the output files associated with this node.

Parameters:
  • filename - filename of output file

add_parent(self, node)

source code 

Add a parent to this node. This node will not be executed until the parent node has run sucessfully.

Parameters:
  • node - CondorDAGNode to add as a parent.

add_pegasus_profile(self, namespace, key, value)

source code 

Add a Pegasus profile to this job which will be written to the dax as <profile namespace="NAMESPACE" key="KEY">VALUE</profile> This can be used to add classads to particular jobs in the DAX

Parameters:
  • namespace - A valid Pegasus namespace, e.g. condor.
  • key - The name of the attribute.
  • value - The value of the attribute.

add_var_arg(self, arg, quote=False)

source code 

Add a variable (or macro) argument to the condor job. The argument is added to the submit file and a different value of the argument can be set for each node in the DAG.

Parameters:
  • arg - name of option to add.

add_var_condor_cmd(self, command, value)

source code 

Add a variable (macro) condor command for this node. If the command specified does not exist in the CondorJob, it is added so the submit file will be correct. PLEASE NOTE: AS with other add_var commands, the variable must be set for all nodes that use the CondorJob instance.

Parameters:
  • command - command name
  • value - Value of the command for this node in the DAG/DAX.

add_var_opt(self, opt, value, short=False)

source code 

Add a variable (macro) option for this node. If the option specified does not exist in the CondorJob, it is added so the submit file will be correct when written.

Parameters:
  • opt - option name.
  • value - value of the option for this node in the DAG.

get_args(self)

source code 

Return the arguments for this node. Note that this returns only the arguments for this instance of the node and not those associated with the underlying job template.

get_opts(self)

source code 

Return the opts for this node. Note that this returns only the options for this instance of the node and not those associated with the underlying job template.

get_post_script(self)

source code 

returns the name of the post script that is executed before the DAG node is run.

Parameters:
  • script - path to script

get_retry(self)

source code 

Return the number of times that this node in the DAG should retry.

Parameters:
  • retry - number of times to retry node.

set_grid_start(self, gridstart)

source code 

Set the grid starter that pegasus will use. 4.1 options are none (the default), kickstart and pegasuslite

set_log_file(self, log)

source code 

Set the Condor log file to be used by this CondorJob.

Parameters:
  • log - path of Condor log file.

set_post_script(self, script)

source code 

Sets the name of the post script that is executed before the DAG node is run.

Parameters:
  • script - path to script

set_pre_script(self, script)

source code 

Sets the name of the pre script that is executed before the DAG node is run.

Parameters:
  • script - path to script

set_retry(self, retry)

source code 

Set the number of times that this node in the DAG should retry.

Parameters:
  • retry - number of times to retry node.

set_vds_group(self, group)

source code 

Set the name of the VDS group key when generating a DAX

Parameters:
  • group - name of group for thus nore

write_category(self, fh)

source code 

Write the DAG entry for this node's category to the DAG file descriptor.

Parameters:
  • fh - descriptor of open DAG file.

write_input_files(self, fh)

source code 

Write as a comment into the DAG file the list of input files for this DAG node.

Parameters:
  • fh - descriptor of open DAG file.

write_job(self, fh)

source code 

Write the DAG entry for this node's job to the DAG file descriptor.

Parameters:
  • fh - descriptor of open DAG file.

write_output_files(self, fh)

source code 

Write as a comment into the DAG file the list of output files for this DAG node.

Parameters:
  • fh - descriptor of open DAG file.

write_parents(self, fh)

source code 

Write the parent/child relations for this job to the DAG file descriptor.

Parameters:
  • fh - descriptor of open DAG file.

write_post_script(self, fh)

source code 

Write the post script for the job, if there is one

Parameters:
  • fh - descriptor of open DAG file.

write_pre_script(self, fh)

source code 

Write the pre script for the job, if there is one

Parameters:
  • fh - descriptor of open DAG file.

write_priority(self, fh)

source code 

Write the DAG entry for this node's priority to the DAG file descriptor.

Parameters:
  • fh - descriptor of open DAG file.

write_vars(self, fh)

source code 

Write the variable (macro) options and arguments to the DAG file descriptor.

Parameters:
  • fh - descriptor of open DAG file.

   Home       Trees       Indices       Help   
glue
Generated by Epydoc 3.0.1 on Tue Dec 12 00:34:20 2017 http://epydoc.sourceforge.net