Package src :: Module pyflow :: Class TaskDAG

Class TaskDAG

Holds all tasks and their dependencies.

Also responsible for task state persistence/continue across interrupted runs. Object is accessed by both the workflow and taskrunner threads, so it needs to be thread-safe.

Instance Methods

[hide private]

__init__(self, isContinue, isForceContinue, isDryRun, taskInfoFile, taskStateFile, workflowClassName, startFromTasks, ignoreTasksAfter, resetTasks, flowLog)
No other object gets to access the taskStateFile, file locks are not required (but thread locks are)

source code

isTaskPresent(self, *args, **kw)

source code

getTask(self, *args, **kw)

source code

getHeadNodes(self, *args, **kw)
all tasks with no parents

source code

getTailNodes(self, *args, **kw)
all tasks with no (runnable) children

source code

getAllNodes(self, *args, **kw)
get all nodes in this namespace

source code

_isRunExhaustedNode(self, node, searched)

source code

isRunExhausted(self, *args, **kw)
Returns true if the run is as complete as possible due to errors

source code

isRunComplete(self, *args, **kw)
returns true if run is complete and error free

source code

_getReadyTasksFromNode(self, node, ready, searched)
helper function for getReadyTasks

source code

getReadyTasks(self, *args, **kw)
Go through DAG from the tail nodes and find all tasks which have all prerequisites completed:

source code

_markCheckPointsCompleteFromNode(self, node, completed, searched)
helper function for markCheckPointsComplete

source code

markCheckPointsComplete(self, *args, **kw)
traverse from tail nodes up, marking any checkpoint tasks (task.cmd=None) jobs that are ready as complete, return set of newly completed tasks:

source code

addTask(self, *args, **kw)
add new task to the DAG

source code

writeTaskStatus(self, *args, **kw)
(atomic on *nix) update of the runstate and errorstate for all tasks

source code

getTaskStatus(self, *args, **kw)
Enumerate status of command tasks (but look at sub-workflows to determine if specification is complete)

source code

writeTaskInfo(self, *args, **kw)
appends a description of all new tasks to the taskInfo file

source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __repr__, __setattr__, __sizeof__, __str__, __subclasshook__