Package src :: Module pyflow :: Class TaskManager
[hide private]
[frames] | no frames]

Class TaskManager

source code


This class runs on a separate thread from workflowRunner, launching jobs based on the current state of the TaskDAG

Instance Methods [hide private]
 
__init__(self, cdata, tdag)
This constructor should always be called with keyword arguments.
source code
 
run(self)
TaskManager runs so long as there are outstanding jobs
source code
 
_getCommandTaskRunner(self, task)
assist launch of a command-task
source code
 
_getWorkflowTaskRunner(self, task)
assist launch of a workflow-task
source code
 
_launchTask(self, task)
launch a specific task
source code
 
_startTasks(self, *args, **kw)
determine what tasks, if any, can be started
source code
 
_removeTaskFromRunningSet(self, task)
Given a running task which is already shown to be finished running, remove it from the running set, and recover allocated resources.
source code
 
harvestTasks(self, *args, **kw)
Check the set of running tasks to see if they've completed and update Node status accordingly:
source code
 
cancelTaskTree(self, *args, **kw)
Cancel a task and all of its children, without labeling the canceled tasks as errors
source code
 
stop(self, *args, **kw)
thread specific stop method, may be overridden to add async thread-specific kill behavior
source code
 
_areTasksDead(self, *args, **kw) source code
 
_isTerm(self) source code
 
_flowLog(self, msg, logState) source code
 
_infoLog(self, msg) source code
 
_errorLog(self, msg) source code

Inherited from StoppableThread: stopped

Inherited from threading.Thread: __repr__, getName, isAlive, isDaemon, is_alive, join, setDaemon, setName, start

Inherited from threading.Thread (private): _reset_internal_locks, _set_daemon, _set_ident

Inherited from threading._Verbose (private): _note

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __str__, __subclasshook__

Static Methods [hide private]

Inherited from StoppableThread: stopAll

Class Variables [hide private]

Inherited from StoppableThread (private): _stopAll

Properties [hide private]

Inherited from threading.Thread: daemon, ident, name

Inherited from threading.Thread (private): _block

Inherited from object: __class__

Method Details [hide private]

__init__(self, cdata, tdag)
(Constructor)

source code 

This constructor should always be called with keyword arguments. Arguments are:

*group* should be None; reserved for future extension when a ThreadGroup class is implemented.

*target* is the callable object to be invoked by the run() method. Defaults to None, meaning nothing is called.

*name* is the thread name. By default, a unique name is constructed of the form "Thread-N" where N is a small decimal number.

*args* is the argument tuple for the target invocation. Defaults to ().

*kwargs* is a dictionary of keyword arguments for the target invocation. Defaults to {}.

If a subclass overrides the constructor, it must make sure to invoke the base class constructor (Thread.__init__()) before doing anything else to the thread.

Parameters:
  • cdata - data from WorkflowRunner instance which will be constant during the lifetime of the TaskManager, should be safe to lookup w/o locking
  • tdag - task graph
Overrides: object.__init__

run(self)

source code 

TaskManager runs so long as there are outstanding jobs

Overrides: threading.Thread.run

_startTasks(self, *args, **kw)

source code 

determine what tasks, if any, can be started

Note that the lock is here to protect self.runningTasks

Decorators:
  • @lockMethod

harvestTasks(self, *args, **kw)

source code 

Check the set of running tasks to see if they've completed and update Node status accordingly:

Decorators:
  • @lockMethod

cancelTaskTree(self, *args, **kw)

source code 

Cancel a task and all of its children, without labeling the canceled tasks as errors

A canceled task will be stopped if it is running, or unqueued if it is waiting, and will be put into the waiting/ignored state unless it has already completed.

Decorators:
  • @lockMethod

stop(self, *args, **kw)

source code 

thread specific stop method, may be overridden to add async thread-specific kill behavior

Decorators:
  • @lockMethod
Overrides: StoppableThread.stop
(inherited documentation)

_areTasksDead(self, *args, **kw)

source code 
Decorators:
  • @lockMethod