job.tasks package¶

Submodules¶

job.tasks.base_task module¶

Defines the abstract base class for all tasks

class job.tasks.base_task.AtomicCounter¶

Bases: object

Represents an atomic counter

get_next()¶

Returns the next integer

Returns:	The next integer
Return type:	int

class job.tasks.base_task.Task(task_id, task_name, agent_id)¶

Bases: object

Abstract base class for a task

check_timeout(when)¶

Checks this task’s progress against the given current time and times out the task if it has exceeded a timeout threshold

Parameters:	when (`datetime.datetime`) – The current time
Returns:	Whether this task has timed out
Return type:	bool

command¶

Returns the command to execute for the task

Returns:	The command to execute
Return type:	string

command_arguments¶

Returns the command to execute for the task

Returns:	The command to execute
Return type:	string

container_name¶

Returns the container name for the task, possibly None

Returns:	The container name
Return type:	string

docker_image¶

Returns the name of the Docker image to run for this task, possibly None

Returns:	The Docker image name
Return type:	string

docker_params¶

Returns the Docker parameters used to run this task

Returns:	The Docker parameters
Return type:	`list()`

ended¶

When this task ended, possibly None

Returns:	When this task ended
Return type:	`datetime.datetime`

exit_code¶

Returns the exit code for this task, possibly None

Returns:	The exit code
Return type:	int

force_kill()¶: Sets a task setting to force kill messages for this task

force_reconciliation()¶: Sets a task setting to force immediate reconciliation for this task

get_resources()¶

Returns the resources that are required/have been scheduled for this task

Returns:	The scheduled resources for this task
Return type:	`node.resources.node_resources.NodeResources`

has_been_launched¶

Indicates whether this task has been launched

Returns:	True if this task has been launched, False otherwise
Return type:	bool

has_ended¶

Indicates whether this task has ended

Returns:	True if this task has ended, False otherwise
Return type:	bool

has_started¶

Indicates whether this task has started

Returns:	True if this task has started, False otherwise
Return type:	bool

has_timed_out¶

Indicates whether this task has timed out

Returns:	True if this task has timed out, False otherwise
Return type:	bool

id¶

Returns the unique ID of the task

Returns:	The task ID
Return type:	string

is_docker_privileged¶

Indicates whether this task’s Docker container should be run in privileged mode

Returns:	True if the container should be run in privileged mode, False otherwise
Return type:	bool

launch(when)¶

Marks this task as having been launched

Parameters:	when (`datetime.datetime`) – The time that the task was launched

launched¶

When this task launched, possibly None

Returns:	When this task launched
Return type:	`datetime.datetime`

name¶

Returns the name of the task

Returns:	The task name
Return type:	string

needs_killed()¶

Indicates whether this task needs to be killed

Returns:	Whether this task needs to be killed
Return type:	bool

needs_reconciliation(when)¶

Indicates whether this task needs to be reconciled due to its latest status update being stale

Parameters:	when (`datetime.datetime`) – The current time
Returns:	Whether this task needs to be reconciled
Return type:	bool

started¶

When this task started, possibly None

Returns:	When this task started
Return type:	`datetime.datetime`

update(task_update)¶

Handles the given task update

Parameters:	task_update (`job.tasks.update.TaskStatusUpdate`) – The task update

uses_docker¶

Indicates whether this task uses Docker or not

Returns:	True if this task uses Docker, False otherwise
Return type:	bool

job.tasks.health_task module¶

Defines the class for a node health check task

class job.tasks.health_task.HealthTask(framework_id, agent_id)¶

Bases: job.tasks.node_task.NodeTask

Represents a task that performs a health check on a node. This class is thread-safe.

BAD_DAEMON_CODE = 2¶

BAD_LOGSTASH_CODE = 4¶

LOW_DOCKER_SPACE_CODE = 3¶

get_resources()¶: See job.tasks.base_task.Task.get_resources()

job.tasks.manager module¶

Defines the class that managers the currently running tasks

class job.tasks.manager.TaskManager¶

Bases: object

This class manages all currently running tasks. This class is thread-safe.

generate_status_json(nodes_list)¶

Generates the portion of the status JSON that describes the currently running node and system tasks

Parameters:	nodes_list (`list()`) – The list of nodes within the status JSON

get_all_tasks()¶

Returns all of current tasks

Returns:	The list of all current tasks
Return type:	[`job.tasks.base_task.Task`]

get_task(task_id)¶

Returns the task with the given ID, possibly None

Parameters:	task_id (int) – The task ID
Returns:	The task with the given ID
Return type:	`job.tasks.base_task.Task`

get_tasks_to_kill()¶

Returns all of the tasks that need to be killed

Returns:	The list of tasks to kill
Return type:	`list()`

get_tasks_to_reconcile(when)¶

Returns all of the tasks that need to be reconciled

Parameters:	when (`datetime.datetime`) – The current time
Returns:	The list of tasks that require reconciliation
Return type:	`list()`

get_timeout_tasks(when)¶

Returns all of the tasks that have timed out

Parameters:	when (`datetime.datetime`) – The current time
Returns:	The list of tasks that timed out
Return type:	`list()`

handle_task_update(task_update)¶

Handles the given task update

Parameters:	task_update (`job.tasks.update.TaskStatusUpdate`) – The task update

launch_tasks(tasks, when)¶

Adds the new tasks to the manager and marks them as launched

Parameters:	tasks ([`job.tasks.base_task.Task`]) – The tasks to add and launch when (`datetime.datetime`) – The time that the tasks were launched

job.tasks.node_task module¶

Defines the abstract base class for all node tasks

class job.tasks.node_task.NodeTask(task_id, task_name, agent_id)¶

Bases: job.tasks.base_task.Task

Abstract base class for a node task

job.tasks.pull_task module¶

Defines the class for a Docker pull task

class job.tasks.pull_task.PullTask(framework_id, agent_id)¶

Bases: job.tasks.node_task.NodeTask

Represents a task that pulls Docker images from the registry. This class is thread-safe.

get_resources()¶: See job.tasks.base_task.Task.get_resources()

job.tasks.pull_task.create_pull_command(image_name, check_exists=False)¶

Creates the Docker pull command to pull the given image name

Parameters:	image_name (string) – The name of the Docker image to pull check_exists (bool) – If True, skips pull if image already exists, False pulls image regardless
Returns:	The Docker pull command
Return type:	string

job.tasks.update module¶

Defines the class that represents a task status update

class job.tasks.update.TaskStatusUpdate(task_update_model, agent_id, data)¶

Bases: object

This class represents a task status update. This class is thread-safe.

EXIT_CODE_PATTERN = <_sre.SRE_Pattern object>¶

FAILED = u'FAILED'¶

FINISHED = u'FINISHED'¶

KILLED = u'KILLED'¶

LOST = u'LOST'¶

RUNNING = u'RUNNING'¶

STAGING = u'STAGING'¶

TASK_STATUS_CONVERSION = {u'TASK_ERROR': u'FAILED', u'TASK_RUNNING': u'RUNNING', u'TASK_LOST': u'LOST', u'TASK_KILLED': u'KILLED', u'TASK_STARTING': u'STAGING', u'TASK_STAGING': u'STAGING', u'TASK_FAILED': u'FAILED', u'TASK_FINISHED': u'FINISHED'}¶

TERMINAL_STATUSES = [u'FINISHED', u'FAILED', u'KILLED']¶

UNKNOWN = u'UNKNOWN'¶

job.tasks package¶

Submodules¶

job.tasks.base_task module¶

job.tasks.health_task module¶

job.tasks.manager module¶

job.tasks.node_task module¶

job.tasks.pull_task module¶

job.tasks.update module¶

Module contents¶