node.resources package

Submodules

node.resources.exceptions module

Defines exceptions that can occur when interacting with resources

exception node.resources.exceptions.InvalidResources

Bases: exceptions.Exception

Exception indicating that the provided resources schema was invalid

node.resources.gpu_manager module

Defines the class that keeps track of GPU resources on a per node basis

class node.resources.gpu_manager.GPUManager

Bases: object

The class that holds the GPU library and manages it. Not thread safe!!!

classmethod assign_gpus_for_job(node_id, job_id, required_gpu_count)

method that assigns a specific GPU to a job id

param node_id:the node ID
type node_id:int
param job_id:the job ID
type job_id:int
param required_gpu_count:
 required GPUs
type required_gpu_count:
 int
returns:success or failure of assignment
rtype:bool
classmethod define_node_gpus(node_id, gpu_count)

Method that will create new GPUs in the dictionary based on offered resources from MESOS/DCOS

param node_id:the node ID
type node_id:int
param gpu_count:
 required GPUs
type gpu_count:int
classmethod get_available_gpu_for_node(node_id)

method to retrieve number of available GPUs for a specific node

param node_id:node ID
type node_id:int
returns:amount of GPUs available for a specific mode
rtype:int
classmethod get_gpu_count_for_node(node_id)

method to retrieve amount of GPUs for a specific node

param node_id:node ID
type node_id:int
returns:total amount of GPUs for a specific mode
rtype:int
classmethod get_nvidia_docker_label(node_id, job_id)

method that returns a formatted string that is to be used as a variable for nvidia-docker which specifies which GPUs will be used

param node_id:the node ID
type node_id:int
param job_id:the job ID
type job_id:int
returns:comma delimited list of GPUs to use
rtype:string
classmethod release_gpus(node_id, job_id)

Method sets GPU status back to available for completed jobs

param node_id:node ID
type node_id:int
param job_id:job ID
type job_id:int
classmethod reserve_gpus_for_job(node_id, required_gpu_count)

Method that reserves GPUs for future assignment to a job id

param node_id:the node id
type node_id:int
param required_gpu_count:
 requred GPUs
type required_gpu_count:
 int
returns:whether or not reserve was successful
rtype:bool
classmethod reset_gpu_dict()

resets the GPU dictionary.

node.resources.node_resources module

Defines the class that represents a set of resources on a node

class node.resources.node_resources.NodeResources(resources=None)

Bases: object

This class encapsulates a set of node resources

add(node_resources)

Adds the given resources

Parameters:node_resources (node.resources.NodeResources) – The resources to add
copy()

Returns a deep copy of these resources. Editing one of the resources objects will not affect the other.

Returns:A copy of these resources
Return type:node.resources.node_resources.NodeResources
cpus

The number of CPUs

Returns:The number of CPUs
Return type:float
disk

The amount of disk space in MiB

Returns:The amount of disk space
Return type:float
generate_status_json(resources_dict, key_name)

Generates the portion of the status JSON that describes these resources

Parameters:
  • resources_dict (dict) – The dict for all resources
  • key_name (string) – The key name for describing these resources
get_json()

Returns these resources as a JSON schema

Returns:The resources as a JSON schema
Return type:node.resources.json.Resources
gpus

The number of GPUs

Returns:The amount of GPUs
Return type:float
increase_up_to(node_resources)

Increases each resource up to the value in the given node resources

Parameters:node_resources (node.resources.NodeResources) – The resources
is_equal(node_resources)

Indicates if these resources are equal. This should be used for testing only.

Parameters:node_resources (node.resources.NodeResources) – The resources to compare
Returns:True if these resources are equal, False otherwise
Return type:bool
is_sufficient_to_meet(node_resources)

Indicates if these resources are sufficient to meet the requested resources

Parameters:node_resources (node.resources.NodeResources) – The requested resources
Returns:True if these resources are sufficient for the request, False otherwise
Return type:bool
limit_to(node_resources)

Limits each resource, subtracting any amount that goes over the amount in the given node resources

Parameters:node_resources (node.resources.NodeResources) – The resources
mem

The amount of memory in MiB

Returns:The amount of memory
Return type:float
remove_resource(name)

Removes the resource with the given name

Parameters:name (string) – The name of the resource to remove
resources

The list of resources

Returns:The list of resources
Return type:list()
round_values()

Rounds all of the resource values

subtract(node_resources)

Subtracts the given resources

Parameters:node_resources (node.resources.NodeResources) – The resources to subtract

node.resources.resource module

Defines the node resource classes

class node.resources.resource.Cpus(value)

Bases: node.resources.resource.ScalarResource

A scalar resource representing the number of CPUs

class node.resources.resource.Disk(value)

Bases: node.resources.resource.ScalarResource

A scalar resource representing the amount of disk space in MiB

class node.resources.resource.Gpus(value)

Bases: node.resources.resource.ScalarResource

A scalar resource representing the number of GPUs

class node.resources.resource.Mem(value)

Bases: node.resources.resource.ScalarResource

A scalar resource representing the amount of memory in MiB

class node.resources.resource.Resource(name, resource_type)

Bases: object

Abstract class for a node resource

copy()

Returns a deep copy of this resource. Editing one of the resource objects will not affect the other.

Returns:A copy of this resource
Return type:node.resources.resource.Resource
class node.resources.resource.ScalarResource(name, value)

Bases: node.resources.resource.Resource

A type of resource represented by a scalar floating point value

copy()

See node.resources.resource.Resource.copy()

Module contents