job package

Subpackages

Submodules

job.apps module

Defines the application configuration for the job application

class job.apps.JobConfig(app_name, app_module)

Bases: django.apps.config.AppConfig

Configuration for the job app

label = u'job'
name = u'job'
ready()

Registers components related to jobs

verbose_name = u'Job'

job.clock module

Contains the functionality of the Scale clock process

exception job.clock.ClockEventError

Bases: exceptions.Exception

Error class used when a clock event processor encounters a problem.

class job.clock.ClockEventProcessor

Bases: object

Base class used to process triggered clock events.

process_event(event, last_event=None)

Callback when a new event is triggered that sub-classes have registered to process.

Parameters:

:raises job.clock.ClockEventError: If the event should be cancelled for any reason.

job.clock.perform_tick()

Performs an iteration of the Scale clock.

A clock iteration consists of inspecting any trigger rules and corresponding events to see if there are any that have exceeded their scheduled time threshold. For rules that are due to run, a new event is created of the configured type and the registered clock function is executed.

job.clock.register_processor(name, processor_class)

Registers the given processor class definition to be called by the Scale clock at the given interval.

Processors from other applications can be registered during their ready() method.

Parameters:
  • name (string) – The system name of the processor, which is used in trigger rule configurations.
  • processor_class (job.clock.ClockProcessor) – The processor class to invoke when the associated event is triggered.

job.deprecation module

class job.deprecation.JobConnectionSunset

Bases: object

Class responsible for providing backward compatibility for old JobConnection interfaces as well as new Seed compliant connections.

static create(interface)

Instantiate an appropriately typed Job connection based on interface type

class job.deprecation.JobDataSunset

Bases: object

Class responsible for providing backward compatibility for old JobData interfaces as well as new Seed compliant ones.

static create(interface, data=None)

Instantiate an appropriately typed Job data based on version

class job.deprecation.JobInterfaceSunset

Bases: object

Class responsible for providing backward compatibility support for old style JobType interfaces as well as new Seed compliant interfaces.

static create(interface_dict, do_validate=True)

Instantiate an instance of the JobInterface based on inferred type

Parameters:
  • interface_dict (dict) – deserialized JSON interface
  • do_validate (bool) – whether schema validation should be applied
Returns:

instance of the job interface appropriate for input data
Return type:

job.configuration.interface.job_interface.JobInterface or job.seed.manifest.SeedManifest
static is_seed(interface)

Determines whether a given interface instance is Seed

Parameters:interface (job.configuration.interface.job_interface.JobInterface or job.seed.manifest.SeedManifest) – instance of JSON interface
Returns:whether interface is Seed compliant or not
Return type:bool
static is_seed_dict(interface_dict)

Determines whether a given interface dict is Seed

Parameters:interface_dict (dict) – deserialized JSON interface
Returns:whether interface is Seed compliant or not
Return type:bool

job.errors module

Defines functions for getting basic job errors

job.errors.get_invalid_manifest_error()

Returns the error for invalid results manifest

Returns:The invalid results error
Return type:error.models.Error
job.errors.get_missing_output_error()

Returns the error for missing a required output

Returns:The missing output error
Return type:error.models.Error
job.errors.get_unexpected_multiple_error()

job.exceptions module

Defines exceptions that can occur when interacting with jobs and job types

exception job.exceptions.InactiveJobType

Bases: exceptions.Exception

Exception indicating that a job type is inactive

exception job.exceptions.InvalidJobField

Bases: exceptions.Exception

Exception indicating that a job type or job field was given an invalid value

exception job.exceptions.NonSeedJobType

Bases: exceptions.Exception

Exception indicating that a job type is not a seed job type

job.job_type_serializers module

Defines the serializers for jobs and job types

class job.job_type_serializers.JobTypeBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts job type model fields to REST output

class job.job_type_serializers.JobTypeDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.job_type_serializers.JobTypeSerializerV6

Converts job type model fields to REST output.

class RecipeTypeSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeBaseSerializerV6

Converts recipe type model fields to REST output.

definition = None
job_types = None
sub_recipe_types = None
super_recipe_types = None
class job.job_type_serializers.JobTypeFailedStatusSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type failed status model and extra statistic fields to REST output.

class ErrorSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: error.serializers.ErrorBaseSerializerV6

Converts error model fields to REST output

class job.job_type_serializers.JobTypeFailedStatusSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type failed status model and extra statistic fields to REST output.

class ErrorSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: error.serializers.ErrorBaseSerializerV6

Converts error model fields to REST output

class job.job_type_serializers.JobTypeListSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.job_type_serializers.JobTypeBaseSerializerV6

Converts job type model fields to REST output

id = None
version = None
class job.job_type_serializers.JobTypePendingStatusSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type pending status model and extra statistic fields to REST output.

class job.job_type_serializers.JobTypePendingStatusSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type pending status model and extra statistic fields to REST output.

class job.job_type_serializers.JobTypeRevisionBaseSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts job type revision model fields to REST output.

class job.job_type_serializers.JobTypeRevisionDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.job_type_serializers.JobTypeRevisionSerializerV6

Converts job type revision model fields to REST output.

class job.job_type_serializers.JobTypeRevisionSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.job_type_serializers.JobTypeRevisionBaseSerializer

Converts job type revision model fields to REST output.

class job.job_type_serializers.JobTypeRunningStatusSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type running status model and extra statistic fields to REST output.

class job.job_type_serializers.JobTypeRunningStatusSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type running status model and extra statistic fields to REST output.

class job.job_type_serializers.JobTypeSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.job_type_serializers.JobTypeBaseSerializerV6

Converts job type model fields to REST output

class job.job_type_serializers.JobTypeStatusCountsSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts node status count object fields to REST output.

class job.job_type_serializers.JobTypeStatusSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type status model and extra statistic fields to REST output.

class job.job_type_serializers.JobTypeStatusSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts job type status model and extra statistic fields to REST output.

job.models module

Defines the database models for jobs and job types

class job.models.Job(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a job to be run on the cluster. A model lock must be obtained using select_for_update() on any job model before updating its status or superseding it.

Parameters:
  • job_type (django.db.models.ForeignKey) – The type of this job
  • job_type_rev (django.db.models.ForeignKey) – The revision of the job type when this job was created
  • event (django.db.models.ForeignKey) – The event that triggered the creation of this job
  • ingest_event (django.db.models.ForeignKey) – The ingest event that triggered the creation of this job
  • root_recipe (django.db.models.ForeignKey) – The root recipe that contains this job
  • recipe (django.db.models.ForeignKey) – The original recipe that created this job
  • batch (django.db.models.ForeignKey) – The batch that contains this job
  • is_superseded (django.db.models.BooleanField) – Whether this job has been superseded and is obsolete. This may be true while superseded_by_job (the reverse relationship of superseded_job) is null, indicating that this job is obsolete (its recipe has been superseded), but there is no new job that has directly taken its place.
  • root_superseded_job (django.db.models.ForeignKey) – The first job in the chain of superseded jobs. This field will be null for the first job in the chain (i.e. jobs that have a null superseded_job field).
  • superseded_job (django.db.models.ForeignKey) – The job that was directly superseded by this job. The reverse relationship can be accessed using ‘superseded_by_job’.
  • delete_superseded (django.db.models.BooleanField) – Whether this job should delete the products of the job that it has directly superseded
  • status (django.db.models.CharField) – The status of the job
  • node (django.db.models.ForeignKey) – The node on which the job is/was running (should only be set if status is RUNNING or final status)
  • error (django.db.models.ForeignKey) – The error that caused the failure (should only be set when status is FAILED)
  • max_tries (django.db.models.IntegerField) – The maximum number of times to try executing this job in case of errors (minimum one)
  • num_exes (django.db.models.IntegerField) – The number of executions this job has had
  • input (django.contrib.postgres.fields.JSONField) – JSON description defining the data for this job. This field must be populated when the job is first queued.
  • input_file_size (django.db.models.FloatField) – The amount of disk space in MiB required for input files for this job
  • output (django.contrib.postgres.fields.JSONField) – JSON description defining the results for this job. This field is populated when the job is successfully completed.
  • configuration (django.contrib.postgres.fields.JSONField) – JSON describing the overriding job configuration for this job instance
  • source_started (django.db.models.DateTimeField) – The start time of the source data for this job
  • source_ended (django.db.models.DateTimeField) – The end time of the source data for this job
  • source_sensor_class (django.db.models.CharField) – The class of sensor used to produce the source file for this job
  • source_sensor (django.db.models.CharField) – The specific identifier of the sensor used to produce the source file for this job
  • source_collection (django.db.models.CharField) – The collection of the source file for this job
  • source_task (django.db.models.CharField) – The task that produced the source file for this job
  • created (django.db.models.DateTimeField) – When the job was created
  • queued (django.db.models.DateTimeField) – When the job was added to the queue to be run when resources are available
  • started (django.db.models.DateTimeField) – When the job started running
  • ended (django.db.models.DateTimeField) – When the job stopped running, which could be due to successful completion or an error condition
  • last_status_change (django.db.models.DateTimeField) – When the job’s last status change occurred
  • superseded (django.db.models.DateTimeField) – When this job was superseded
  • last_modified (django.db.models.DateTimeField) – When the job was last modified
ALPHABETIZE_FIELDS = [u'job_type', u'recipe', u'status', u'error.category', u'error.title']
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

FINAL_STATUSES = [u'FAILED', u'COMPLETED', u'CANCELED']
JOB_STATUSES = ((u'PENDING', u'PENDING'), (u'BLOCKED', u'BLOCKED'), (u'QUEUED', u'QUEUED'), (u'RUNNING', u'RUNNING'), (u'FAILED', u'FAILED'), (u'COMPLETED', u'COMPLETED'), (u'CANCELED', u'CANCELED'))
exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

batch

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

batch_creator_job

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

batch_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

can_be_blocked()

Indicates whether this job can be set to BLOCKED status

Returns:True if the job can be set to BLOCKED status, false otherwise
Return type:bool
can_be_canceled()

Indicates whether this job can be set to CANCELED status

Returns:True if the job can be set to CANCELED status, false otherwise
Return type:bool
can_be_completed()

Indicates whether this job can be set to COMPLETED status

Returns:True if the job can be set to COMPLETED status, false otherwise
Return type:bool
can_be_failed()

Indicates whether this job can be set to FAILED status

Returns:True if the job can be set to FAILED status, false otherwise
Return type:bool
can_be_pending()

Indicates whether this job can be set to PENDING status

Returns:True if the job can be set to PENDING status, false otherwise
Return type:bool
can_be_queued()

Indicates whether this job can be placed on the queue for the first time

Returns:True if the job can be placed on the queue for the first time, false otherwise
Return type:bool
can_be_requeued()

Indicates whether this job can be placed back on the queue (re-queued)

Returns:True if the job can be placed back on the queue, false otherwise
Return type:bool
can_be_running()

Indicates whether this job can be set to RUNNING status

Returns:True if the job can be set to RUNNING status, false otherwise
Return type:bool
can_be_uncanceled()

Indicates whether this job can be uncanceled. Jobs can be uncanceled if they are currently canceled and have never been queued before.

Returns:True if the job can be uncanceled, false otherwise
Return type:bool
configuration

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

delete_superseded

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

ended

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

error

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

error_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

event

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

event_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_input_data()

Returns the input data for this job

Returns:The input data for this job
Return type:data.data.data.Data
get_input_files_json()

Returns the input files data with more details

Returns:The detailed input files dict for this job
Return type:dict
get_job_configuration()

Returns the job configuration for this job

Returns:The job configuration for this job
Return type:job.configuration.configuration.JobConfiguration
get_job_data()

Returns the data for this job

Returns:The data for this job
Return type:job.configuration.data.job_data.JobData or job.data.job_data.JobData
get_job_interface()

Returns the interface for this job

Returns:The interface for this job
Return type:job.configuration.interface.job_interface.JobInterface or job.seed.manifest.SeedManifest
get_job_results()

Returns the results for this job

Returns:The results for this job
Return type:job.configuration.results.job_results.JobResults or job.seed.results.job_results.JobResults
get_next_by_created(*moreargs, **morekwargs)
get_next_by_last_modified(*moreargs, **morekwargs)
get_output_data()

Returns the output data for this job

Returns:The output data for this job
Return type:data.data.data.Data
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_last_modified(*moreargs, **morekwargs)
get_resources()

Returns the resources required for this job

Returns:The required resources
Return type:node.resources.node_resources.NodeResources
get_resources_dict()

Gathers resources information and returns it as a dict.

Returns:The job resources dict
Return type:dict
get_status_display(*moreargs, **morekwargs)
get_v6_configuration_json()

Returns the job configuration in v6 of the JSON schema

Returns:The job configuration in v6 of the JSON schema
Return type:dict
get_v6_input_data_json()

Returns the input data for this job as v6 json with the version stripped

Returns:The v6 JSON input data dict for this job
Return type:dict
get_v6_output_data_json()

Returns the output data for this job as v6 json with the version stripped

Returns:The v6 JSON output data dict for this job
Return type:dict
get_v6_resources_json()

Returns the job resources in v6 of the JSON schema

Returns:The job resources in v6 of the JSON schema
Return type:dict
has_been_queued()

Indicates whether this job has been queued at least once

Returns:True if the job has been queued at least once, false otherwise.
Return type:bool
has_input()

Indicates whether this job has its input

Returns:True if the job has its input, false otherwise.
Return type:bool
has_output()

Indicates whether this job has its output

Returns:True if the job has its output, false otherwise.
Return type:bool
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

ingest_event

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

ingest_event_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

ingest_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

input

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

input_file_size

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_ready_for_children()

Indicates whether this job is ready for its children jobs to be queued

Returns:True if this job is ready for its children jobs, false otherwise
Return type:bool
is_superseded

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_type_rev

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_rev_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobexecution_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobexecutionend_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobexecutionoutput_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobinputfile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

last_modified

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

last_status_change

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

max_tries

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

node

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

node_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

num_exes

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <job.models.JobManager object>
output

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

queue_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

queued

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipenode_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

root_recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

root_recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

root_superseded_job

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

root_superseded_job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

scalefile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

set_input(job_input)

Validates and sets the input for this job model. No database update is applied. This job should have its related job_type and job_type_rev models populated.

Parameters:job_input (job.configuration.data.job_data.JobData) – JSON description defining the job input
Raises:job.configuration.data.exceptions.InvalidData – If the job input is invalid
source_collection

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_ended

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_sensor

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_sensor_class

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_started

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_task

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

started

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

strike_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

superseded

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

superseded_by_job

Accessor to the related object on the reverse side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

place.restaurant is a ReverseOneToOneDescriptor instance.

superseded_by_jobs

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

superseded_job

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

restaurant.place is a ForwardOneToOneDescriptor instance.

superseded_job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

update_database_with_input(when)

Updates the database with this job’s input JSON

Parameters:when (datetime.datetime) – The current time
class job.models.JobExecution(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a job execution that has been scheduled to run on a node

Parameters:
  • job (django.db.models.ForeignKey) – The job that was scheduled
  • job_type (django.db.models.ForeignKey) – The type of the job that was scheduled
  • recipe (django.db.models.ForeignKey) – The original recipe that created this job
  • batch (django.db.models.ForeignKey) – The batch that contains this job
  • exe_num (django.db.models.IntegerField) – The number of the job’s execution
  • cluster_id (django.db.models.CharField) – This is an ID for the job execution that is unique in the context of the cluster, allowing Scale components (task IDs, Docker volume names, etc) to have unique names within the cluster
  • node (django.db.models.ForeignKey) – The node on which the job execution is scheduled
  • timeout (django.db.models.IntegerField) – The maximum amount of time to allow this execution to run before being killed (in seconds)
  • input_file_size (django.db.models.FloatField) – The total amount of disk space in MiB for all input files for this job execution
  • resources (django.contrib.postgres.fields.JSONField) – JSON description describing the resources allocated to this job execution
  • configuration (django.contrib.postgres.fields.JSONField) – JSON description describing the configuration for how the job execution should be run
  • queued (django.db.models.DateTimeField) – When the job execution was added to the queue
  • started (django.db.models.DateTimeField) – When the job execution was started (scheduled)
  • created (django.db.models.DateTimeField) – When this model was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

batch

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

batch_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

cluster_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

configuration

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

create_canceled_job_exe_end_model(when)

Creates and returns a canceled job execution end for this job execution

Parameters:when (datetime.datetime) – When the job execution was canceled
Returns:The job execution end model
Return type:job.models.JobExecutionEnd
create_job_exe_end_model(task_results, status, error_id, when)

Creates and returns a job execution end model for this job execution

Parameters:
  • task_results (job.execution.tasks.json.results.task_results.TaskResults) – The task results for the execution
  • status (str) – The final job execution status
  • error_id (int) – The ID of the error (for failed executions), possibly None
  • when (datetime.datetime) – When the job execution ended
Returns:

The job execution end model
Return type:

job.models.JobExecutionEnd
created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

docker_image

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exe_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_cluster_id()

Gets the cluster ID for the job execution

Returns:The cluster ID for the job execution
Return type:string
get_execution_configuration()

Returns the configuration for this job execution

Returns:The configuration for this job execution
Return type:job.execution.configuration.json.exe_config.ExecutionConfiguration
get_log_json(include_stdout=True, include_stderr=True, since=None)

Get log data from elasticsearch as a dict (from the raw JSON).

Parameters:
  • include_stdout (bool) – If True, include stdout in the result
  • include_stderr (bool) – If True include stderr in the result
  • since (datetime.datetime or None) – If present, only retrieve logs since this timestamp (non-inclusive).
Return type:

tuple of (dict, datetime.datetime) with the results or None and the last modified timestamp
get_log_text(include_stdout=True, include_stderr=True, since=None, html=False)

Get log data from elasticsearch.

Parameters:
  • include_stdout (bool) – If True, include stdout in the result
  • include_stderr (bool) – If True include stderr in the result
  • since (datetime.datetime or None) – If present, only retrieve logs since this timestamp (non-inclusive).
  • html (bool) – If True, wrap the lines in div elements with stdout/stderr css classes, otherwise use plain text
Return type:

tuple of (str, datetime.datetime) with the log or None and last modified timestamp
get_next_by_created(*moreargs, **morekwargs)
get_next_by_queued(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_queued(*moreargs, **morekwargs)
get_resources()

Returns the resources allocated to this job execution

Returns:The allocated resources
Return type:node.resources.node_resources.NodeResources
get_status()

Returns the status of this job execution

Returns:The status of the job execution
Return type:string
get_v6_resources_json()

Returns the resources allocated to this job execution in v6 of the JSON schema

Returns:The job resources in v6 of the JSON schema
Return type:dict
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

input_file_size

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobexecutionend

Accessor to the related object on the reverse side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

place.restaurant is a ReverseOneToOneDescriptor instance.

jobexecutionoutput

Accessor to the related object on the reverse side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

place.restaurant is a ReverseOneToOneDescriptor instance.

node

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

node_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <job.models.JobExecutionManager object>
static parse_cluster_id(task_id)

Parses and returns the cluster ID from the given task ID

Parameters:task_id (string) – The task ID
Returns:The cluster ID
Return type:string
queued

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

resources

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

scalefile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

set_cluster_id(framework_id, job_id, exe_num)

Sets the unique cluster ID for this job execution

Parameters:
  • framework_id (string) – The scheduling framework ID
  • job_id (int) – The job ID
  • exe_num (int) – The number of the execution
started

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

taskupdate_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

timeout

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class job.models.JobExecutionEnd(*args, **kwargs)

Bases: django.db.models.base.Model

Represents the end of a job execution, including the execution’s final status and end time

Parameters:
  • job_exe (django.db.models.ForeignKey) – The primary key to the scheduled job execution
  • job (django.db.models.ForeignKey) – The job that was executed
  • job_type (django.db.models.ForeignKey) – The type of the job that was executed
  • exe_num (django.db.models.IntegerField) – The number of the job’s execution
  • task_results (django.contrib.postgres.fields.JSONField) – JSON description of the task results
  • status (django.db.models.CharField) – The final status of the execution
  • error (django.db.models.ForeignKey) – The error that caused the failure (should only be set when status is FAILED)
  • node (django.db.models.ForeignKey) – The node on which the job execution was run (None if it was canceled before being scheduled)
  • queued (django.db.models.DateTimeField) – When the job execution was added to the queue
  • started (django.db.models.DateTimeField) – When the job execution was started (scheduled)
  • seed_started (django.db.models.DateTimeField) – When the Seed container was started (main task)
  • seed_ended (django.db.models.DateTimeField) – When the Seed container ended (main task)
  • ended (django.db.models.DateTimeField) – When the job execution ended
  • created (django.db.models.DateTimeField) – When this model was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

JOB_EXE_END_STATUSES = ((u'FAILED', u'FAILED'), (u'COMPLETED', u'COMPLETED'), (u'CANCELED', u'CANCELED'))
exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

ended

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

error

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

error_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exe_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*moreargs, **morekwargs)
get_next_by_ended(*moreargs, **morekwargs)
get_next_by_queued(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_ended(*moreargs, **morekwargs)
get_previous_by_queued(*moreargs, **morekwargs)
get_status_display(*moreargs, **morekwargs)
get_task_results()

Returns the task results for this job execution

Returns:The task results for this job execution
Return type:job.execution.tasks.json.results.task_results.TaskResults
job

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_exe

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

restaurant.place is a ForwardOneToOneDescriptor instance.

job_exe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

node

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

node_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <job.models.JobExecutionEndManager object>
queued

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

seed_ended

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

seed_started

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

started

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

task_results

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class job.models.JobExecutionEndManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling job execution ends.

get_recent_job_exe_end_metrics(when)

Returns the recent job_exe_end models that were COMPLETED or FAILED after the given time. The related job_exe, job_type, and error models will be populated.

Parameters:when (datetime.datetime) – Returns executions that finished after this time
Returns:The list of job_exe_end models with related job_exe, job_type, and error models
Return type:list()
class job.models.JobExecutionManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling job executions.

get_job_exe_details(job_id, exe_num)

Returns additional details about a job execution related to the given job identifier and execution number.

Parameters:
  • job_id (int) – Query job executions associated with the job identifier.
  • exe_num (int) – Query job executions associated with the execution number.
Returns:

Details about the job execution that match the job execution identifier.
Return type:

[job.models.JobExecution]
get_job_exe_with_job_and_job_type(job_id, exe_num)

Gets a job execution with its related job and job_type models populated using only one database query

Parameters:
  • job_id (int) – The job ID
  • exe_num (int) – The execution number
Returns:

The job execution model with related job and job_type models populated
Return type:

job.models.JobExecution
get_job_exes(job_id, started=None, ended=None, statuses=None, node_ids=None, error_ids=None, error_categories=None, order=None)

Returns a list of job executions for the given job.

Parameters:
  • job_id (int) – Query job executions associated with the job identifier.
  • started (datetime.datetime) – Query job executions updated after this amount of time.
  • ended (datetime.datetime) – Query job executions updated before this amount of time.
  • statuses ([string]) – Query job executions with the a specific status.
  • node_ids ([int]) – Query job executions that ran on a node with the identifier.
  • error_ids ([int]) – Query job executions that had an error with the identifier.
  • error_categories ([string]) – Query job executions that had an error with the given category.
  • order ([string]) – A list of fields to control the sort order.
Returns:

The list of job executions that match the job identifier.
Return type:

[job.models.JobExecution]
get_latest_execution(job_id)

Gets the latest job execution for the given job ID and returns it with the related job model

Parameters:job_id (int) – The job ID
Returns:The job execution model with related job and job_type models populated
Return type:job.models.JobExecution
get_logs(job_exe_id)

Gets additional details for the given job execution model based on related model attributes.

Parameters:job_exe_id (int) – The unique identifier of the job execution.
Returns:The job execution with extra related attributes.
Return type:job.models.JobExecution
get_unfinished_job_exes()

Returns the job executions for the jobs that are unfinished. Unfinished jobs are jobs where the latest execution has been scheduled, but the job is still in QUEUED or RUNNING status. The returned job executions will not have any of their JSON fields populated. The returned list is a queryset iterator, so only access it once.

Returns:The job execution models for the unfinished jobs
Return type:list()
class job.models.JobExecutionOutput(*args, **kwargs)

Bases: django.db.models.base.Model

Represents the output of a job execution

Parameters:
  • job_exe (django.db.models.ForeignKey) – The primary key to the scheduled job execution
  • job (django.db.models.ForeignKey) – The job that was executed
  • job_type (django.db.models.ForeignKey) – The type of the job that was executed
  • exe_num (django.db.models.IntegerField) – The number of the job’s execution
  • output (django.contrib.postgres.fields.JSONField) – JSON description of the job execution’s output
  • created (django.db.models.DateTimeField) – When this model was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

exe_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*moreargs, **morekwargs)
get_output()

Returns the output for this job execution

Returns:The output for this job execution
Return type:job.configuration.results.job_results.JobResults
get_previous_by_created(*moreargs, **morekwargs)
job

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_exe

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

restaurant.place is a ForwardOneToOneDescriptor instance.

job_exe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
output

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class job.models.JobInputFile(*args, **kwargs)

Bases: django.db.models.base.Model

Links a job and its input files together. A file can be used as input to multiple jobs and a job can accept multiple input files.

Parameters:
  • job (django.db.models.ForeignKey) – The job that the input file is linked to
  • input_file (django.db.models.ForeignKey) – The input file that is linked to the job
  • job_input (django.db.models.CharField) – The name of the job input that the file was passed into
  • created (django.db.models.DateTimeField) – When this link was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

input_file

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

input_file_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_input

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <job.models.JobInputFileManager object>
class job.models.JobInputFileManager

Bases: django.db.models.manager.Manager

Provides additional methods for handleing JobInputFiles

get_job_input_files(job_id, started=None, ended=None, time_field=None, file_name=None, job_input=None)

Returns a query for Input Files filtered on the given fields.

Parameters:
  • job_id (int) – The job ID
  • started (datetime.datetime) – Query Scale files updated after this amount of time.
  • ended (datetime.datetime) – Query Scale files updated before this amount of time.
  • time_field (string) – The time field to use for filtering.
  • file_name (str) – Query Scale files with the given file name.
  • job_input (str) – The name of the job input that the file was passed into
Returns:

The Scale file query
Return type:

django.db.models.QuerySet
class job.models.JobManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling jobs

create_job_v6(job_type_rev, event_id=None, ingest_event_id=None, input_data=None, root_recipe_id=None, recipe_id=None, batch_id=None, superseded_job=None, job_config=None)

Creates a new job for the given job type revision and returns the (unsaved) job model

Parameters:
  • job_type_rev (job.models.JobTypeRevision) – The job type revision (with populated job_type model) of the job to create
  • event_id (int) – The event ID that triggered the creation of this job
  • ingest_event_id (int) – The ingest event that triggered the creation of this job
  • input_data (data.data.data.Data) – The job’s input data (optional)
  • root_recipe_id (int) – The ID of the root recipe that contains this job
  • recipe_id (int) – The ID of the original recipe that created this job
  • batch_id (int) – The ID of the batch that contains this job
  • superseded_job (job.models.Job) – The job that the created job is superseding, possibly None
  • job_config (job.configuration.configuration.JobConfiguration) – The configuration overrides for running this job, possibly None
Returns:

The new job model
Return type:

job.models.Job

:raises data.data.exceptions.InvalidData: If the input data is invalid

filter_jobs(started=None, ended=None, source_started=None, source_ended=None, source_sensor_classes=None, source_sensors=None, source_collections=None, source_tasks=None, statuses=None, job_ids=None, job_type_ids=None, job_type_names=None, job_type_categories=None, batch_ids=None, recipe_ids=None, error_categories=None, error_ids=None, is_superseded=False, order=None)

Returns a query for job models that filters on the given fields

Parameters:
  • started (datetime.datetime) – Query jobs updated after this amount of time.
  • ended (datetime.datetime) – Query jobs updated before this amount of time.
  • source_started (datetime.datetime) – Query jobs where source collection started after this time.
  • source_ended (datetime.datetime) – Query jobs where source collection ended before this time.
  • source_sensor_classes (list()) – Query jobs with the given source sensor class.
  • source_sensor (list()) – Query jobs with the given source sensor.
  • source_collection (list()) – Query jobs with the given source class.
  • source_tasks (list()) – Query jobs with the given source tasks.
  • statuses (list()) – Query jobs with the a specific status.
  • job_ids (list()) – Query jobs associated with the identifier.
  • job_type_ids (list()) – Query jobs of the type associated with the identifier.
  • job_type_names (list()) – Query jobs of the type associated with the name.
  • job_type_categories (list()) – Query jobs of the type associated with the category.
  • batch_ids (list()) – Query jobs associated with the given batch identifiers.
  • recipe_ids ([int]) – Query jobs associated with the given recipe identifiers.
  • error_categories (list()) – Query jobs that failed due to errors associated with the category.
  • error_ids (list()) – Query jobs that failed due to these errors.
  • is_superseded (bool) – Query jobs that match the is_superseded flag.
  • order (list()) – A list of fields to control the sort order.
Returns:

The job query
Return type:

django.db.models.QuerySet

Returns a query for job models that filters on the given fields. The returned query includes the related job_type, job_type_rev, event, and error fields, except for the job_type.manifest and job_type_rev.manifest fields.

Parameters:
  • started (datetime.datetime) – Query jobs updated after this amount of time.
  • ended (datetime.datetime) – Query jobs updated before this amount of time.
  • source_started (datetime.datetime) – Query jobs where source collection started after this time.
  • source_ended (datetime.datetime) – Query jobs where source collection ended before this time.
  • source_sensor_classes (list()) – Query jobs with the given source sensor class.
  • source_sensor (list()) – Query jobs with the given source sensor.
  • source_collection (list()) – Query jobs with the given source class.
  • source_tasks (list()) – Query jobs with the given source tasks.
  • statuses ([string]) – Query jobs with the a specific execution status.
  • job_ids ([int]) – Query jobs associated with the identifier.
  • job_type_ids ([int]) – Query jobs of the type associated with the identifier.
  • job_type_names ([string]) – Query jobs of the type associated with the name.
  • batch_ids ([int]) – Query jobs associated with the given batch identifiers.
  • recipe_ids ([int]) – Query jobs associated with the given recipe identifiers.
  • error_categories ([string]) – Query jobs that failed due to errors associated with the category.
  • error_ids ([int]) – Query jobs that failed with errors with the given identifiers.
  • is_superseded (bool) – Query jobs that match the is_superseded flag.
  • order ([string]) – A list of fields to control the sort order.
Returns:

The job query
Return type:

django.db.models.QuerySet
get_basic_jobs(job_ids)

Returns the basic job models for the given IDs with no related fields

Parameters:job_ids (list()) – The job IDs
Returns:The job models
Return type:list()
get_details(job_id)

Gets additional details for the given job model based on related model attributes.

The additional fields include: input files, recipe, job executions, and generated products.

Parameters:job_id (int) – The unique identifier of the job.
Returns:The job with extra related attributes.
Return type:job.models.Job
get_job_with_interfaces(job_id)

Gets the job model for the given ID with related job_type_rev and recipe__recipe_type_rev models

Parameters:job_id (int) – The job ID
Returns:The job model with related job_type_rev and recipe__recipe_type_rev models
Return type:job.models.Job
get_jobs_v6(started=None, ended=None, source_started=None, source_ended=None, source_sensor_classes=None, source_sensors=None, source_collections=None, source_tasks=None, statuses=None, job_ids=None, job_type_ids=None, job_type_names=None, batch_ids=None, recipe_ids=None, error_categories=None, error_ids=None, is_superseded=None, order=None)

Returns a list of jobs within the given time range.

Parameters:
  • started (datetime.datetime) – Query jobs updated after this amount of time.
  • ended (datetime.datetime) – Query jobs updated before this amount of time.
  • source_started (datetime.datetime) – Query jobs where source collection started after this time.
  • source_ended (datetime.datetime) – Query jobs where source collection ended before this time.
  • source_sensor_classes (list()) – Query jobs with the given source sensor class.
  • source_sensor (list()) – Query jobs with the given source sensor.
  • source_collection (list()) – Query jobs with the given source class.
  • source_tasks (list()) – Query jobs with the given source task.
  • statuses ([string]) – Query jobs with the a specific execution status.
  • job_ids ([int]) – Query jobs associated with the identifier.
  • job_type_ids ([int]) – Query jobs of the type associated with the identifier.
  • job_type_names ([string]) – Query jobs of the type associated with the name.
  • batch_ids ([int]) – Query jobs associated with batches with the given identifiers.
  • recipe_ids ([int]) – Query jobs associated with recipes with the given identifiers.
  • error_categories ([string]) – Query jobs that failed due to errors associated with the category.
  • error_ids ([int]) – Query jobs that failed with errors with the given identifiers.
  • is_superseded (bool) – Query jobs that match the is_superseded flag.
  • order ([string]) – A list of fields to control the sort order.
Returns:

The list of jobs that match the time range.
Return type:

[job.models.Job]

Gets the job models for the given IDs with related job_type, job_type_rev, and batch models

Parameters:job_ids (list()) – The job IDs
Returns:The job models
Return type:list()
get_locked_job(job_id)

Locks and returns the job model for the given ID with no related fields. Caller must be within an atomic transaction.

Parameters:job_id (int) – The job ID
Returns:The job model
Return type:job.models.Job
get_locked_jobs(job_ids)

Locks and returns the job models for the given IDs with no related fields. Caller must be within an atomic transaction.

Parameters:job_ids (list()) – The job IDs
Returns:The job models
Return type:list()
increment_max_tries(job_ids, when)

Increments the max_tries of the given jobs to be their current number of executions plus the max_tries setting of their associated job type.

Parameters:
  • job_ids (list()) – The job IDs
  • when (datetime.datetime) – The current time
populate_input_files(jobs)

Populates each of the given jobs with its input file references in a field called “input_files”.

Parameters:jobs ([job.models.Job]) – The list of jobs to augment with input files.
process_job_input(job)

Processes the input data for the given job to populate its input file models and input meta-data fields. The caller must have obtained a model lock on the given job model.

Parameters:job (job.models.Job) – The locked job model
process_job_output(job_ids, when)

Processes the job output for the given job IDs. The caller must have obtained model locks on the job models in an atomic transaction. All jobs that are both COMPLETED and have their execution output stored, will have the output saved in the job model. The list of job IDs for models that are both COMPLETED and have output will be returned.

Parameters:
  • job_ids (list()) – The job IDs
  • when (datetime.datetime) – The current time
Returns:

The list of job IDs that are both COMPLETED and have output
Return type:

list()
set_job_input_data_v6(job, input_data)

Sets the given input data as a v6 JSON for the given job. The job model must have its related job_type_rev model populated.

Parameters:

:raises data.data.exceptions.InvalidData: If the data is invalid

supersede_jobs(job_ids, when)

Updates the given job IDs to be superseded

Parameters:
  • job_ids (list()) – The job IDs to supersede
  • when (datetime.datetime) – The time that the jobs were superseded
update_jobs_node(job_ids, node_id, when)

Updates the jobs with the given IDs to have the given node and start time

Parameters:
  • job_ids (list()) – A list of job IDs to update
  • node_id (int) – The node ID
  • when (datetime.datetime) – The start time
update_jobs_to_blocked(jobs, when)

Updates the given job models to the BLOCKED status and returns the IDs of the models that were successfully set to BLOCKED. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid state for being BLOCKED will be ignored.

Parameters:
  • jobs (list()) – The job models to set to BLOCKED
  • when (datetime.datetime) – The status change time
Returns:

The list of job IDs that were successfully set to BLOCKED
Return type:

list()
update_jobs_to_canceled(jobs, when)

Updates the given job models to the CANCELED status and returns the IDs of the models that were successfully set to CANCELED. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid state for being CANCELED will be ignored.

Parameters:
  • jobs (list()) – The job models to set to CANCELED
  • when (datetime.datetime) – The status change time
Returns:

The list of job IDs that were successfully set to CANCELED
Return type:

list()
update_jobs_to_completed(jobs, when)

Updates the given job models to the COMPLETED status and returns the IDs of the models that were successfully set to COMPLETED. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid state for being COMPLETED will be ignored.

Parameters:
  • jobs (list()) – The job models to set to COMPLETED
  • when (datetime.datetime) – The ended time
Returns:

The list of job IDs that were successfully set to COMPLETED
Return type:

list()
update_jobs_to_failed(jobs, error_id, when)

Updates the given job models to the FAILED status and returns the IDs of the models that were successfully set to FAILED. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid state for being FAILED will be ignored.

Parameters:
  • jobs (list()) – The job models to set to FAILED
  • error_id (int) – The ID of the error that caused the failure
  • when (datetime.datetime) – The ended time
Returns:

The list of job IDs that were successfully set to FAILED
Return type:

list()
update_jobs_to_pending(jobs, when)

Updates the given job models to the PENDING status and returns the IDs of the models that were successfully set to PENDING. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid state for being PENDING will be ignored.

Parameters:
  • jobs (list()) – The job models to set to PENDING
  • when (datetime.datetime) – The status change time
Returns:

The list of job IDs that were successfully set to PENDING
Return type:

list()
update_jobs_to_queued(jobs, when_queued, requeue=False)

Updates the given job models to the QUEUED status and returns the IDs of the models that were successfully set to QUEUED. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid status for being queued, are without job input, or are superseded will be ignored.

Parameters:
  • jobs (list()) – The job models to set to QUEUED
  • when_queued (datetime.datetime) – The time that the jobs are queued
  • requeue (bool) – Whether this is a re-queue (True) or a first queue (False)
Returns:

The list of job IDs that were successfully set to QUEUED
Return type:

list()
update_jobs_to_running(jobs, when)

Updates the given job models to the RUNNING status and returns the IDs of the models that were successfully set to RUNNING. The caller must have obtained model locks on the job models in an atomic transaction. Any jobs that are not in a valid state for RUNNING will be ignored.

Parameters:
  • jobs (list()) – The job models to set to RUNNING
  • when (datetime.datetime) – The start time
Returns:

The list of job IDs that were successfully set to RUNNING
Return type:

list()
class job.models.JobType(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a type of job that can be run on the cluster. Any updates to a job type model requires obtaining a lock on the model using select_for_update().

Parameters:
  • name (django.db.models.CharField) – The identifying name of the job type used by clients for queries
  • version (django.db.models.CharField) – The version of the job type
  • version_array (list()) – The version of the job type split into SemVer integer components (major,minor,patch,prerelease)
  • is_system (django.db.models.BooleanField) – Whether this is a system type
  • is_long_running (django.db.models.BooleanField) – Whether this type is long running. A job of this type is intended to run for a long time, potentially indefinitely, without timing out and always being re-queued after a failure
  • is_active (django.db.models.BooleanField) – Whether the job type is active (false once job type is deprecated)
  • is_paused (django.db.models.BooleanField) – Whether the job type is paused (while paused no jobs of this type will be scheduled off of the queue)
  • is_published (django.db.models.BooleanField) – Whether this job type has outputs that are published.
  • unmet_resources (django.db.models.CharField) – List of resource names that currently don’t exist or aren’t sufficient in the cluster
  • max_scheduled (django.db.models.IntegerField) – The maximum number of jobs of this type that may be scheduled to run at the same time
  • max_tries (django.db.models.IntegerField) – The maximum number of times to try executing a job in case of errors (minimum one)
  • icon_code (string of a FontAwesome icon code) – A font-awesome icon code (like ‘f013’ for gear) to use when representing this job type
  • revision_num (django.db.models.IntegerField) – The current revision number of the interface, starts at one.
  • docker_image (django.db.models.CharField) – The Docker image containing the code to run for this job (if uses_docker is True)
  • manifest (django.contrib.postgres.fields.JSONField) – JSON description defining the manifest for running a job of this type (previously interface).
  • configuration (django.contrib.postgres.fields.JSONField) – JSON describing the default job configuration for jobs of this type
  • created (django.db.models.DateTimeField) – When the job type was created
  • deprecated (django.db.models.DateTimeField) – When the job type was deprecated (no longer active)
  • paused (django.db.models.DateTimeField) – When the job type was paused
  • last_modified (django.db.models.DateTimeField) – When the job type was last modified
ALPHABETIZE_FIELDS = [u'name', u'version']
BASE_FIELDS = (u'id', u'name', u'version', u'manifest', u'configuration', u'icon_code', u'is_active', u'is_paused', u'is_published')
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

UNEDITABLE_FIELDS = (u'version_array', u'is_system', u'is_long_running', u'is_active', u'created', u'deprecated', u'last_modified', u'paused', u'revision_num')
configuration

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

deprecated

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

docker_image

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_description()

Gets the job description from the manifest

Returns:The description
Return type:str
get_error_mapping()

Returns the error mapping for this job type

Returns:The error mapping
Return type:job.error.mapping.JobErrorMapping
get_job_configuration()

Returns the job configuration for this job type

Returns:The job configuration for this job type
Return type:job.configuration.configuration.JobConfiguration
get_job_interface()

Returns the interface for running jobs of this type

Returns:The job interface for this type
Return type:job.seed.manifest.SeedManifest
get_job_version()

Gets the Job version either from field or manifest

Returns:the version
Return type:str
get_job_version_array(version)

Gets the Job version either from field or manifest as an array of integers for sorting using the semver package. The result will be an array of length 4 with the first three being integers containing major,minor and patch version numbers. The fourth will be either a None value or if a prerelease value is present this function will attempt to convert it into an integer for sorting.

Parameters:version (django.db.models.CharField) – The version of the job type
Returns:the version array
Return type:array
get_next_by_created(*moreargs, **morekwargs)
get_next_by_last_modified(*moreargs, **morekwargs)
get_package_version()

Gets the package version from manifest

This value is None in legacy job type

Returns:the version
Return type:str
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_last_modified(*moreargs, **morekwargs)
get_resources()

Returns the resources required for jobs of this type

Returns:The required resources
Return type:node.resources.node_resources.NodeResources
get_secrets_key()

Returns the reference key for job type secrets stored in the secrets backend.

Returns:The job_type name and version concatenated
Return type:str
get_title()

Gets the job title from the manifest

Returns:The Title
Return type:str
get_v6_configuration_json()

Returns the job configuration in v6 of the JSON schema

Returns:The job configuration in v6 of the JSON schema
Return type:dict
icon_code

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_active

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_long_running

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_paused

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_published

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_system

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

job_types_for_recipe_type

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobexecution_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobexecutionend_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobexecutionoutput_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobload_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobtyperevision_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobtypetag_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

last_modified

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

manifest

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

max_scheduled

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

max_tries

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

metricsjobtype_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

natural_key()

Django method to define the natural key for a job type as the combination of name and version

Returns:A tuple representing the natural key
Return type:tuple(string, string)
objects = <job.models.JobTypeManager object>
paused

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

populate_from_manifest(manifest)

Set job type fields with values from given seed manifest

Parameters:manifest (job.seed.manifest.SeedManifest) – The Seed Manifest defining the interface for running a job of this type
queue_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

revision_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

scalefile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

unmet_resources

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

version

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

version_array

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class job.models.JobTypeFailedStatus(job_type, error, count=0, first_error=None, last_error=None)

Bases: object

Represents job type system failure statistics.

Parameters:
  • job_type (job.models.JobType) – The job type being counted.
  • count (int) – The number of job executions failed for the associated job type.
  • first_error (datetime.datetime) – The date/time of the first job execution failed for the associated job type.
  • last_error (datetime.datetime) – The date/time of the last job execution failed for the associated job type.
class job.models.JobTypeKey(name, version)

Bases: tuple

name

Alias for field number 0

version

Alias for field number 1

class job.models.JobTypeManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling job types

create_job_type_v6(*args, **kwargs)

Creates a new Seed job type and saves it in the database. All database changes occur in an atomic transaction.

Parameters:
  • docker_image (string) – The docker image containing the code to run for this job.
  • manifest (job.seed.manifest.SeedManifest) – The Seed Manifest defining the interface for running a job of this type
  • icon_code (string) – A font-awesome icon code to use when representing this job type.
  • max_scheduled (integer) – Maximum number of jobs of this type that may be scheduled to run at the same time.
  • configuration (job.configuration.configuration.JobConfiguration) – The configuration for running a job of this type, possibly None
  • is_published (bool) – Whether this job type has outputs that are published.
Returns:

The new job type
Return type:

job.models.JobType

:raises job.exceptions.InvalidJobField: If a given job type field has an invalid value

edit_job_type_v6(*args, **kwargs)

Edits the given job type and saves the changes in the database. All database changes occur in an atomic transaction. An argument of None for a field indicates that the field should not change.

Parameters:
  • job_type_id (int) – The unique identifier of the job type to edit
  • manifest (job.seed.manifest.SeedManifest) – The Seed Manifest defining the interface for running a job of this type
  • docker_image (string) – The docker image containing the code to run for this job.
  • icon_code (string) – A font-awesome icon code to use when representing this job type.
  • is_active (bool) – Whether this job type is active or deprecated.
  • is_paused (bool) – Whether this job type is paused and should not have jobs scheduled or not
  • is_published (bool) – Whether this job type has outputs that are published.
  • max_scheduled (integer) – Maximum number of jobs of this type that may be scheduled to run at the same time.
  • configuration (job.configuration.configuration.JobConfiguration) – The configuration for running a job of this type, possibly None
  • auto_update (bool) – If true, recipes that contain this job type will automatically be updated

:raises job.exceptions.InvalidJobField: If a given job type field has an invalid value

get_by_natural_key(name, version)

Django method to retrieve a job type for the given natural key

Parameters:
  • name (string) – The human-readable name of the job type
  • version (string) – The version of the job type
Returns:

The job type defined by the natural key
Return type:

job.models.JobType
get_clock_job_type()

Returns the Scale Clock job type

Returns:The clock job type
Return type:job.models.JobType
get_details_v6(name=None, version=None, id=None)

Returns the job type for the given name and version with all detail fields included.

The additional fields include: errors, job_counts_6h, job_counts_12h, and job_counts_24h.

Parameters:
  • name (string) – The name of the job type.
  • version (string) – The version of the job type.
Returns:

The job type with all detail fields included.
Return type:

job.models.JobType
get_failed_status()

Returns all job types that have failed due to system errors.

The results consist of standard job type models, plus additional computed statistics fields including a total count of associated jobs and the last status change of a running job.

Returns:The list of each job type with additional statistic fields.
Return type:[job.models.JobTypeFailedStatus]
get_job_type_names_v6(keywords=None, ids=None, is_active=None, is_system=None, order=None)

Returns a list of the latest version of job types

Parameters:
  • keywords (list()) – Query job types with name, title, description or tag matching one of the specified keywords
  • ids – Query job types with a version matching the given ids
  • is_active (bool) – Query job types that are actively available for use.
  • is_system (bool) – Query job types that are system job types.
  • order (list()) – A list of fields to control the sort order.
Returns:

The list of latest version of job types that match the given parameters.
Return type:

list()
get_job_type_versions_v6(name, is_active=None, order=None)

Returns a list of the versions of the job type with the given name

Parameters:
  • name (string) – Name of the job type
  • is_active (bool) – Query job types that are actively available for use.
  • order ([string]) – A list of fields to control the sort order.
Returns:

The list of versions of the job type that match the given parameters.
Return type:

[job.models.JobType]
get_job_types_v6(keywords=None, ids=None, is_active=None, is_system=None, order=None)

Returns a list of all job types

Parameters:
  • keywords (list()) – Query job types with name, title, description or tag matching one of the specified keywords
  • ids – Query job types with a version matching the given ids
  • is_active (bool) – Query job types that are actively available for use.
  • is_system – Query job types that are system job types.
  • order (list()) – A list of fields to control the sort order.
Returns:

The list of latest version of job types that match the given parameters.
Return type:

list()
get_pending_status()

Returns a status overview of all currently pending job types.

The results consist of standard job type models, plus additional computed statistics fields including a total count of associated jobs and the longest pending job.

Returns:The list of each job type with additional statistic fields.
Return type:[job.models.JobTypePendingStatus]
get_performance(job_type_id, started, ended=None)

Returns the job count statistics for a given job type and time range.

Parameters:
  • job_type_id (int) – The unique identifier of the job type.
  • started (datetime.datetime) – Query job types updated after this amount of time.
  • ended (datetime.datetime) – Query job types updated before this amount of time.
Returns:

A list of job counts organized by status.
Return type:

[job.models.JobTypeStatusCounts]
get_recipe_job_type_ids(definition)

Gets the model ids of the job types contained in the given RecipeDefinition

Parameters:definition (recipe.definition.definition.RecipeDefinition) – RecipeDefinition to search for job types
Returns:set of JobType ids
Return type:set[int]
get_running_status()

Returns a status overview of all currently running job types.

The results consist of standard job type models, plus additional computed statistics fields including a total count of associated jobs and the longest running job.

Returns:The list of each job type with additional statistic fields.
Return type:[job.models.JobTypeRunningStatus]
get_status(started, ended=None, is_active=None)

Returns a list of job types with counts broken down by job status.

Note that all running job types are counted regardless of date/time filters.

Parameters:
  • started (datetime.datetime) – Query job types updated after this amount of time.
  • ended (datetime.datetime) – Query job types updated before this amount of time.
  • is_active (bool) – Query job types that are actively available for use.
Returns:

The list of job types with supplemented statistics.
Return type:

[job.models.JobTypeStatus]
get_timeline_jobs_json(started=None, ended=None, type_ids=None, type_names=None, type_versions=None)

Returns the timeline information for the specified job types/time range

Parameters:
  • started (datetime.datetime) – The start date of the timeline
  • ended (datetime.datetime) – The end date of the timeline
  • type_ids ([int]) – List of job type ids to limit the results
  • type_names ([string]) – List of job type names to limit the results
  • type_versions ([string]) – List of job type versions to limit the results
Returns:

JSON response for timeline information
Return type:

dict
set_job_type_secrets(secrets_key, secrets)

Sends request to SecretsHandler to write secrets for a job type.

Parameters:
  • secrets_key (str) – Reference pointer for job_type settings stored in secrets backend
  • secrets (dict) – Secret settings required by this job type.
validate_job_type_v6(manifest_dict, configuration_dict=None)

Validates a new job type prior to attempting a save

Parameters:
  • manifest_dict (dict) – The Seed Manifest defining the interface for running a job of this type
  • configuration_dict (dict) – The configuration for running a job of this type, possibly None
Returns:

The job type validation.
Return type:

job.models.JobTypeValidation
class job.models.JobTypePendingStatus(job_type, count=0, longest_pending=None)

Bases: object

Represents job type pending statistics.

Parameters:
  • job_type (job.models.JobType) – The job type being counted.
  • count (int) – The number of job executions pending for the associated job type.
  • longest_pending (datetime.datetime) – The date/time of the last job execution for the associated job type.
class job.models.JobTypeRevision(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a revision of a job type. New revisions are created when the manifest of a job type changes. Any inserts of a job type revision model requires obtaining a lock using select_for_update() on the corresponding job type model.

Parameters:
  • job_type (django.db.models.ForeignKey) – The job type for this revision
  • revision_num (django.db.models.IntegerField) – The number for this revision, starting at one
  • manifest (django.contrib.postgres.fields.JSONField) – The JSON seed manifest for this revision of the job type (previously interface)
  • created (django.db.models.DateTimeField) – When this revision was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

docker_image

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_input_interface()

Returns the input interface for this revision

Returns:The input interface for this revision
Return type:data.interface.interface.Interface
get_job_interface()

Returns the job type interface for this revision

Returns:The job type interface for this revision
Return type:job.configuration.interface.job_interface.JobInterface or job.seed.manifest.SeedManifest
get_next_by_created(*moreargs, **morekwargs)
get_output_interface()

Returns the output interface for this revision

Returns:The output interface for this revision
Return type:data.interface.interface.Interface
get_previous_by_created(*moreargs, **morekwargs)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

manifest

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

natural_key()

Django method to define the natural key for a job type revision as the combination of job type and revision number

Returns:A tuple representing the natural key
Return type:tuple(string, int)
objects = <job.models.JobTypeRevisionManager object>
revision_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class job.models.JobTypeRevisionManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling job type revisions

create_job_type_revision(job_type)

Creates a new revision for the given job type. The job type’s interface and revision number must already be updated. The caller must have obtained a lock using select_for_update() on the given job type model.

Parameters:job_type (job.models.JobType) – The job type
get_by_natural_key(job_type, revision_num)

Django method to retrieve a job type revision for the given natural key

Parameters:
  • job_type (job.models.JobType) – The job type
  • revision_num (int) – The revision number
Returns:

The job type revision defined by the natural key
Return type:

job.models.JobTypeRevision
get_details_v6(name, version, revision_num)

Returns the job type revision for the given name, version and revision number with all detail fields included.

Parameters:
  • name (string) – The name of the job type.
  • version (string) – The version of the job type.
  • revision_num (int) – The revision number of the job type revision.
Returns:

The job type revision with all detail fields included.
Return type:

job.models.JobTypeRevision
get_job_type_revisions_v6(name, version, order=None)

Returns a list of the versions of the job type with the given name

Parameters:
  • name (string) – Name of the job type
  • version (string) – The version of the job type.
  • order ([string]) – A list of fields to control the sort order.
Returns:

The list of job type revisions that match the given parameters.
Return type:

[job.models.JobTypeRevision]
get_revision(job_type_name, job_type_version, revision_num)

Returns the revision (with populated job_type model) for the given job type and revision number

Parameters:
  • job_type_name (string) – The name of the job type
  • job_type_version (string) – The version of the job type
  • revision_num (int) – The revision number
Returns:

The revision
Return type:

job.models.JobTypeRevision
get_revisions(revision_tuples)

Returns a dict that maps revision ID to job type revision for the job type revisions that match the given values. Each revision model will have its related job type model populated.

Parameters:revision_tuples (list()) – A list of tuples (job type name, job type version, revision num) for additional revisions to return
Returns:The revisions stored by revision ID
Return type:dict
class job.models.JobTypeRunningStatus(job_type, count=0, longest_running=None)

Bases: object

Represents job type running statistics.

Parameters:
  • job_type (job.models.JobType) – The job type being counted.
  • count (int) – The number of job executions running for the associated job type.
  • longest_running (datetime.datetime) – The date/time of the last job execution for the associated job type.
class job.models.JobTypeStatus(job_type, job_counts=None)

Bases: object

Represents job type statistics.

Parameters:
class job.models.JobTypeStatusCounts(status, count=0, most_recent=None, category=None)

Bases: object

Represents job counts for a job type.

Parameters:
  • status (string) – The job execution status being counted.
  • count (int) – The number of job executions for the associated status.
  • most_recent (datetime.datetime) – The date/time of the last job execution for the associated status.
  • category (string) – The category of the job execution status being counted. Note that currently this will only be populated for types of ERROR status values.
class job.models.JobTypeTag(*args, **kwargs)

Bases: django.db.models.base.Model

Stores a job type and tag combination

Parameters:
  • type (job) – The job type
  • tag (django.db.models.CharField) – The tag associated with the job type
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <job.models.JobTypeTagManager object>
tag

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class job.models.JobTypeTagManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling job type tags

clear_job_type_tags(job_type_id)

Removes all job type tag objects for the specified job type. Useful when updating the revision and repopulating with new tags

Parameters:job_type_id (int) – The job type ID to remove tags for
create_job_type_tags(job_type, tags)

Creates a set of job type tags and saves them in the database

Parameters:
  • job_type (job.models.JobType) – The job type
  • tag (list()) – The tags of the job type image
Returns:

The new job type tags
Return type:

list()
update_job_type_tags(job_type, manifest)

Updates the tags for the given job type

Parameters:
class job.models.JobTypeValidation(is_valid, errors, warnings)

Bases: tuple

errors

Alias for field number 1

is_valid

Alias for field number 0

warnings

Alias for field number 2

class job.models.TaskUpdate(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a status update received for a task

Parameters:
  • job_exe (django.db.models.ForeignKey) – The job execution that the task belongs to
  • task_id (django.db.models.CharField) – The task ID
  • status (django.db.models.CharField) – The status of the task
  • timestamp (django.db.models.DateTimeField) – When the status update occurred (may be None)
  • source (django.db.models.CharField) – An optional source of the task status update
  • reason (django.db.models.CharField) – An optional reason for the task status update
  • message (django.db.models.TextField) – An optional message related to the task status update
  • created (django.db.models.DateTimeField) – When the task update was saved in the database
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_exe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_exe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

message

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
reason

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

task_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

timestamp

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job.serializers module

Defines the serializers for jobs and job types

class job.serializers.JobBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts job model fields to REST output.

class job.serializers.JobDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobSerializerV6

Converts job model and related fields to REST output.

class job.serializers.JobExecutionBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts job execution model fields to REST output

class job.serializers.JobExecutionDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobExecutionSerializerV6

Converts job execution model fields to REST output

class job.serializers.JobExecutionLogSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobExecutionSerializerV6

Converts job execution model fields to REST output

class job.serializers.JobExecutionSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobExecutionBaseSerializerV6

Converts job execution model fields to REST output

class ErrorBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts error model fields to REST output

class NodeBaseSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts node model fields to REST output.

class job.serializers.JobRevisionSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobSerializerV6

Converts job model fields to REST output.

class job.serializers.JobSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobBaseSerializerV6

Converts job model fields to REST output.

class BatchBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Base serializer for batches

class ErrorBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts error model fields to REST output

class IngestEventSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: ingest.ingest_event_serializers.IngestEventBaseSerializerV6

Converts ingest event model fields to REST output

class RecipeBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts recipe model fields to REST output.

class TriggerEventSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: trigger.serializers.TriggerEventBaseSerializerV6

Converts trigger event model fields to REST output.

class job.serializers.SeedJsonSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts Seed formatted input / output JSON to REST ouput

job.urls module

Defines the URLs for the RESTful job services

job.views module

class job.views.CancelJobsView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for canceling jobs

parser_classes = (<class 'rest_framework.parsers.JSONParser'>,)
post(request)

Submit command message to cancel jobs that fit the given filter criteria

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Returns:the HTTP response to send back to the user
queryset
serializer_class

alias of JobSerializerV6

class job.views.JobDetailsView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for retrieving details about a single job.

get(request, job_id)

Retrieves jobs and returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • job_id (int encoded as a str) – The ID for the job.
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
queryset
serializer_class

alias of JobDetailsSerializerV6

class job.views.JobExecutionDetailsView(**kwargs)

Bases: rest_framework.generics.RetrieveAPIView

This view is the endpoint for viewing job execution detail

queryset
retrieve(request, job_id, exe_num)

Gets job execution and associated job_type id, name, and version

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • job_id (int encoded as a str) – The ID for the job.
  • exe_num (int encoded as a str) – the execution number
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
serializer_class

alias of JobExecutionDetailsSerializerV6

class job.views.JobExecutionSpecificLogView(**kwargs)

Bases: rest_framework.generics.RetrieveAPIView

This view is the endpoint for viewing the text of specific job execution logs

renderer_classes = (<class 'rest_framework.renderers.JSONRenderer'>, <class 'util.rest.PlainTextRenderer'>, <class 'rest_framework.renderers.StaticHTMLRenderer'>)
retrieve(request, job_exe_id, log_id)

Gets job execution log specified.

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • job_exe_id (int) – the job execution id
  • log_id (str) – the log name to get (stdout, stderr, or combined)
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
class job.views.JobExecutionsView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for viewing job executions and their associated job_type id, name, and version

list(request, job_id=None)

Gets job executions and their associated job_type id, name, and version

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • job_id (int encoded as a str) – The ID for the job.
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
list_v6(request, job_id)

Gets job executions and their associated job_type id, name, and version

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
serializer_class

alias of JobExecutionSerializerV6

class job.views.JobInputFilesView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This is the endpoint for retrieving details about input files associated with a job.

get(request, job_id)

Retrieve detailed information about the input files for a job

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • job_id (int encoded as a str) – The ID for the job.
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
queryset
serializer_class

alias of ScaleFileSerializerV6

class job.views.JobTypeDetailsView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for retrieving/updating details of a version of a job type.

get(request, name, version)

Retrieves the details for a job type version and return them in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the job type
  • version (string) – The version of the job type
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
patch(request, name, version)

Edits an existing seed job type and returns the updated details

Parameters:
  • request (rest_framework.request.Request) – the HTTP PATCH request
  • job_type_id (int encoded as a str) – The ID for the job type.
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
queryset
serializer_class

alias of JobTypeDetailsSerializerV6

class job.views.JobTypeNamesView(**kwargs)

Bases: rest_framework.generics.ListCreateAPIView

This view is the endpoint for retrieving the list of all job types.

list(request)

Retrieves the list of all job types and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
list_v6(request)

Retrieves the list of all job type names and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
serializer_class

alias of JobTypeListSerializerV6

class job.views.JobTypeRevisionDetailsView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for retrieving/updating details of a version of a job type.

get(request, name, version, revision_num)

Retrieves the details for a job type version and return them in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the job type
  • version (string) – The version of the job type
  • revision_num (int encoded as a str) – The revision number of the job type
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
queryset
serializer_class

alias of JobTypeRevisionDetailsSerializerV6

class job.views.JobTypeRevisionsView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving revisions of a job type.

list(request, name, version)

Retrieves the list of versions for a job type with the given name and return them in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the job type
  • version (string) – The version of the job type
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
queryset
serializer_class

alias of JobTypeRevisionSerializerV6

class job.views.JobTypeVersionsView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving versions of a job type.

list(request, name)

Retrieves the list of versions for a job type with the given name and return them in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the job type
Return type:

rest_framework.response.Response
Returns:

the HTTP response to send back to the user
queryset
serializer_class

alias of JobTypeSerializerV6

class job.views.JobTypesPendingView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving the status of all currently pending job types.

get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

list(request)

Retrieves the current status of pending job types and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class job.views.JobTypesRunningView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving the status of all currently running job types.

get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

list(request)

Retrieves the current status of running job types and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class job.views.JobTypesStatusView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving overall job type status information.

get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

list(request)

Retrieves the list of all job types with status and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class job.views.JobTypesSystemFailuresView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for viewing system errors organized by job type.

get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

list(request)

Retrieves the job types that have failed with system errors and returns them in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class job.views.JobTypesValidationView(**kwargs)

Bases: rest_framework.views.APIView

This view is the endpoint for validating a new job type before attempting to actually create it

post(request)

Validates a new job type and returns any warnings discovered

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class job.views.JobTypesView(**kwargs)

Bases: rest_framework.generics.ListCreateAPIView

This view is the endpoint for retrieving the list of all job types.

create(request)

Creates a new job type and returns a link to the detail URL

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
create_v6(request)

Creates or edits a Seed job type and returns a link to the detail URL

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
list(request)

Retrieves the list of all job types and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
serializer_class

alias of JobTypeSerializerV6

class job.views.JobsView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving a list of all available jobs.

list(request)

Retrieves jobs and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
post(request)

Creates a new job, places it on the queue, and returns the new job information in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
serializer_class

alias of JobSerializerV6

class job.views.RequeueJobsView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for re-queuing jobs that have failed or been canceled

parser_classes = (<class 'rest_framework.parsers.JSONParser'>,)
post(request)

Submit command message to re-queue jobs that fit the given filter criteria

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Returns:the HTTP response to send back to the user
queryset
serializer_class

alias of JobSerializerV6

Module contents