recipe package

Subpackages

Submodules

recipe.apps module

Defines the application configuration for the recipe application

class recipe.apps.RecipeConfig(app_name, app_module)

Bases: django.apps.config.AppConfig

Configuration for the recipe app

label = u'recipe'
name = u'recipe'
ready()

Registers components related to recipes

verbose_name = u'Recipe'

recipe.exceptions module

Defines exceptions that can occur when interacting with recipes and recipe types

exception recipe.exceptions.CreateRecipeError

Bases: exceptions.Exception

Exception indicating that a recipe cannot be created

exception recipe.exceptions.InactiveRecipeType

Bases: exceptions.Exception

Exception indicating that a recipe type is inactive

exception recipe.exceptions.ReprocessError

Bases: exceptions.Exception

Exception indicating that a reprocessing request cannot be completed

exception recipe.exceptions.SupersedeError

Bases: exceptions.Exception

Exception indicating that a recipe cannot be superseded

recipe.models module

Defines the database models for recipes and recipe types

class recipe.models.ForcedNodesValidation(is_valid, errors, warnings, forced_nodes)

Bases: tuple

errors

Alias for field number 1

forced_nodes

Alias for field number 3

is_valid

Alias for field number 0

warnings

Alias for field number 2

class recipe.models.Recipe(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a recipe to be run on the cluster. A model lock must be obtained using select_for_update() on any recipe model before adding new jobs to it or superseding it.

Parameters:
  • recipe_type (django.db.models.ForeignKey) – The type of this recipe
  • recipe_type_rev (django.db.models.ForeignKey) – The revision of the recipe type when this recipe was created
  • event (django.db.models.ForeignKey) – The event that triggered the creation of this recipe
  • ingest_event (django.db.models.ForeignKey) – The ingest event that triggered the creation of this recipe
  • root_recipe (django.db.models.ForeignKey) – The root recipe that contains this recipe
  • recipe (django.db.models.ForeignKey) – The original recipe that created this recipe
  • batch (django.db.models.ForeignKey) – The batch that contains this recipe
  • is_superseded (django.db.models.BooleanField) – Whether this recipe has been superseded and is obsolete. This may be true while superseded_by_recipe (the reverse relationship of superseded_recipe) is null, indicating that this recipe is obsolete, but there is no new recipe that has directly taken its place.
  • root_superseded_recipe (django.db.models.ForeignKey) – The first recipe in the chain of superseded recipes. This field will be null for the first recipe in the chain (i.e. recipes that have a null superseded_recipe field).
  • superseded_recipe (django.db.models.ForeignKey) – The recipe that was directly superseded by this recipe. The reverse relationship can be accessed using ‘superseded_by_recipe’.
  • input (django.contrib.postgres.fields.JSONField) – JSON description defining the input for this recipe
  • input_file_size (django.db.models.FloatField) – The total size in MiB for all input files in this recipe
  • configuration (django.contrib.postgres.fields.JSONField) – JSON describing the overriding recipe configuration for this recipe instance
  • source_started (django.db.models.DateTimeField) – The start time of the source data for this recipe
  • source_ended (django.db.models.DateTimeField) – The end time of the source data for this recipe
  • source_sensor_class (django.db.models.CharField) – The class of sensor used to produce the source file for this recipe
  • source_sensor (django.db.models.CharField) – The specific identifier of the sensor used to produce the source file for this recipe
  • source_collection (django.db.models.CharField) – The collection of the source file for this recipe
  • source_task (django.db.models.CharField) – The task that produced the source file for this recipe
  • jobs_total (django.db.models.IntegerField) – The total count of all jobs within this recipe
  • jobs_pending (django.db.models.IntegerField) – The count of all PENDING jobs within this recipe
  • jobs_blocked (django.db.models.IntegerField) – The count of all BLOCKED jobs within this recipe
  • jobs_queued (django.db.models.IntegerField) – The count of all QUEUED jobs within this recipe
  • jobs_running (django.db.models.IntegerField) – The count of all RUNNING jobs within this recipe
  • jobs_failed (django.db.models.IntegerField) – The count of all FAILED jobs within this recipe
  • jobs_completed (django.db.models.IntegerField) – The count of all COMPLETED within this recipe
  • jobs_canceled (django.db.models.IntegerField) – The count of all CANCELED jobs within this recipe
  • sub_recipes_total (django.db.models.IntegerField) – The total count for all sub-recipes within this recipe
  • sub_recipes_completed (django.db.models.IntegerField) – The count for all completed sub-recipes within this recipe
  • is_completed (django.db.models.BooleanField) – Whether this recipe has completed all of its jobs
  • created (django.db.models.DateTimeField) – When the recipe was created
  • completed (django.db.models.DateTimeField) – When every job in the recipe was completed successfully
  • superseded (django.db.models.DateTimeField) – When this recipe was superseded
  • last_modified (django.db.models.DateTimeField) – When the recipe was last modified
ALPHABETIZE_FIELDS = [u'recipe_type.name']
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

batch

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

batch_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

completed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

conditions_for_recipe

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

conditions_for_root_recipe

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

configuration

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

contained_by

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

contains

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

event

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

event_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_definition()

Returns the definition for this recipe

Returns:The definition for this recipe
Return type:recipe.definition.definition.RecipeDefinition
get_input_data()

Returns the input data for this recipe

Returns:The input data for this recipe
Return type:data.data.data.Data
get_next_by_created(*moreargs, **morekwargs)
get_next_by_last_modified(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_last_modified(*moreargs, **morekwargs)
get_v6_input_data_json()

Returns the input data for this recipe as v6 json with the version stripped

Returns:The v6 JSON input data dict for this recipe
Return type:dict
get_v6_recipe_instance_json()

Returns the recipe instance details as json

Returns:The v6 JSON instance details dict for this recipe
Return type:dict
has_input()

Indicates whether this recipe has its input

Returns:True if the recipe has its input, false otherwise.
Return type:bool
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

ingest_event

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

ingest_event_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

input

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

input_file_size

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_completed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_superseded

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobexecution_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobs_blocked

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_canceled

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_completed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_failed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_for_recipe

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobs_for_root_recipe

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

jobs_pending

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_queued

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_running

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

jobs_total

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

last_modified

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <recipe.models.RecipeManager object>
queue_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipe_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipe_type_rev

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_type_rev_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipeinputfile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

root_recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

root_recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

root_superseded_recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

root_superseded_recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

scalefile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

source_collection

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_ended

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_sensor

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_sensor_class

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_started

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

source_task

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

sub_recipes

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

sub_recipes_completed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

sub_recipes_for_root

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

sub_recipes_total

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

superseded

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

superseded_by_recipe

Accessor to the related object on the reverse side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

place.restaurant is a ReverseOneToOneDescriptor instance.

superseded_by_recipes

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

superseded_recipe

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

restaurant.place is a ForwardOneToOneDescriptor instance.

superseded_recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class recipe.models.RecipeCondition(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a conditional decision within a recipe. If the condition is accepted then the dependent nodes will be created and processed, while if the condition is not accepted the dependent nodes will never be created.

Parameters:
  • root_recipe (django.db.models.ForeignKey) – The root recipe that contains this condition
  • recipe (django.db.models.ForeignKey) – The original recipe that created this condition
  • batch (django.db.models.ForeignKey) – The batch that contains this condition
  • data (django.contrib.postgres.fields.JSONField) – JSON description defining the data processed by this condition
  • is_processed (django.db.models.BooleanField) – Whether the condition has been processed
  • is_accepted (django.db.models.BooleanField) – Whether the condition has been accepted
  • created (django.db.models.DateTimeField) – When this condition was created
  • processed (django.db.models.DateTimeField) – When this condition was processed
  • last_modified (django.db.models.DateTimeField) – When the condition was last modified
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

batch

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

batch_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

data

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_data()

Returns the data for this condition

Returns:The data for this condition
Return type:data.data.data.Data
get_next_by_created(*moreargs, **morekwargs)
get_next_by_last_modified(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_last_modified(*moreargs, **morekwargs)
has_data()

Indicates whether this condition has its data

Returns:True if the condition has its data, false otherwise.
Return type:bool
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_accepted

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_processed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

last_modified

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <recipe.models.RecipeConditionManager object>
processed

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipenode_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

root_recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

root_recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class recipe.models.RecipeConditionManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling recipe conditions

create_condition(recipe_id, root_recipe_id=None, batch_id=None)

Creates a new condition for the given recipe and returns the (unsaved) condition model

Parameters:
  • recipe_id (int) – The ID of the original recipe that created this condition, possibly None
  • root_recipe_id (int) – The ID of the root recipe that contains this condition, possibly None
  • batch_id (int) – The ID of the batch that contains this condition, possibly None
Returns:

The new condition model

Return type:

recipe.models.RecipeCondition

get_condition_with_interfaces(condition_id)

Gets the condition model for the given ID with related recipe__recipe_type_rev model

Parameters:condition_id (int) – The condition ID
Returns:The condition model with related recipe__recipe_type_rev model
Return type:job.models.Job
set_condition_data_v6(condition, data, node_name)

Sets the given data as a v6 JSON for the given condition. The condition model must have its related recipe__recipe_type_rev model populated.

Parameters:

:raises data.data.exceptions.InvalidData: If the data is invalid

set_processed(condition_id, is_accepted)

Sets the condition with the given ID as being processed

Parameters:
  • condition_id (int) – The condition ID
  • is_accepted (bool) – Whether the condition was accepted
class recipe.models.RecipeInputFile(*args, **kwargs)

Bases: django.db.models.base.Model

Links a recipe and its input files together. A file can be used as input to multiple recipes and a recipe can accept multiple input files. This model is useful for determining relevant recipes to run during re-processing.

Parameters:
  • recipe (django.db.models.ForeignKey) – The recipe that the input file is linked to
  • scale_file (django.db.models.ForeignKey) – The input file that the recipe is linked to
  • recipe_input (django.db.models.CharField) – The name of the recipe input parameter
  • created (django.db.models.DateTimeField) – When the recipe was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

input_file

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

input_file_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <recipe.models.RecipeInputFileManager object>
recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

recipe_input

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class recipe.models.RecipeInputFileManager

Bases: django.db.models.manager.Manager

Provides additional methods for handleing RecipeInputFiles

get_recipe_input_files_v6(recipe_id, started=None, ended=None, time_field=None, file_name=None, recipe_input=None)

Returns a query for Input Files filtered on the given fields.

Parameters:
  • recipe_id (int) – The recipe ID
  • started (datetime.datetime) – Query Scale files updated after this amount of time.
  • ended (datetime.datetime) – Query Scale files updated before this amount of time.
  • time_field (string) – The time field to use for filtering.
  • file_name (str) – Query Scale files with the given file name.
  • recipe_input (str) – The name of the recipe input that the file was passed into
Returns:

The Scale file query

Return type:

django.db.models.QuerySet

class recipe.models.RecipeManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling recipes

complete_recipes(recipe_ids, when)

Marks the recipes with the given IDs as being completed

Parameters:
  • recipe_ids (list()) – The recipe IDs
  • when (datetime.datetime) – The time that the recipes were completed
create_recipe_v6(recipe_type_rev, event_id=None, ingest_id=None, input_data=None, root_recipe_id=None, recipe_id=None, recipe_config=None, batch_id=None, superseded_recipe=None, copy_superseded_input=False)

Creates a new recipe for the given recipe type revision and returns the (unsaved) recipe model

Parameters:
  • recipe_type_rev (recipe.models.RecipeTypeRevision) – The recipe type revision (with populated recipe_type model) of the recipe to create
  • event_id (int) – The event ID that triggered the creation of this recipe
  • ingest_id (int) – The ingest event ID that triggered the creation of this recipe
  • input_data (data.data.data.Data) – The recipe’s input data, possibly None
  • root_recipe_id (int) – The ID of the root recipe that contains this sub-recipe, possibly None
  • recipe_id (int) – The ID of the original recipe that created this sub-recipe, possibly None
  • batch_id (int) – The ID of the batch that contains this recipe, possibly None
  • recipe_config (recipe.configuration.configuration.RecipeConfiguration) – The configuration for running this recipe, possibly None
  • superseded_recipe (recipe.models.Recipe) – The recipe that the created recipe is superseding, possibly None
  • copy_superseded_input (bool) – Whether to copy the input data from the superseded recipe
Returns:

The new recipe model

Return type:

recipe.models.Recipe

:raises data.data.exceptions.InvalidData: If the input data is invalid :raises recipe.exceptions.InactiveRecipeType: If the recipe type is inactive

get_details(recipe_id)

Gets the details for a given recipe including its associated jobs and input files.

Parameters:recipe_id (:int) – The unique identifier of the recipe to fetch.
Returns:A recipe with additional information.
Return type:recipe.models.Recipe
get_locked_recipe(recipe_id)

Locks and returns the recipe model for the given ID with no related fields. Caller must be within an atomic transaction.

Parameters:recipe_id (int) – The recipe ID
Returns:The recipe model
Return type:recipe.models.Recipe
get_locked_recipes(recipe_ids)

Locks and returns the recipe models for the given IDs with no related fields. Caller must be within an atomic transaction.

Parameters:recipe_ids (list()) – The recipe IDs
Returns:The recipe models
Return type:list()
get_locked_recipes_from_root(root_recipe_ids)

Locks and returns the latest (non-superseded) recipe model for each recipe family with the given root recipe IDs. The returned models have no related fields populated. Caller must be within an atomic transaction.

Parameters:root_recipe_ids (list()) – The root recipe IDs
Returns:The recipe models
Return type:list()
get_recipe_for_job(job_id)

Returns the original recipe for the job with the given ID (returns None if the job is not in a recipe). The returned model will have its related recipe_type and recipe_type_rev models populated. If the job exists in multiple recipes due to superseding, the original (first) recipe is returned.

Parameters:job_id (int) – The job ID
Returns:The recipe_job model with related recipe_type and recipe_type-rev, possibly None
Return type:recipe.models.RecipeNode
get_recipe_ids_for_jobs(job_ids)

Returns the IDs of all recipes that contain the jobs with the given IDs. This will include superseded recipes.

Parameters:job_ids (list()) – The job IDs
Returns:The recipe IDs
Return type:list()
get_recipe_ids_for_sub_recipes(sub_recipe_ids)

Returns the IDs of all recipes that contain the sub-recipes with the given IDs. This will include superseded recipes.

Parameters:sub_recipe_ids (list()) – The sub-recipe IDs
Returns:The recipe IDs
Return type:list()
get_recipe_instance(recipe_id)

Returns the recipe instance for the given recipe ID

Parameters:recipe_id (int) – The recipe ID
Returns:The recipe instance
Return type:recipe.instance.recipe.RecipeInstance
get_recipe_instance_from_root(root_recipe_id)

Returns the non-superseded recipe instance for the given root recipe ID

Parameters:root_recipe_id (int) – The root recipe ID
Returns:The recipe instance
Return type:recipe.instance.recipe.RecipeInstance
get_recipe_with_interfaces(recipe_id)

Gets the recipe model for the given ID with related recipe_type_rev and recipe__recipe_type_rev models

Parameters:recipe_id (int) – The recipe ID
Returns:The recipe model with related recipe_type_rev and recipe__recipe_type_rev models
Return type:recipe.models.Recipe
get_recipes_v6(started=None, ended=None, source_started=None, source_ended=None, source_sensor_classes=None, source_sensors=None, source_collections=None, source_tasks=None, ids=None, type_ids=None, type_names=None, batch_ids=None, is_superseded=None, is_completed=None, order=None, root_recipe_ids=None)

Returns a list of recipes within the given time range.

Parameters:
  • started (datetime.datetime) – Query recipes updated after this amount of time.
  • ended (datetime.datetime) – Query recipes updated before this amount of time.
  • source_started (datetime.datetime) – Query recipes where source collection started after this time.
  • source_ended (datetime.datetime) – Query recipes where source collection ended before this time.
  • source_sensor_classes (list()) – Query recipes with the given source sensor class.
  • source_sensor (list()) – Query recipes with the given source sensor.
  • source_collection (list()) – Query recipes with the given source class.
  • source_tasks (list()) – Query recipes with the given source tasks.
  • ids ([int]) – Query recipes associated with the given identifiers.
  • type_ids ([int]) – Query recipes of the type associated with the identifiers.
  • type_names ([string]) – Query recipes of the type associated with the name.
  • batch_ids ([int]) – Query recipes associated with batches with the given identifiers.
  • is_superseded (bool) – Query recipes that match the is_superseded flag.
  • is_completed (bool) – Query recipes that match the is_completed flag.
  • order ([string]) – A list of fields to control the sort order.
  • root_recipe_ids ([int]) – Query recipes with the listed parent recipe ids
Returns:

The list of recipes that match the time range.

Return type:

[recipe.models.Recipe]

process_recipe_input(recipe)

Processes the input data for the given recipe to populate its input file models and input meta-data fields. The caller must have obtained a model lock on the given recipe model.

Parameters:recipe (recipe.models.Recipe) – The locked recipe models
set_recipe_input_data_v6(recipe, input_data)

Sets the given input data as a v6 JSON for the given recipe. The recipe model must have its related recipe_type_rev model populated.

Parameters:

:raises data.data.exceptions.InvalidData: If the data is invalid

supersede_recipes(recipe_ids, when)

Updates the given recipes to be superseded

Parameters:
  • recipe_ids (list()) – The recipe IDs to supersede
  • when (datetime.datetime) – The time that the recipes were superseded
update_recipe_metrics(recipe_ids)

Updates the metrics for the recipes with the given IDs

Parameters:recipe_ids (list()) – The recipe IDs
class recipe.models.RecipeNode(*args, **kwargs)

Bases: django.db.models.base.Model

Links a recipe with a node within that recipe. Nodes within a recipe may represent either a job or another recipe. The same node may exist in multiple recipes due to superseding. For an original node and recipe combination, the is_original flag is True. When recipe B supersedes recipe A, the non-superseded nodes from recipe A that are being copied to recipe B will have models with is_original set to False.

Parameters:
  • recipe (django.db.models.ForeignKey) – The recipe that contains the node
  • node_name (django.db.models.CharField) – The unique name of the node within the recipe
  • is_original (django.db.models.BooleanField) – Whether this is the original recipe for the node (True) or the node is copied from a superseded recipe (False)
  • condition (django.db.models.ForeignKey) – If not null, this node is a condition node and this field is the condition within the recipe
  • job (django.db.models.ForeignKey) – If not null, this node is a job node and this field is the job that the recipe contains
  • sub_recipe (django.db.models.ForeignKey) – If not null, this node is a recipe node and this field is the sub-recipe that the recipe contains
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

condition

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

condition_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_original

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

node_name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <recipe.models.RecipeNodeManager object>
recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

sub_recipe

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

sub_recipe_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class recipe.models.RecipeNodeCopy(superseded_recipe_id, recipe_id, node_names)

Bases: tuple

node_names

Alias for field number 2

recipe_id

Alias for field number 1

superseded_recipe_id

Alias for field number 0

class recipe.models.RecipeNodeManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling jobs linked to a recipe

copy_recipe_nodes(recipe_copies)

Copies the given nodes from the superseded recipes to the new recipes

Parameters:recipe_copies (list()) – A list of RecipeNodeCopy tuples
count_subrecipes(recipe_id, recurse=False)

Counts the number of sub-recipes that belong to the given recipe :param recipe_id: The recipe to count :type recipe_id: int :param recurse: Recurses into any sub-recieps if true :type recurse: bool :returns: The number of sub-recipes associated with the recipe_id :rtype: int

create_recipe_condition_nodes(recipe_id, conditions)

Creates and returns the recipe node models (unsaved) for the given recipe and conditions

Parameters:
  • recipe_id (int) – The recipe ID
  • conditions (dict) – A dict of condition models stored by node name
Returns:

The list of recipe_node models

Return type:

list()

create_recipe_job_nodes(recipe_id, recipe_jobs)

Creates and returns the recipe node models (unsaved) for the given recipe and jobs

Parameters:
  • recipe_id (int) – The recipe ID
  • recipe_jobs (dict) – A dict of job models stored by node name
Returns:

The list of recipe_node models

Return type:

list()

create_subrecipe_nodes(recipe_id, sub_recipes)

Creates and returns the recipe node models (unsaved) for the given recipe and sub-recipes

Parameters:
  • recipe_id (int) – The recipe ID
  • sub_recipes (dict) – A dict of recipe models stored by node name
Returns:

The list of recipe_node models

Return type:

list()

get_recipe_jobs(recipe_id)

Returns the job models that belong to the given recipe

Parameters:recipe_id (int) – The recipe ID
Returns:A dict of job models stored by node name
Return type:dict
get_recipe_node_outputs(recipe_id)

Returns the output data for each recipe node for the given recipe ID

Parameters:recipe_id (int) – The recipe ID
Returns:The RecipeNodeOutput tuples stored in a dict by node name
Return type:dict
get_recipe_nodes(recipe_id)

Returns the recipe_node models with related condition, job, and sub_recipe models for the given recipe ID

Parameters:recipe_id (int) – The recipe ID
Returns:The recipe_node models for the recipe
Return type:list()
get_subrecipes(recipe_id)

Returns the sub-recipe models that belong to the given recipe

Parameters:recipe_id (int) – The recipe ID
Returns:A dict of recipe models stored by node name
Return type:dict
supersede_recipe_jobs(recipe_ids, when, node_names, all_nodes=False)

Supersedes the jobs for the given recipe IDs and node names

Parameters:
  • recipe_ids (list()) – The recipe IDs
  • when (datetime.datetime) – The time that the jobs were superseded
  • node_names (list()) – The node names of the jobs to supersede
  • all_nodes (bool) – Whether all nodes should be superseded
supersede_subrecipes(recipe_ids, when, node_names, all_nodes=False)

Supersedes the sub-recipes for the given recipe IDs and node names

Parameters:
  • recipe_ids (list()) – The recipe IDs
  • when (datetime.datetime) – The time that the sub-recipes were superseded
  • node_names (list()) – The node names of the sub-recipes to supersede
  • all_nodes (bool) – Whether all nodes should be superseded
class recipe.models.RecipeNodeOutput(node_name, node_type, id, output_data)

Bases: tuple

id

Alias for field number 2

node_name

Alias for field number 0

node_type

Alias for field number 1

output_data

Alias for field number 3

class recipe.models.RecipeType(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a type of recipe that can be run on the cluster. Any updates to a recipe type model requires obtaining a lock on the model using select_for_update().

Parameters:
  • name (django.db.models.CharField) – The identifying name of the recipe type used by clients for queries
  • title (django.db.models.CharField) – The human-readable name of the recipe type
  • description (django.db.models.CharField) – An optional description of the recipe type
  • is_system (django.db.models.BooleanField) – Whether this is a system recipe type
  • is_active (django.db.models.BooleanField) – Whether the recipe type is active (false once recipe type is archived)
  • definition (django.contrib.postgres.fields.JSONField) – JSON definition for running a recipe of this type
  • revision_num (django.db.models.IntegerField) – The current revision number of the definition, starts at one
  • created (django.db.models.DateTimeField) – When the recipe type was created
  • deprecated (django.db.models.DateTimeField) – When the recipe type was deprecated (no longer active)
  • last_modified (django.db.models.DateTimeField) – When the recipe type was last modified
ALPHABETIZE_FIELDS = [u'name', u'title', u'description']
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

batch_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

definition

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

deprecated

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

description

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_definition()

Returns the definition for this recipe type

Returns:The definition for this recipe type
Return type:recipe.definition.definition.RecipeDefinition
get_next_by_created(*moreargs, **morekwargs)
get_next_by_last_modified(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
get_previous_by_last_modified(*moreargs, **morekwargs)
get_v6_definition_json()

Returns the recipe type definition in v6 of the JSON schema

Returns:The recipe type definition in v6 of the JSON schema
Return type:dict
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_active

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

is_system

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

last_modified

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

natural_key()

Django method to define the natural key for a recipe type as the recipe type name

Returns:A string representing the natural key
Return type:tuple(string)
objects = <recipe.models.RecipeTypeManager object>
parent_recipe_type

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

recipe_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

recipe_types_for_job_type

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

recipetyperevision_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

revision_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

scalefile_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

sub_recipe_type

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

title

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

Bases: django.db.models.base.Model

Represents a link between a recipe type and a job type.

Parameters:
  • recipe_type (django.db.models.ForeignKey) – The related recipe type
  • job_type (django.db.models.ForeignKey) – The related job type
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

job_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

job_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <recipe.models.RecipeTypeJobLinkManager object>
recipe_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class recipe.models.RecipeTypeJobLinkManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling recipe type to job type links

Creates the appropriate link for the given recipe and job type. All database changes are made in an atomic transaction.

Parameters:
  • recipe_type_id (int) – recipe type ID
  • job_type_id (int) – job type ID.

Creates the appropriate links for the given recipe and job types. All database changes are made in an atomic transaction.

Parameters:
  • recipe_type_ids ([int]) – List of recipe type IDs
  • job_type_ids ([int]) – List of job type IDs.

Goes through a recipe type definition and gets all the job types it contains and creates the appropriate links

Parameters:recipe_type (recipe.models.RecipeType) – New/updated recipe type

:raises recipe.models.JobType.DoesNotExist: If it contains a job type that does not exist

get_job_type_ids(recipe_type_ids)

Returns a list of the job type IDs for the given recipe type IDs.

Parameters:recipe_type_ids (list()) – The recipe type IDs
Returns:The list of job type IDs
Return type:list()
get_recipe_type_ids(job_type_ids)

Returns a list of recipe_type IDs for the given job type IDs.

Parameters:job_type_ids (list()) – The sub recipe type IDs
Returns:The list of recipe type IDs
Return type:list()
class recipe.models.RecipeTypeManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling recipe types

create_recipe_type_v6(name, title, description, definition)

Creates a new recipe type and saves it in the database. All database changes occur in an atomic transaction.

Parameters:
  • name (str) – The system name of the recipe type
  • title (str) – The human-readable name of the recipe type
  • description (str) – An optional description of the recipe type
  • definition (recipe.definition.definition.RecipeDefinition) – The definition for running a recipe of this type
Returns:

The new recipe type

Return type:

recipe.models.RecipeType

:raises recipe.definition.exceptions.InvalidDefinition: If any part of the recipe
definition violates the specification
edit_recipe_type_v6(recipe_type_id, title=None, description=None, definition=None, auto_update=None, is_active=None)

Edits the given recipe type and saves the changes in the database. All database changes occur in an atomic transaction. An argument of None for a field indicates that the field should not change.

Parameters:
  • recipe_type_id (int) – The unique identifier of the recipe type to edit
  • title (str) – The human-readable name of the recipe type, possibly None
  • description (str) – A description of the recipe type, possibly None
  • definition (recipe.definition.definition.RecipeDefinition) – The definition for running a recipe of this type, possibly None
  • auto_update (bool) – If true, recipes that contain this recipe type will automatically be updated
  • is_active (bool) – If true, make this recipe type active; if false, recipes of this type will no longer be created
:raises recipe.definition.exceptions.InvalidDefinition: If any part of the recipe
definition violates the specification
get_by_natural_key(name)

Django method to retrieve a recipe type for the given natural key

Parameters:name (string) – The human-readable name of the recipe type
Returns:The recipe type defined by the natural key
Return type:recipe.models.RecipeType
get_details_v6(name)

Gets additional details for the given recipe type model based on related model attributes.

The additional fields include: job_types, sub_recipe_types.

Parameters:name (string) – The unique recipe type name.
Returns:The recipe type with extra related attributes.
Return type:recipe.models.RecipeType
get_interfaces(definition)

Gets the input and output interfaces for each node in this recipe

Returns:A dict of input interfaces and a dict of output interfaces
Return type:dict, dict
:raises recipe.definition.exceptions.InvalidDefinition: If any part of the recipe
definition violates the specification
get_recipe_types_v6(keywords=None, is_active=None, is_system=None, order=None)

Returns a list of recipe types within the given time range.

Parameters:
  • keywords (list()) – Query recipe types with name, title, description or tag matching a keyword
  • is_active (bool) – Query recipe types that are actively available for use.
  • is_system (bool) – Query recipe types that are system recipe types.
  • order (list()) – A list of fields to control the sort order.
Returns:

The list of recipe types that match the given parameters.

Return type:

list()

get_timeline_recipes_json(started=None, ended=None, type_ids=None, type_names=None, revisions=None)

Returns the timeline recipe type information

Parameters:
  • started (datetime.datetime) – The start date of the timeline
  • ended (datetime.datetime) – The end date of the timeline
  • type_ids ([int]) – List of recipe type ids to limit the results
  • type_names ([string]) – List of recipe type names to limit the results
  • revisions ([int]) – List of revisions to limit the results
Returns:

JSON timeline response

Return type:

dict

validate_recipe_type_v6(name, definition_dict)

Validates a recipe type prior to attempting a save

Parameters:
  • name (str) – The optional system name of a recipe type being updated
  • definition_dict (dict) – The definition for running a recipe of this type
Returns:

The recipe type validation.

Return type:

recipe.models.RecipeTypeValidation

validate_recursive_subs(recipe_type_name, definition, source_recipe_type_name=None)

Validates the sub-recipes to ensure we don’t end up in a recipeception loop :param recipe_type_name: The name of the recipe type we’re looking for :type recipe_type_name: String :param definition: The RecipeDefinition we’re searching :type definition: recipe.definition.definition.RecipeDefinition :param source_recipe_type_name: The recipe type we’re searching the definition of :type source_recipe_type_name: String

Raises:InvalidDefinition if a recipe type contains a sub-recipe of itself
class recipe.models.RecipeTypeRevision(*args, **kwargs)

Bases: django.db.models.base.Model

Represents a revision of a recipe type. New revisions are created when the definition of a recipe type changes. Any inserts of a recipe type revision model requires obtaining a lock using select_for_update() on the corresponding recipe type model.

Parameters:
  • recipe_type (django.db.models.ForeignKey) – The recipe type for this revision
  • revision_num (django.db.models.IntegerField) – The number for this revision, starting at one
  • definition (django.contrib.postgres.fields.JSONField) – The JSON definition for this revision of the recipe type
  • created (django.db.models.DateTimeField) – When this revision was created
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

batch_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

created

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

definition

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_definition()

Returns the definition for this recipe type revision

Returns:The definition for this revision
Return type:recipe.definition.definition.RecipeDefinition
get_input_interface()

Returns the input interface for this revision

Returns:The input interface for this revision
Return type:data.interface.interface.Interface
get_next_by_created(*moreargs, **morekwargs)
get_previous_by_created(*moreargs, **morekwargs)
get_v6_definition_json()

Returns the revision definition in v6 of the JSON schema

Returns:The revision definition in v6 of the JSON schema
Return type:dict
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

natural_key()

Django method to define the natural key for a recipe type revision as the combination of recipe type and revision number

Returns:A tuple representing the natural key
Return type:tuple(string, int)
objects = <recipe.models.RecipeTypeRevisionManager object>
recipe_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

recipe_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

revision_num

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

validate_forced_nodes(forced_nodes_json)

Validates a forced nodes object against the definition for this recipe type revision

Parameters:forced_nodes_json (dict) – The definition of nodes to be forced to reprocess
Returns:The ForcedNodes validation.
Return type:recipe.models.ForcedNodesValidation
class recipe.models.RecipeTypeRevisionManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling recipe type revisions

create_recipe_type_revision(recipe_type)

Creates a new revision for the given recipe type. The recipe type’s definition and revision number must already be updated. The caller must have obtained a lock using select_for_update() on the given recipe type model.

Parameters:recipe_type (recipe.models.RecipeType) – The recipe type
get_by_natural_key(recipe_type, revision_num)

Django method to retrieve a recipe type revision for the given natural key

Parameters:
Returns:

The recipe type revision defined by the natural key

Return type:

recipe.models.RecipeTypeRevision

get_revision(name, revision_num)

Returns the revision (with populated recipe_type model) for the given recipe type and revision number

Parameters:
  • name (string) – The name of the recipe type
  • revision_num (int) – The revision number
Returns:

The revision

Return type:

recipe.models.RecipeTypeRevision

get_revision_map(revision_ids, revision_tuples)

Returns a dict that maps revision ID to recipe type revision for the recipe type revisions that match the given values. Each revision model will have its related recipe type model populated.

Parameters:
  • revision_ids (list()) – A list of revision IDs to return
  • revision_tuples (list()) – A list of tuples (recipe type name, revision num) for additional revisions to return
Returns:

The revisions stored by revision ID

Return type:

dict

get_revisions(name)

Returns the revision (with populated recipe_type model) for the given recipe type and revision number

Parameters:name (string) – The name of the recipe type
Returns:The recipe type revisions for the given recipe type name
Return type:[recipe.models.RecipeTypeRevision]

Bases: django.db.models.base.Model

Represents a link between a recipe type and a sub-recipe type.

Parameters:
  • recipe_type (django.db.models.ForeignKey) – The related recipe type
  • sub_recipe_type (django.db.models.ForeignKey) – The related sub recipe type
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <recipe.models.RecipeTypeSubLinkManager object>
recipe_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

recipe_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

sub_recipe_type

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

child.parent is a ForwardManyToOneDescriptor instance.

sub_recipe_type_id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class recipe.models.RecipeTypeSubLinkManager

Bases: django.db.models.manager.Manager

Provides additional methods for handling recipe type sub links

count_subrecipes(recipe_type_id, recurse)

Counts the number of sub-recipes within the specified recipe type

Creates the appropriate link for the given recipe and job type. All database changes are made in an atomic transaction.

Parameters:
  • recipe_type_id (int) – recipe type ID
  • sub_recipe_type_id (int) – sub recipe type ID.

Creates the appropriate links for the given parent and child recipe types. All database changes are made in an atomic transaction.

Parameters:
  • recipe_type_ids ([int]) – List of parent recipe type IDs
  • sub_recipe_type_ids ([int]) – List of child recipe type IDs.

Goes through a recipe type definition, gets all the recipe types it contains and creates the appropriate links

Parameters:recipe_type (recipe.models.RecipeType) – New/updated recipe type

:raises recipe.models.RecipeType.DoesNotExist: If it contains a sub recipe type that does not exist

get_recipe_type_ids(sub_recipe_type_ids)

Returns a list of the parent recipe_type IDs for the given sub recipe type IDs.

Parameters:sub_recipe_type_ids (list()) – The sub recipe type IDs
Returns:The list of parent recipe type IDs
Return type:list()
get_sub_recipe_type_ids(recipe_type_ids)

Returns a list of the sub recipe type IDs for the given recipe type IDs.

Parameters:recipe_type_ids (list()) – The recipe type IDs
Returns:The list of sub recipe type IDs
Return type:list()
class recipe.models.RecipeTypeValidation(is_valid, errors, warnings, diff)

Bases: tuple

diff

Alias for field number 3

errors

Alias for field number 1

is_valid

Alias for field number 0

warnings

Alias for field number 2

recipe.serializers module

class recipe.serializers.RecipeBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts recipe model fields to REST output.

class recipe.serializers.RecipeDetailsInputSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts recipe detail model input fields to REST output

to_representation(obj)
class recipe.serializers.RecipeDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeSerializerV6

Converts related recipe model fields to REST output.

class IngestEventDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: ingest.ingest_event_serializers.IngestEventBaseSerializerV6

Converts ingest event model fields to REST output

class JobTypeBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts job type model fields to REST output

class TriggerEventDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: trigger.serializers.TriggerEventBaseSerializerV6

Converts trigger event model fields to REST output.

class recipe.serializers.RecipeJobsDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeJobsSerializerV6

Converts related recipe model fields to REST output.

class JobRevisionSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobSerializerV6

Converts job model fields to REST output.

class recipe.serializers.RecipeJobsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: rest_framework.serializers.Serializer

Converts recipe model fields to REST output.

class JobSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.serializers.JobBaseSerializerV6

Converts job model fields to REST output.

class BatchBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Base serializer for batches

class ErrorBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts error model fields to REST output

class IngestEventSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: ingest.ingest_event_serializers.IngestEventBaseSerializerV6

Converts ingest event model fields to REST output

class RecipeBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts recipe model fields to REST output.

class TriggerEventSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: trigger.serializers.TriggerEventBaseSerializerV6

Converts trigger event model fields to REST output.

class recipe.serializers.RecipeSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeBaseSerializerV6

Converts recipe model fields to REST output.

class BatchBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Base serializer for batches

class IngestEventBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts ingest event model fields to REST output

class TriggerEventBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts trigger event model fields to REST output.

superseded_by_recipe = None
class recipe.serializers.RecipeTypeBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Base serializer for recipe types

class recipe.serializers.RecipeTypeDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeSerializerV6

Converts recipe type model fields to REST output.

class JobTypeDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: job.job_type_serializers.JobTypeSerializerV6

Converts job type model fields to REST output.

class RecipeTypeSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeBaseSerializerV6

Converts recipe type model fields to REST output.

definition = None
job_types = None
sub_recipe_types = None
super_recipe_types = None
class recipe.serializers.RecipeTypeListSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

List serializer for recipe types

super_recipe_types = None
class recipe.serializers.RecipeTypeRevisionBaseSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Converts recipe type revision model fields to REST output.

class recipe.serializers.RecipeTypeRevisionBaseSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: util.rest.ModelIdSerializer

Base serializer for recipe type revisions

class recipe.serializers.RecipeTypeRevisionDetailsSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeRevisionSerializerV6

Serializer for recipe type revisions

class recipe.serializers.RecipeTypeRevisionSerializer(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeRevisionBaseSerializer

Converts recipe type revision model fields to REST output.

class recipe.serializers.RecipeTypeRevisionSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeRevisionBaseSerializerV6

Serializer for recipe type revisions

class recipe.serializers.RecipeTypeSerializerV6(instance=None, data=<class rest_framework.fields.empty>, **kwargs)

Bases: recipe.serializers.RecipeTypeBaseSerializerV6

Converts recipe type model fields to REST output.

definition = None
job_types = None
sub_recipe_types = None
super_recipe_types = None

recipe.urls module

Defines the URLs for the RESTful recipe services

recipe.views module

class recipe.views.RecipeDetailsView(**kwargs)

Bases: rest_framework.generics.RetrieveAPIView

This view is the endpoint for retrieving details of a recipe

queryset
retrieve(request, recipe_id)

Retrieves the details for a recipe and returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • recipe_id (int encoded as a str) – The id of the recipe
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

serializer_class

alias of RecipeDetailsSerializerV6

class recipe.views.RecipeInputFilesView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This is the endpoint for retrieving details about input files associated with a given recipe.

get(request, recipe_id)

Retrieve detailed information about the input files for a recipe

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • recipe_id (int encoded as a str) – The ID for the recipe.
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

queryset
class recipe.views.RecipeReprocessView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for scheduling a reprocess of a recipe

post(request, recipe_id)

Schedules a recipe for reprocessing and returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • recipe_id (int encoded as a str) – The id of the recipe
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

queryset
serializer_class

alias of RecipeDetailsSerializerV6

class recipe.views.RecipeTypeDetailsView(**kwargs)

Bases: rest_framework.generics.GenericAPIView

This view is the endpoint for retrieving details of a recipe type

get(request, name)

Retrieves the details for a recipe type and return them in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

get_v6(request, name)

Retrieves the details for a recipe type and return them in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

patch(request, name)

Edits an existing recipe type and returns the updated details

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

patch_v6(request, name)

Edits an existing recipe type and returns the updated details

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

queryset
serializer_class

alias of RecipeTypeDetailsSerializerV6

class recipe.views.RecipeTypeRevisionDetailsView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving the list of all recipe types

get(request, name, revision_num)

Retrieves the list of all recipe type revisions and returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
  • revision_num (int encoded as a str) – The revision number of the job type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

get_v6(request, name, revision_num)

Retrieves the list of all recipe type revisions returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
  • revision_num (int encoded as a str) – The revision number of the job type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

queryset
serializer_class

alias of RecipeTypeRevisionDetailsSerializerV6

class recipe.views.RecipeTypeRevisionsView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving the list of all recipe types

list(request, name)

Retrieves the list of all recipe type revisions and returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

list_v6(request, name)

Retrieves the list of all recipe type revisions returns it in JSON form

Parameters:
  • request (rest_framework.request.Request) – the HTTP GET request
  • name (string) – The name of the recipe type
Return type:

rest_framework.response.Response

Returns:

the HTTP response to send back to the user

queryset
serializer_class

alias of RecipeTypeRevisionSerializerV6

class recipe.views.RecipeTypesValidationView(**kwargs)

Bases: rest_framework.views.APIView

This view is the endpoint for validating a new recipe type before attempting to actually create it

post(request)

Validates a new recipe type and returns any warnings discovered

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class recipe.views.RecipeTypesView(**kwargs)

Bases: rest_framework.generics.ListCreateAPIView

This view is the endpoint for retrieving the list of all recipe types

create(request)

Creates a new recipe type and returns a link to the detail URL

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

list(request)

Retrieves the list of all recipe types and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
list_v6(request)

Retrieves the list of all recipe types returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset
class recipe.views.RecipesView(**kwargs)

Bases: rest_framework.generics.ListAPIView

This view is the endpoint for retrieving the list of all recipes

get_serializer_class()

Returns the appropriate serializer based off the requests version of the REST API.

list(request)

Retrieves the list of all recipes and returns it in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP GET request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
post(request)

Queue a recipe and returns the new job information in JSON form

Parameters:request (rest_framework.request.Request) – the HTTP POST request
Return type:rest_framework.response.Response
Returns:the HTTP response to send back to the user
queryset

Module contents