ingest.scan.scanners package¶

Submodules¶

ingest.scan.scanners.dir_scanner module¶

Defines a scanner that scans an S3 bucket backed workspace for files

class ingest.scan.scanners.dir_scanner.DirScanner¶

Bases: ingest.scan.scanners.scanner.Scanner

A scanner for an S3 bucket backed workspace

load_configuration(configuration)¶: See ingest.scan.scanners.scanner.Scanner.load_configuration()

validate_configuration(configuration)¶: See ingest.scan.scanners.scanner.Scanner.validate_configuration()

ingest.scan.scanners.exceptions module¶

Defines the exceptions related to Scan scanners

exception ingest.scan.scanners.exceptions.InvalidScannerConfiguration(description)¶

Bases: ingest.scan.configuration.exceptions.InvalidScanConfiguration

Exception indicating that a scanner configuration was invalid

exception ingest.scan.scanners.exceptions.ScanIngestJobAlreadyLaunched(detail=None, code=None)¶

Bases: rest_framework.exceptions.APIException

Exception indicating that a scanner has already spawned an ingest scan job

default_code = 'conflict'¶

default_detail = 'Ingest Scan already launched'¶

status_code = 409¶

exception ingest.scan.scanners.exceptions.ScannerInterruptRequested¶

Bases: exceptions.Exception

Exception indicating that a scanner run was interrupted

ingest.scan.scanners.factory module¶

Defines the factory for creating monitors

ingest.scan.scanners.factory.add_scanner_type(scanner_class)¶

Registers a scanner class so it can be used for Scale Scans

Parameters:	scanner_class (class:ingest.scan.scanners.scanner.Scanner) – The class definition for a scanner

ingest.scan.scanners.factory.get_scanner(scanner_type)¶

Returns a scanner of the given type that is set to scan the given workspace

Parameters:	scanner_type (string) – The unique identifier of a registered scanner
Returns:	A scanner for storing and retrieving files.
Return type:	`ingest.scan.scanners.scanner.Scanner`

ingest.scan.scanners.factory.get_scanner_types()¶

Returns a list of type identifiers for all registered scanners

Returns:	A list of scanner types
Return type:	[string]

ingest.scan.scanners.s3_scanner module¶

Defines a scanner that scans an S3 bucket backed workspace for files

class ingest.scan.scanners.s3_scanner.S3Scanner¶

Bases: ingest.scan.scanners.scanner.Scanner

A scanner for an S3 bucket backed workspace

load_configuration(configuration)¶: See ingest.scan.scanners.scanner.Scanner.load_configuration()

validate_configuration(configuration)¶: See ingest.scan.scanners.scanner.Scanner.validate_configuration()

ingest.scan.scanners.scanner module¶

Defines the base scanner class

class ingest.scan.scanners.scanner.Scanner(scanner_type, supported_broker_types)¶

Bases: object

Abstract class for a scanner that processes existing files to ingest. Sub-classes must have a no-argument constructor that passes in the correct scanner type and supported broker types and should override the load_configuration(), run(), stop(), and validate_configuration() methods.

load_configuration(configuration)¶

Loads the given configuration

Parameters:	configuration (dict) – The configuration as a dictionary

run(dry_run=False)¶

Runs the scanner until signaled to stop by the stop() method or processing complete.

Parameters:	dry_run (bool) – Flag to enable file scanning only, no file ingestion will occur

scanner_type¶

The type of this scanner

Returns:	The scanner type
Return type:	string

set_recursive(recursive)¶

Support configuration of scanner recursive property

Parameters:	recursive (bool) – The flag indicating whether workspace will be recursively scanned

setup_workspaces(scanned_workspace, file_handler)¶

Sets up the workspaces that will be used by this scanner

Parameters:	scanned_workspace (string) – The name of the workspace that is being scanned file_handler (`ingest.handlers.file_handler.FileHandler`) – The file handler configured for this monitor

stop()¶

Signals the scanner to stop running.

It is the responsibility of sub-classes that override the run() call to monitor this instance variable and stop as gracefully and quickly as possible.

supported_broker_types¶

The broker types supported by this scanner

Returns:	The broker types supported by this scanner
Return type:	[string]

validate_configuration(configuration)¶

Validates the given configuration.

Sub-classes must validate scanner type specific configuration and return a list of warnings if encountered.

Parameters:	configuration (dict) – The configuration as a dictionary
Returns:	A list of warnings discovered during validation
Return type:	[`ingest.scan.configuration.scan_configuration.ValidationWarning`]

:raises ingest.scan.scanners.exceptions.InvalidScannerConfiguration: If the given configuration is: invalid