v6 Scan Services

These services allow a user to create, view, and manage Scan processes.

v6 Scan List

Example GET /v6/scans/ API call

Request: GET http://.../v6/scans/

Response: 200 OK

{
    "count": 3,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 1,
            "name": "my-scan-process",
            "title": "My Scan Process",
            "description": "This is my Scan process for detecting my favorite files!",
            "file_count": 50,
            "job": {
                "id": 7,
                "job_type": {
                    "id": 2,
                    "name": "scale-scan",
                    "version": "1.0.0",
                    "title": "Scale Scan",
                    "description": "Scans a workspace for existing source files to ingest",
                    "is_active": true,
                    "is_paused": false,
                    "is_published": true,
                    "icon_code": "f013",
                    "unmet_resources": "chocolate,vanilla"
                },
                "job_type_rev": {
                    "id": 2
                },
                "status": "RUNNING"
            },
            "dry_run_job": {
                "id": 6,
                "job_type": {
                    "id": 2,
                    "name": "scale-scan",
                    "version": "1.0.0",
                    "title": "Scale scan",
                    "description": "Scans a workspace for existing source files to ingest",
                    "is_active": true,
                    "is_paused": false,
                    "is_published": true,
                    "icon_code": "f013",
                    "unmet_resources": "chocolate,vanilla"
                },
                "status": "RUNNING"
            },
           "created": "2015-03-11T00:00:00Z",
           "last_modified": "2015-03-11T00:00:00Z"
        },
        ...
    ]
}
Scan List
Returns a list of all Scan processes.
GET /v6/scans/
Query Parameters
page Integer Optional The page of the results to return. Defaults to 1.
page_size Integer Optional The size of the page to use for pagination of results. Defaults to 100, and can be anywhere from 1-1000.
started ISO-8601 Datetime Optional The start of the time range to query. Supports the ISO-8601 date/time format, (ex: 2015-01-01T00:00:00Z). Supports the ISO-8601 duration format, (ex: PT3H0M0S).
ended ISO-8601 Datetime Optional End of the time range to query, defaults to the current time. Supports the ISO-8601 date/time format, (ex: 2015-01-01T00:00:00Z). Supports the ISO-8601 duration format, (ex: PT3H0M0S).
name String Optional Return only Scan processes with a given name. Duplicate it to filter by multiple values.
order String Optional One or more fields to use when ordering the results. Duplicate it to multi-sort, (ex: order=name&order=description). Prefix fields with a dash to reverse the sort, (ex: order=-name).
Successful Response
Status 200 OK
Content Type application/json
JSON Fields
count Integer The total number of results that match the query parameters.
next URL A URL to the next page of results.
previous URL A URL to the previous page of results.
results Array List of result JSON objects that match the query parameters.
.id Integer The unique identifier of the model. Can be passed to the details API. (See Scan Details)
.name String The identifying name of the Scan process used for queries.
.title String The human readable display name of the Scan process.
.description String A longer description of the Scan process.
.file_count Integer Count of files identified from last scan operation (either dry run or ingest).
.job JSON Object The job that is associated with the Scan process. (See Job Details)
.dry_run_job JSON Object The dry run job that is associated with the Scan process. (See Job Details)
.created ISO-8601 Datetime When the associated database model was initially created.
.last_modified ISO-8601 Datetime When the associated database model was last saved.

v6 Create Scan

Example POST /v6/scans/ API call

Request: POST http://.../v6/scans/

{
    "title": "My Scan Process",
    "description": "This is my Scan process for detecting my favorite files!",
    "configuration": {
        "workspace": "my-workspace",
        "scanner": {
            "type": "dir",
        },
        "recursive": true,
        "files_to_ingest": [{
            "filename_regex": ".*txt"
        }],
        "recipe": {
            "name": "my-recipe",
            "revision_num": 1
        }
    }
}

Response: 201 Created Headers: Location http://.../v6/scans/105/

{
    "id": 1,
    "name": "my-scan-process",
    "title": "My Scan Process",
    "description": "This is my Scan process for detecting my favorite files!",
    "file_count": 50,
    "job": {
        "id": 7,
        "job_type": {
            "id": 2,
            "name": "scale-scan",
            "version": "1.0.0",
            "title": "Scale Scan",
            "description": "Scans a workspace for existing source files to ingest",
            "is_active": true,
            "is_paused": false,
            "is_published": true,
            "icon_code": "f013",
            "unmet_resources": "chocolate,vanilla"
        },
        "status": "RUNNING"
    },
    "dry_run_job": {
        "id": 6,
        "job_type": {
            "id": 2,
            "name": "scale-scan",
            "version": "1.0.0",
            "title": "Scale Scan",
            "description": "Scans a workspace for existing source files to ingest",
            "is_active": true,
            "is_paused": false,
            "is_published": true,
            "icon_code": "f013",
            "unmet_resources": "chocolate,vanilla"
        },
        "status": "RUNNING"
    },
    "created": "2015-03-11T00:00:00Z",
    "last_modified": "2015-03-11T00:00:00Z",
    "configuration": {
        "workspace": "my-workspace",
        "monitor": {
            "type": "dir"
        },
        "recursive": true,
        "files_to_ingest": [{
            "filename_regex": ".*txt"
        }],
        "recipe": {
            "name": "my-recipe",
            "revision_num": 1
        }
    }
}
Create Scan
Creates a new Scan. To start a dry run or actual scan job, use the /scans/{id}/process/ endpoint.
POST /v6/scans/
Content Type application/json
JSON Fields
title String Required The human readable display name of the Scan process.
description String Optional A longer description of the Scan process.
configuration JSON Object Required JSON defining the Scan configuration. (See Scan Configuration JSON)
Successful Response
Status 201 CREATED
Location URL pointing to the details for the newly created scan process
Content Type application/json
JSON Fields
  JSON Object All fields are the same as the Scan process details model. (See Scan Details)

v6 Scan Details

Example GET /v6/scans/{id}/ API call

Request: GET http://.../v6/scans/{id}/

Response: 200 OK

{
    "id": 1,
    "name": "my-scan-process",
    "title": "My Scan Process",
    "description": "This is my Scan process for detecting my favorite files!",
    "file_count": 50,
    "job": {
        "id": 7,
        "job_type": {
            "id": 2,
            "name": "scale-scan",
            "version": "1.0.0",
            "title": "Scale Scan",
            "description": "Scans a workspace for existing source files to ingest",
            "is_active": true,
            "is_paused": false,
            "is_published": true,
            "icon_code": "f013",
            "unmet_resources": "chocolate,vanilla"
        },
        "status": "RUNNING"
    },
    "dry_run_job": {
        "id": 6,
        "job_type": {
            "id": 2,
            "name": "scale-scan",
            "version": "1.0.0",
            "title": "Scale Scan",
            "description": "Scans a workspace for existing source files to ingest",
            "is_active": true,
            "is_paused": false,
            "is_published": true,
            "icon_code": "f013",
            "unmet_resources": "chocolate,vanilla"
        },
        "status": "RUNNING"
    },
    "created": "2015-03-11T00:00:00Z",
    "last_modified": "2015-03-11T00:00:00Z",
    "configuration": {
        "workspace": "my-workspace",
        "monitor": {
            "type": "dir"
        },
        "recursive": true,
        "files_to_ingest": [{
            "filename_regex": ".*txt"
        }],
        "recipe": {
            "name": "my-recipe",
            "revision_num": 1
        }
    }
}
Scan Details
Returns Scan process details
GET /v6/scans/{id}/
Where {id} is the unique identifier of an existing model.
Successful Response
Status 200 OK
Content Type application/json
JSON Fields
id Integer The unique identifier of the model. Can be passed to the details API. (See Scan Details)
name String The identifying name of the Scan process used for queries.
title String The human readable display name of the Scan process.
description String A longer description of the Scan process.
file_count Integer Count of files identified from last scan operation (either dry run or ingest).
job JSON Object The job that is associated with the Scan process. (See Job Details)
dry_run_job JSON Object The dry run job that is associated with the Scan process. (See Job Details)
created ISO-8601 Datetime When the associated database model was initially created.
last_modified ISO-8601 Datetime When the associated database model was last saved.
configuration JSON Object JSON defining the Scan configuration. (See Scan Configuration JSON)

v6 Validate Scan

Example POST /v6/scans/validation/ API call

Request: POST http://.../v6/scans/validation/

{
    "title": "My Scan Process",
    "description": "This is my Scan process for detecting my favorite files!",
    "configuration": {
        "workspace": "my-workspace",
        "monitor": {
            "type": "dir"
        },
        "recursive": true,
        "files_to_ingest": [{
            "filename_regex": ".*txt"
        }],
        "recipe": {
            "name": "my-recipe",
            "revision_num": 1
        }
    }
}

Response: 200 OK

{
   "is_valid": true,
   "errors": [],
   "warnings": [{"name": "EXAMPLE_WARNING", "description": "This is an example warning."}],
}
Validate Scan
Validates a new Scan process configuration without actually saving it
POST /v6/scans/validation/
Content Type application/json
JSON Fields
title String Required The human readable display name of the Scan process.
description String Optional A longer description of the Scan process.
configuration JSON Object Required JSON defining the Scan configuration. (See Scan Configuration JSON)
Successful Response
Status 200 OK
Content Type application/json
JSON Fields
is_valid Boolean Indicates if the given fields were valid for creating a new scan. If this is true, then submitting the same fields to the /scans/ API will successfully create a new scan.
errors Array Lists any errors causing is_valid to be false. The errors are JSON objects with name and description string fields.
warnings Array A list of warnings discovered during validation.
.id String An identifier for the warning.
.details String A human-readable description of the problem.

v6 Edit Scan

Example PATCH /v6/scans/{id}/ API call

Request: PATCH http://.../v6/scans/{id}/

{
    "title": "My Scan Process",
    "description": "This is my Scan process for detecting my favorite files!",
    "configuration": {
        "workspace": "my-workspace",
        "monitor": {
            "type": "dir"
        },
        "recursive": true,
        "files_to_ingest": [{
            "filename_regex": ".*txt"
        }],
        "recipe": {
            "name": "my-recipe",
            "revision_num": 1
        }
    }
}

Response: 204 NO CONTENT

Edit Scan
Edits an existing Scan process with associated configuration
PATCH /v6/scans/{id}/
Where {id} is the unique identifier of an existing model.
Content Type application/json
JSON Fields
title String Optional The human readable display name of the Scan process.
description String Optional A longer description of the Scan process.
configuration JSON Object Optional JSON defining the Scan configuration. (See Scan Configuration JSON)
Successful Response
Status 204 No Content

v6 Process Scan

Example POST /v6/scans/{id}/process/ API call

Request: POST http://.../v6/scans/{id}/process/

{
  "ingest": true
}

Response: 200 OK

{
    "id": 1,
    "name": "my-scan-process",
    "title": "My Scan Process",
    "description": "This is my Scan process for detecting my favorite files!",
    "file_count": 50,
    "job": {
        "id": 7,
        "job_type": {
            "id": 2,
            "name": "scale-scan",
            "version": "1.0.0",
            "title": "Scale Scan",
            "description": "Scans a workspace for existing source files to ingest",
            "is_active": true,
            "is_paused": false,
            "is_published": true,
            "icon_code": "f013",
            "unmet_resources": "chocolate,vanilla"
        },
        "status": "RUNNING"
    },
    "dry_run_job": {
        "id": 6,
        "job_type": {
            "id": 2,
            "name": "scale-scan",
            "version": "1.0.0",
            "title": "Scale Scan",
            "description": "Scans a workspace for existing source files to ingest",
            "is_active": true,
            "is_paused": false,
            "is_published": true,
            "icon_code": "f013",
            "unmet_resources": "chocolate,vanilla"
        },
        "status": "RUNNING"
    },
    "created": "2015-03-11T00:00:00Z",
    "last_modified": "2015-03-11T00:00:00Z",
    "configuration": {
        "workspace": "my-workspace",
        "monitor": {
            "type": "dir"
        },
        "recursive": true,
        "files_to_ingest": [{
            "filename_regex": ".*txt"
        }],
        "recipe": {
            "name": "my-recipe",
            "revision_num": 1
        }
    }
}
Process Scan
Launches an existing Scan with associated configuration
POST /v6/scans/{id}/process/
Where {id} is the unique identifier of an existing model.
Content Type application/json
JSON Fields
ingest Boolean Optional Whether a dry run or ingest triggering scan should be run. Defaults to false when unset.
Successful Response
Status 200 OK
Content Type application/json
JSON Fields
id Integer The unique identifier of the model. Can be passed to the details API. (See Scan Details)
name String The identifying name of the Scan process used for queries.
title String The human readable display name of the Scan process.
description String A longer description of the Scan process.
file_count Integer Count of files identified from last scan operation (either dry run or ingest).
job JSON Object The job that is associated with the Scan process. (See Job Details)
dry_run_job JSON Object The dry run job that is associated with the Scan process. (See Job Details)
created ISO-8601 Datetime When the associated database model was initially created.
last_modified ISO-8601 Datetime When the associated database model was last saved.
configuration JSON Object JSON defining the Scan configuration. (See Scan Configuration JSON)

v6 Cancel Scan

This call will send a cancel job request for the job associated with the scan, as well as all queued or running ingest jobs spawned from the scan. recipes triggered by completed ingest jobs will not be effected. Returns the job ids of all canceled jobs.

Example POST /v6/scans/cancel/{id} API call

Request: POST http://.../v6/scans/cancel/{id}/

Response: 202 Accepted

{
  "id": 1,
  "canceled_jobs": [1,2,3]
}
Cancel Scan
Cancels a scan and associated ingest jobs
POST /v6/scans/cancel/{id}/
Where {id} is the unique identifier of an existing scan model.
Successful Response
Status 202 ACCEPTED
Content Type application/json
JSON Fields
id Integer The unique identifier of the scan.
canceled_jobs Array List of the job ids associated with the scan that were canceled.

Scan Configuration JSON

A scan configuration JSON describes a set of configuration settings that affect how a scanner executes.

Example interface:

{
  "workspace" : "workspace_name",
  "scanner" : {
    "type" : "dir",
    "transfer_suffix" : "_tmp"
  },
  "recursive" : true,
  "files_to_ingest":[
    {
      "filename_regex" : ".*txt",
      "data_types": [ "type1", "type2" ],
      "new_workspace" : "workspace_name",
      "new_file_path" : "wksp/path"
    }
  ],
  "recipe": {
    "name": "my-recipe",
    "revision_num": 1
  }
}
Scan Configuration
workspace String Required String that specifies the name of the workspace that is being scanned. The type of the workspace (its broker type) will determine which types of scanner can be used.
scanner JSON Object Required JSON object representing the type and configuration of the scanner that will scan workspace for files.
.type String Required The type of the scanner. Must be either ‘dir’ or ‘s3’
.transfer_suffix String Optional Defines a suffix that is used on the file names to indicate that files are still transferring and have not yet finished being copied into the scanned directory
recursive Boolean Optional Indicates whether a scanner should be limited to the root of a workspace (false) or traverse the entire tree (true). If ommitted, the default is true
files_to_ingest Array Required List of JSON objects that define the rules for how to handle files that appear in the scanned workspace. The array must contain at least one item.
.filename_regex String Required Regular expression to check against the names of new files in the scanned workspace. When a new file appears in the workspace, the file’s name is checked against each expression in order of the files_to_ingest array. If an expression matches the new file name in the workspace, that file is ingested according to the other fields in the JSON object and all subsequent rules in the list are ignored (first rule matched is applied).
.data_types Array Optional Any file that matches the corresponding file name regular expression will have these data type strings “tagged” with the file. If not provided, data_types defaults to an empty array.
.new_workspace String Optional Specifies the name of a new workspace to which the file should be copied. This allows the ingest process to move files to a different workspace after they appear in the scanned workspace.
.new_file_path String Optional Specifies a new relative path for storing new files. If new_workspace is also specified, the file is moved to the new workspace at this new path location (instead of using the current path the new file originally came in on). If new_workspace is not specified, the file is moved to this new path location within the original scanned workspace. In either of these cases, three additional and dynamically named directories, for the current year, month, and day, will be appended to the new_file_path value automatically by the Scale system (i.e. workspace_path/YYYY/MM/DD).
recipe JSON Object Required Specifies the natural key of the recipe the Strike will start when a file is ingested.
.name String Required Specifies the name of the recipe the Scan will attempt to start when the file is ingested.
.revision_num Integer Optional Specifies the revision number of the recipe the Scan will attempt to start when the file is ingested.