Seed User Guide

This example assumes Python 3 is installed and uses the Pillow library for image processing. For help installing Python, see the Python Beginners Guide. For help installing Pillow, see its installation instructions.

We are going to write our algorithm using basic Linux commands. Use your favorite text editor or IDE to create an image_rotate.py file:

If you have cloned this repository, the image file seed.png will exist in the guide/example directory. If not, you will need to download the file or supply your own image file. Run the algorithm using the following command:

A new image file named ROTATED_180_seed.png will be created in the output directory.

If a ModuleNotFound error occurs, see the Python ModuleNotFound Errors section.

First, create a script called image_rotate.sh in the same directory as the Python file created above:

We will use this script to invoke our image rotation algorithm.

Create a file named Dockerfile in the same directory:

With these files, we can create our initial Docker containerized sample algorithm. Issue the following terminal commands to build the Docker image:

The image may take a minute or two to build. The -t flag is used to specify the name and optional tag for the Docker image in the 'name:tag' format. In this case, we have used the name img-test, but we could use any name. Once it completes successfully, you should see output similar to the following:

Now run the Docker container using the following command:

Output should be similar to the following:

Let’s recap what we’ve done:

There are some observations we should make about what we just accomplished.

With the img-test Docker image created, we could share this with other people on our local machine. We could also tag it and push it to a remote registry (hub.docker.com, quay.io, etc.) and others would be able to run it. For a basic algorithm example, this is fairly simple, but what if we have a more complicated algorithm with specific resource requirements? What if our algorithm requires large supporting reference datasets? What if we need to leverage runtime licenses that must be carefully protected? What if we want all of these requirements to be explicitly documented and transparent to the consumers of your algorithm? Seed helps solve these problems.

GitLab repositories, such as those available through GEOINT Services GitLab, have built-in Docker registries. Registry information can be found by going to Packages → Container Registry in the Project navigation pane.

In order to push to a GitLab Container registry using Seed CLI, it is necessary to first create a Personal Access Token. Be sure to check the box to give the token the "api" scope. The access token name will be used as the username and the token value as the password to log into the container registry.

The following command will publish our Seed image that we built and tested to a GitLab Container Registry:

As can be seen from the output, we are internally performing a series of operations to publish the image. We attach an appropriate tag to the Docker image to comply with the specification that reflects the remote registry gitlab-registry.gs.mil and organization gitlab-group/gitlab-project. In a GitLab Container Registry, the "organization" is a combination of the Group and Project names. This is followed by a push of the image to the repository and cleanup of the remote tags. This leaves the local environment with only the image names built for publishing, which can be verified by running the docker images command.

Docker Hub is a managed registry that makes it easy to publish your Seed image without configuring any additional services of your own. Before you can publish, you will need to register for an account at https://hub.docker.com.

The following command will publish our Seed image that we built and tested to Docker Hub:

As can be seen from the output, we are internally performing a series of operations to publish the image. We attach an appropriate tag to the Docker image to comply with the specification that reflects the remote registry index.docker.io and organization dockerhub-username. This is followed by a push of the image to the repository and cleanup of the remote tags. This leaves the local environment with only the image names built for publishing, which can be verified by running the docker images command.

If a small amount of reference data (e.g. less than 100 MB) is needed for a Seed job, it can be included by adding the file to the Docker image. For larger amounts of reference data, avoid excessively increasing the size of the Docker image by passing the reference data to the job as a mountable directory. For more details about mounting directories, see the Docker documentation about bind mounts and using volumes.

Although it is generally preferable to use the Inputs object to pass in data, the Settings object can be used to specify sensitive information for the job, such as a database password.

Update the job.interface object in our manifest from the [Tutorial] with a settings object:

Re-build the Seed job using the seed build command. MY_SECRET_SETTING will be injected into the job as an environment variable at runtime. It can be specified as a environment variable when executing the job using seed run:

If the environment variable is ommitted from the seed run command, the Seed CLI will output an error:

When running the Seed job on a Scale cluster, secret names and values are input as part of the JobType definition in Scale. Scale then stores the secrets in a {hashicorp-vault-url[HashiCorp Vault] secret store where they are only accessible by running jobs. The system will automatically retrieve any secrets defined in the Seed job and inject them at runtime. You can also contact the Scale team for assistance with adding JobTypes that contain secrets.

Using a Jupyter Notebook algorithm to make a Seed job is possible, but it requires more steps than a standard script. This guide will assume a basic understanding of Jupyter Notebooks.

Use of Anaconda and other large libraries associated with data science and machine learning jobs is common. This makes sense for development on Windows or other environments that make compilation of complex scientific and math libraries challenging but should not be used in a Seed job. As a rule of thumb, in order to avoid excessive strain on Docker registry and image cache resources, Docker images in Seed jobs should be kept under 500 MB. Docker images should be based on minimal operating systems, such as Alpine or Busybox, and layer optimization techniques should be applied. Docker recommends a number of best practices for writing Dockerfiles.

Another strategy for optimizing the size of the Docker image associated with the Seed job is to add commands to clean up after any package installations. Consult the documentation for any package management system used by the operating system (e.g apk or yum) and/or language (e.g. pip or mpm), as the exact clean-up commands will vary depending on the environment and tools being used.

Breaking down larger processing chains into separate jobs will yield benefits throughout the algorithm development lifecycle. Problems can be discovered sooner and component algorithms can be built and tested independently, helping to minimize any single person or team being a bottleneck. The greatest benefits can be realized when extracting general purpose, reusable jobs that can be applied to multiple data types, such as a job for generating a tile pyramid within a GeoTIFF. This also helps ensure that commonly used jobs are both optimized and thoroughly proven by reuse. Of course, there are always exceptions, and some algorithms (e.g. those that are I/O bound) may benefit from combining several processing steps into a single job.

Not having direct access to the file system of your job means the only method for feedback on what is happening inside the container is through console output. Use standard output and/or standard error to indicate any progress or errors for your algorithm. Some languages (such as Python) may require specifying that output should not be buffered until the process exits. This will facilitate live viewing of output for long-running processes.

Specific error conditions represented by unique process exit codes should be used to give clear feedback when a known error has occurred. These can be used to distinguish between data errors (where the data arrived but contained an error that could not be automatically addressed by the job) and errors that occurred within the job execution itself. See also Why should I define error codes? in the FAQ section.

Most jobs have configuration values that are desirable to change at runtime. These could be anything from confidence values for feature detection, to a band number within a multi-band image. Ensure that your job can consume its configuration as either environment variables or command-line arguments. This makes it possible to define your job as a configurable Docker image and avoid requiring the creation of new Docker images for common changes.

Docker images often are set to use the root user by default. Avoid this unless absolutely necessary. Instead of using the root user, add a non-privileged user to the Docker image and set the USER for the image to this non-privileged user.

Docker provides guidance on some additional best practices for writing Dockerfiles.

On Windows, search for Edit environment variables for your account. This will open a dialog showing the user environment variables for your account. Select "Path", then click the "Edit" button, which will open another dialog. Click "New", and enter the path to the folder containing the seed.exe binary file on your system. Ensure the path does not end in a slash and specify only the containing folder, not the full path to seed.exe.

For example, if the full path to seed.exe is C:\Program Files\Seed-CLI\seed.exe, the Path value should be the following:

When using Docker for Windows with a Linux Docker container, the files added to the Docker image must have Unix-style line endings. These can be changed using a text editor such as NotePad++ or a command-line utility such as dos2unix.

On Windows, we recommend to manually specify host paths using forward slashes rather than backsplashes, which are typically used for Windows paths. For example, use "//C/Users/Username/path/to/seed-test" instead of "C:\Users\Username\path\to\seed-test".

Seed is a general standard that was developed to aid in the discovery and running of self-contained algorithms. It is a JSON-defined manifest that is embedded into the Docker image as a label. It defines:

Many resources for learning about Docker can be found on the Docker website. The primary goal of this guide is to explain how to integrate an algorithm with Seed and not focus on how to build a Docker container.

The image should be as small as possibly and only include the necessary files. It is pulled across the network each time it runs on a node, so smaller images result in less network traffic.

It is critical to strike a balance between the minimum resources required to run the algorithm and acceptable runtimes. Scale depends on accurate requirements in order to efficiently schedule jobs on the cluster. If a job requests more resources than it uses, these resources are wasted as they cannot not be allocated to other jobs. In a cloud environment, wasted resources equate to wasted money. When job requirements depend on the input, the Seed manifest supports specifying a multiplicative value for the resource requests via the inputMultiplier property.

It depends. If the total size of the file(s) is sufficiently small (e.g. less than 100 MB), add it into the Docker image. If it has larger requirements, such as a lookup directory, please work with the Scale team to create a mountable directory.

The Seed manifest allows the developer to define "Settings", which will be defined as environment variables at runtime. These "Settings" can be marked as "secret" and will be stored in HashiCorp Vault when running the job in a processing system that consumes Seed images, such as Scale. The values for these environment variables are hidden and only injected at runtime. See Secret Data for more details.

The developer should properly capture and define errors in their code using exit codes. The Seed manifest allows the developer to define specific error codes with name, description, and type (job vs. data). A data error indicates that the input data was invalid and the job will never be able to run it. All other errors are job errors and could potentially be resolved on future executions (re-queues/re-process). A third type, "System" errors, are reserved for the Scale framework.

Code should use proper try/catch or if/else statements to identify the type of error. The code can exit with any value between 0-255. The zero value indicates successful run. The exit code can be checked after a program finishes with "echo $?".

Common examples exiting with code 42:

Python: sys.exit(42)

Matlab: exit(42)

Java: System.exit(42)

C/C++: exit(42)

Shell: exit 42

It depends. Does each step involve heavy processing or create time-consuming intermediate products? If so, they would be better as their own standalone jobs with its output products being passed to a subsequent step in a workflow. If the individual steps are relatively small (in runtime and resources), it can all run within a single container using a shell script or similar to execute each step.

If the following ModuleNotFound error occurs, it likely means the Pillow library is not installed correctly. Please refer to the Pillow installation instructions.

If the following SyntaxError occurs, it likely means you are using Python 2 instead of Python 3. Please refer to these instructions for installing Python.

When using WSL 2 on Windows, some users have reported the following error:

If this occurs, add a symlink to docker-credential-desktop.exe; e.g.