How Deadline Cloud processes jobs - AWS Deadline Cloud

How Deadline Cloud processes jobs

To process a job, AWS Deadline Cloud uses the Open Job Description (OpenJD) job template to determine the resources needed. Deadline Cloud selects a suitable worker for a step from the fleets associated with your queue. The selected worker meets all of the capability attributes required for the step.

Next, Deadline Cloud sends instructions to the workers to set up a session for the step. The software required for the step must be available on the worker instance for the job to run. The service can open sessions on multiple workers if the scaling settings for the fleet have capacity.

You can set up the software in an Amazon Machine Image (AMI), or your worker can load the software at runtime from a repository or package manager. You can use queue, job, or step environments to deploy the software that you prefer.

The Deadline Cloud service uses the OpenJD template to determine the steps required for the job, and the tasks required for each step. Some steps have dependencies on other steps, so Deadline Cloud determines the order to complete the steps. Then, Deadline Cloud sends the tasks for each step to workers to process. When a task is finished, the service sends another task in the same session, or the worker can start a new session.

You can track the progress of the job in the Deadline Cloud monitor, the Deadline Cloud command line interface (Deadline Cloud CLI) or the AWS CLI. For more information about using the monitor, see Using the Deadline Cloud monitor. For more information about using the Deadline Cloud CLI, see Job states in Deadline Cloud.

After all tasks in each step are finished, the job is complete and the output is ready to download to your workstation. Even if the job didn't finish, the output from each step and task that finished is available to download.

Deadline Cloud removes jobs 120 days after they were submitted. When a job is removed, all of the steps and tasks associated with the job are also removed. If you need to re-run the job, submit the OpenJD template for the job again.