# Devices and jobs
Devices can communicate with AWS IoT Jobs using MQTT, HTTP Signature Version 4, or HTTP TLS. To determine the endpoint to use when your device communicates with AWS IoT Jobs, run the **DescribeEndpoint** command. For example, if you run this command:
```
aws iot describe-endpoint --endpoint-type iot:Data-ATS
```
you get a result similar to the following:
```
{
"endpointAddress": "a1b2c3d4e5f6g7-ats.iot.us-west-2.amazonaws.com"
}
```
## Using the MQTT protocol
Devices can communicate with AWS IoT Jobs using MQTT protocol. Devices subscribe to MQTT topics to be notified of new jobs and to receive responses from the AWS IoT Jobs service. Devices publish on MQTT topics to query or update the state of a job launch. Each device has its own general MQTT topic. For more information about publishing and subscribing to MQTT topics, see [Device communication protocols](protocols.md).
With this method of communication, your device uses its device-specific certificate and private key to authenticate with AWS IoT Jobs.
Your devices can subscribe to the following topics. `thing-name` is the name of the thing associated with the device.
+
**`$aws/things/thing-name/jobs/notify`**
Subscribe to this topic to notify you when a job launch is added or removed from the list of pending job launches.
+
**`$aws/things/thing-name/jobs/notify-next`**
Subscribe to this topic to notify you when the next pending job execution has changed.
+
**`$aws/things/thing-name/jobs/request-name/accepted`**
The AWS IoT Jobs service publishes success and failure messages on an MQTT topic. The topic is formed by appending `accepted` or `rejected` to the topic used to make the request. Here, `request-name` is the name of a request such as `Get` and the topic can be: `$aws/things/myThing/jobs/get`. AWS IoT Jobs then publishes success messages on the `$aws/things/myThing/jobs/get/accepted` topic.
+
**`$aws/things/thing-name/jobs/request-name/rejected`**
Here, `request-name` is the name of a request such as `Get`. If the request failed, AWS IoT Jobs publishes failure messages on the `$aws/things/myThing/jobs/get/rejected` topic.
You can also use the following HTTPS API operations:
+ Update the status of a job execution by calling the [https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_UpdateJobExecution.html](https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_UpdateJobExecution.html) API.
+ Query the status of a job execution by calling the [https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_DescribeJobExecution.html](https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_DescribeJobExecution.html) API.
+ Retrieve a list of pending job executions by calling the [https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_GetPendingJobExecutions.html](https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_GetPendingJobExecutions.html) API.
+ Retrieve the next pending job execution by calling the [https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_DescribeJobExecution.html](https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_DescribeJobExecution.html) API with `jobId` as `$next`.
+ Get and start the next pending job execution by calling the [https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_StartNextPendingJobExecution.html](https://docs.aws.amazon.com/iot/latest/apireference/API_iot-jobs-data_StartNextPendingJobExecution.html) API.
## Using HTTP Signature Version 4
Devices can communicate with AWS IoT Jobs using HTTP Signature Version 4 on port 443. This is the method used by the AWS SDKs and CLI. For more information about those tools, see [AWS CLI Command Reference: iot-jobs-data](https://docs.aws.amazon.com/cli/latest/reference/iot-jobs-data/index.html) or [AWS SDKs and Tools](http://aws.amazon.com/tools/#sdk) and refer to the IotJobsDataPlane section for your preferred language.
With this method of communication, your device uses IAM credentials to authenticate with AWS IoT Jobs.
The following commands are available using this method:
+ **DescribeJobExecution**
`aws iot-jobs-data describe-job-execution ...`
+ **GetPendingJobExecutions**
`aws iot-jobs-data get-pending-job-executions ...`
+ **StartNextPendingJobExecution**
`aws iot-jobs-data start-next-pending-job-execution ...`
+ **UpdateJobExecution**
`aws iot-jobs-data update-job-execution ...`
## Using HTTP TLS
Devices can communicate with AWS IoT Jobs using HTTP TLS on port 8443 using a third-party software client that supports this protocol.
With this method, your device uses X.509 certificate-based authentication (for example, its device-specific certificate and private key).
The following commands are available using this method:
+ **DescribeJobExecution**
+ **GetPendingJobExecutions**
+ **StartNextPendingJobExecution**
+ **UpdateJobExecution**
## Programming devices to work with jobs
The examples in this section use MQTT to illustrate how a device works with the AWS IoT Jobs service. Or, you could use the corresponding API or CLI commands. For these examples, we assume a device called `MyThing` that subscribes to the following MQTT topics:
+ `$aws/things/MyThing/jobs/notify` (or `$aws/things/MyThing/jobs/notify-next`)
+ `$aws/things/MyThing/jobs/get/accepted`
+ `$aws/things/MyThing/jobs/get/rejected`
+ `$aws/things/MyThing/jobs/jobId/get/accepted`
+ `$aws/things/MyThing/jobs/jobId/get/rejected`
If you're using code signing for AWS IoT, your device code must verify the signature of your code file. The signature is in the job document in the `codesign` property. For more information about verifying a code file signature, see [Device Agent Sample](https://github.com/aws/aws-iot-device-sdk-js#jobsAgent).
**Topics**
+ [Programming devices to work with jobs](#programming-devices)
+ [Device workflow](jobs-workflow-device-online.md)
+ [Jobs workflow](jobs-workflow-jobs-online.md)
+ [Jobs notifications](jobs-comm-notifications.md)
# Device workflow
A device can handle jobs that it runs using either of the following ways.
+
**Get the next job**
1. When a device first comes online, it should subscribe to the device's `notify-next` topic.
1. Call the [DescribeJobExecution](jobs-mqtt-api.md#mqtt-describejobexecution) MQTT API with jobId `$next` to get the next job, its job document, and other details, including any state saved in `statusDetails`. If the job document has a code file signature, you must verify the signature before proceeding with processing the job request.
1. Call the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API to update the job status. Or, to combine this and the previous step in one call, the device can call [StartNextPendingJobExecution](jobs-mqtt-api.md#mqtt-startnextpendingjobexecution).
1. (Optional) You can add a step timer by setting a value for `stepTimeoutInMinutes` when you call either [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) or [StartNextPendingJobExecution](jobs-mqtt-api.md#mqtt-startnextpendingjobexecution).
1. Perform the actions specified by the job document using the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API to report on the progress of the job.
1. Continue to monitor the job execution by calling the [DescribeJobExecution](jobs-mqtt-api.md#mqtt-describejobexecution) MQTT API with this jobId. If the job execution is deleted, [DescribeJobExecution](jobs-mqtt-api.md#mqtt-describejobexecution) returns a `ResourceNotFoundException`.
The device should be able to recover to a valid state if the job execution is canceled or deleted while the device is running the job.
1. Call the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API when finished with the job to update the job status and report success or failure.
1. Because this job's execution status has been changed to a terminal state, the next job available for execution (if any) changes. The device is notified that the next pending job execution has changed. At this point, the device should continue as described in step 2.
If the device remains online, it continues to receive notifications of the next pending job execution. This includes its job execution data, when it completes a job or a new pending job execution is added. When this occurs, the device continues as described in step 2.
+
**Select from available jobs**
1. When a device first comes online, it should subscribe to the thing's `notify` topic.
1. Call the [GetPendingJobExecutions](jobs-mqtt-api.md#mqtt-getpendingjobexecutions) MQTT API to get a list of pending job executions.
1. If the list contains one or more job executions, select one.
1. Call the [DescribeJobExecution](jobs-mqtt-api.md#mqtt-describejobexecution) MQTT API to get the job document and other details, including any state saved in `statusDetails`.
1. Call the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API to update the job status. If the `includeJobDocument` field is set to `true` in this command, the device can skip the previous step and retrieve the job document at this point.
1. Optionally, you can add a step timer by setting a value for `stepTimeoutInMinutes` when you call [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution).
1. Perform the actions specified by the job document using the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API to report on the progress of the job.
1. Continue to monitor the job execution by calling the [DescribeJobExecution](jobs-mqtt-api.md#mqtt-describejobexecution) MQTT API with this jobId. If the job execution is canceled or deleted while the device is running the job, the device should be able to recover to a valid state.
1. Call the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API when finished with the job to update the job status and to report success or failure.
If the device remains online, it is notified of all pending job executions when a new pending job execution becomes available. When this occurs, the device can continue as described in step 2.
If the device is unable to carry out the job, it should call the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API to update the job status to `REJECTED`.
# Jobs workflow
The following shows the different steps in the jobs workflow from starting a new job to reporting the completion status of a job execution.
## Start a new job
When a new job is created, AWS IoT Jobs publishes a message on the `$aws/things/thing-name/jobs/notify` topic for each target device.
The message contains the following information:
```
{
"timestamp":1476214217017,
"jobs":{
"QUEUED":[{
"jobId":"0001",
"queuedAt":1476214216981,
"lastUpdatedAt":1476214216981,
"versionNumber" : 1
}]
}
}
```
The device receives this message on the `'$aws/things/thingName/jobs/notify'` topic when the job execution is queued.
**Note**
For jobs with the optional `SchedulingConfig`, the job will maintain an initial status state of `SCHEDULED`. When the job reaches the selected `startTime`, the following will occur:
The job status state will update to `IN_PROGRESS`.
The job will begin rollout of the job document to all devices in the target group.
## Get job information
To get more information about a job execution, the device calls the [DescribeJobExecution](jobs-mqtt-api.md#mqtt-describejobexecution) MQTT API with the `includeJobDocument` field set to `true` (the default).
If the request is successful, the AWS IoT Jobs service publishes a message on the `$aws/things/MyThing/jobs/0023/get/accepted` topic:
```
{
"clientToken" : "client-001",
"timestamp" : 1489097434407,
"execution" : {
"approximateSecondsBeforeTimedOut": number,
"jobId" : "023",
"status" : "QUEUED",
"queuedAt" : 1489097374841,
"lastUpdatedAt" : 1489097374841,
"versionNumber" : 1,
"jobDocument" : {
< contents of job document >
}
}
}
```
If the request fails, the AWS IoT Jobs service publishes a message on the `$aws/things/MyThing/jobs/0023/get/rejected` topic.
The device now has the job document that it can use to perform the remote operations for the job. If the job document contains an Amazon S3 presigned URL, the device can use that URL to download any required files for the job.
## Report job execution status
As the device is executing the job, it can call the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API to update the status of the job execution.
For example, a device can update the job execution status to `IN_PROGRESS` by publishing the following message on the `$aws/things/MyThing/jobs/0023/update` topic:
```
{
"status":"IN_PROGRESS",
"statusDetails": {
"progress":"50%"
},
"expectedVersion":"1",
"clientToken":"client001"
}
```
Jobs respond by publishing a message to the `$aws/things/MyThing/jobs/0023/update/accepted` or `$aws/things/MyThing/jobs/0023/update/rejected` topic:
```
{
"clientToken":"client001",
"timestamp":1476289222841
}
```
The device can combine the two previous requests by calling [StartNextPendingJobExecution](jobs-mqtt-api.md#mqtt-startnextpendingjobexecution). That gets and starts the next pending job execution and allows the device to update the job execution status. This request also returns the job document when there is a job execution pending.
If the job contains a [TimeoutConfig](https://docs.aws.amazon.com//iot/latest/apireference/API_TimeoutConfig.html), the in-progress timer starts running. You can also set a step timer for a job execution by setting a value for `stepTimeoutInMinutes` when you call [UpdateJobExecution](https://docs.aws.amazon.com//iot/latest/apireference/API_iot-jobs-data_UpdateJobExecution.html). The step timer applies only to the job execution that you update. You can set a new value for this timer each time you update a job execution. You can also create a step timer when you call [StartNextPendingJobExecution](https://docs.aws.amazon.com//iot/latest/apireference/API_iot-jobs-data_StartNextPendingJobExecution.html). If the job execution remains in the `IN_PROGRESS` status for longer than the step timer interval, it fails and switches to the terminal `TIMED_OUT` status. The step timer has no effect on the in-progress timer that you set when you create a job.
The `status` field can be set to `IN_PROGRESS`, `SUCCEEDED`, or `FAILED`. You cannot update the status of a job execution that is already in a terminal state.
## Report execution completed
When the device is finished executing the job, it calls the [UpdateJobExecution](jobs-mqtt-api.md#mqtt-updatejobexecution) MQTT API. If the job was successful, set `status` to `SUCCEEDED` and, in the message payload, in `statusDetails`, add other information about the job as name-value pairs. The in-progress and step timers end when the job execution is complete.
For example:
```
{
"status":"SUCCEEDED",
"statusDetails": {
"progress":"100%"
},
"expectedVersion":"2",
"clientToken":"client-001"
}
```
If the job was not successful, set `status` to `FAILED` and, in `statusDetails`, add information about the error that occurred:
```
{
"status":"FAILED",
"statusDetails": {
"errorCode":"101",
"errorMsg":"Unable to install update"
},
"expectedVersion":"2",
"clientToken":"client-001"
}
```
**Note**
The `statusDetails` attribute can contain any number of name-value pairs.
When the AWS IoT Jobs service receives this update, it publishes a message on the `$aws/things/MyThing/jobs/notify` topic to indicate that the job execution is complete:
```
{
"timestamp":1476290692776,
"jobs":{}
}
```
## Additional jobs
If there are other job executions pending for the device, they are included in the message published to `$aws/things/MyThing/jobs/notify`.
For example:
```
{
"timestamp":1476290692776,
"jobs":{
"QUEUED":[{
"jobId":"0002",
"queuedAt":1476290646230,
"lastUpdatedAt":1476290646230
}],
"IN_PROGRESS":[{
"jobId":"0003",
"queuedAt":1476290646230,
"lastUpdatedAt":1476290646230
}]
}
}
```
# Jobs notifications
The AWS IoT Jobs service publishes MQTT messages to reserved topics when jobs are pending or when the first job execution in the list changes. Devices can track pending jobs by subscribing to these topics.
## Job notification types
Job notifications are published to MQTT topics as JSON payloads. There are two kinds of notifications:
**ListNotification**
A `ListNotification` contains a list of no more than 15 pending job executions. They are sorted by status (`IN_PROGRESS` job executions before `QUEUED` job executions) and then by the times when they were queued.
A `ListNotification` is published whenever one of the following criteria is met.
+ A new job execution is queued or changes to a non-terminal status (`IN_PROGRESS` or `QUEUED`).
+ An old job execution changes to a terminal status (`FAILED`, `SUCCEEDED`, `CANCELED`, `TIMED_OUT`, `REJECTED`, or `REMOVED`).
For more information about the limits with and without the scheduling configuration, see [Job executions limits](job-limits.md#job-execution-limits).
**NextNotification**
+ A `NextNotification` contains summary information about the job execution that's next in the queue.
A `NextNotification` is published whenever the first job execution in the list changes.
+ A new job execution is added to the list as `QUEUED`, and it's the first one in the list.
+ The status of an existing job execution that wasn't the first one in the list changes from `QUEUED` to `IN_PROGRESS`, and becomes the first one in the list. (This happens when there are no other `IN_PROGRESS` job executions in the list or when the job execution whose status changes from `QUEUED` to `IN_PROGRESS` was queued earlier than any other `IN_PROGRESS` job execution in the list.)
+ The status of the job execution that is first in the list changes to a terminal status and is removed from the list.
For more information about publishing and subscribing to MQTT topics, see [Device communication protocols](protocols.md).
**Note**
Notifications are not available when you use HTTP Signature Version 4 or HTTP TLS to communicate with jobs.
## Job pending
The AWS IoT Jobs service publishes a message on an MQTT topic when a job is added to or removed from the list of pending job executions for a thing or the first job execution in the list changes:
+ `$aws/things/thingName/jobs/notify`
+ `$aws/things/thingName/jobs/notify-next`
The messages contain the following example payloads:
`$aws/things/thingName/jobs/notify`:
```
{
"timestamp" : 10011,
"jobs" : {
"IN_PROGRESS" : [ {
"jobId" : "other-job",
"queuedAt" : 10003,
"lastUpdatedAt" : 10009,
"executionNumber" : 1,
"versionNumber" : 1
} ],
"QUEUED" : [ {
"jobId" : "this-job",
"queuedAt" : 10011,
"lastUpdatedAt" : 10011,
"executionNumber" : 1,
"versionNumber" : 0
} ]
}
}
```
If the job execution called `this-job` originated from a job with the optional scheduling configuration selected and the job document rollout scheduled to take place during a maintenance window, it'll only appear during a recurring maintenance window. Outside of a maintenance window, the job called `this-job` will be excluded from the list of pending job executions as shown in the following example.
```
{
"timestamp" : 10011,
"jobs" : {
"IN_PROGRESS" : [ {
"jobId" : "other-job",
"queuedAt" : 10003,
"lastUpdatedAt" : 10009,
"executionNumber" : 1,
"versionNumber" : 1
} ],
"QUEUED" : []
}
}
```
`$aws/things/thingName/jobs/notify-next`:
```
{
"timestamp" : 10011,
"execution" : {
"jobId" : "other-job",
"status" : "IN_PROGRESS",
"queuedAt" : 10009,
"lastUpdatedAt" : 10009,
"versionNumber" : 1,
"executionNumber" : 1,
"jobDocument" : {"c":"d"}
}
}
```
If the job execution called `other-job` originated from a job with the optional scheduling configuration selected and the job document rollout scheduled to take place during a maintenance window, it'll only appear during a recurring maintenance window. Outside of a maintenance window, the job called `other-job` won't be listed as the next job execution as shown in the following example.
```
{} //No other pending jobs
```
```
{
"timestamp" : 10011,
"execution" : {
"jobId" : "this-job",
"queuedAt" : 10011,
"lastUpdatedAt" : 10011,
"executionNumber" : 1,
"versionNumber" : 0,
"jobDocument" : {"a":"b"}
}
} // "this-job" is pending next to "other-job"
```
Possible job execution status values are `QUEUED`, `IN_PROGRESS`, `FAILED`, `SUCCEEDED`, `CANCELED`, `TIMED_OUT`, `REJECTED`, and `REMOVED`.
The following series of examples show the published notifications to each topic as job executions are created and changed from one state to another.
First, one job, called `job1`, is created. This notification is published to the `jobs/notify` topic:
```
{
"timestamp": 1517016948,
"jobs": {
"QUEUED": [
{
"jobId": "job1",
"queuedAt": 1517016947,
"lastUpdatedAt": 1517016947,
"executionNumber": 1,
"versionNumber": 1
}
]
}
}
```
This notification is published to the `jobs/notify-next` topic:
```
{
"timestamp": 1517016948,
"execution": {
"jobId": "job1",
"status": "QUEUED",
"queuedAt": 1517016947,
"lastUpdatedAt": 1517016947,
"versionNumber": 1,
"executionNumber": 1,
"jobDocument": {
"operation": "test"
}
}
}
```
When another job is created (`job2`), this notification is published to the `jobs/notify` topic:
```
{
"timestamp": 1517017192,
"jobs": {
"QUEUED": [
{
"jobId": "job1",
"queuedAt": 1517016947,
"lastUpdatedAt": 1517016947,
"executionNumber": 1,
"versionNumber": 1
},
{
"jobId": "job2",
"queuedAt": 1517017191,
"lastUpdatedAt": 1517017191,
"executionNumber": 1,
"versionNumber": 1
}
]
}
}
```
A notification is not published to the `jobs/notify-next` topic because the next job in the queue (`job1`) has not changed. When `job1` starts to execute, its status changes to `IN_PROGRESS`. No notifications are published because the list of jobs and the next job in the queue have not changed.
When a third job (`job3`) is added, this notification is published to the `jobs/notify` topic:
```
{
"timestamp": 1517017906,
"jobs": {
"IN_PROGRESS": [
{
"jobId": "job1",
"queuedAt": 1517016947,
"lastUpdatedAt": 1517017472,
"startedAt": 1517017472,
"executionNumber": 1,
"versionNumber": 2
}
],
"QUEUED": [
{
"jobId": "job2",
"queuedAt": 1517017191,
"lastUpdatedAt": 1517017191,
"executionNumber": 1,
"versionNumber": 1
},
{
"jobId": "job3",
"queuedAt": 1517017905,
"lastUpdatedAt": 1517017905,
"executionNumber": 1,
"versionNumber": 1
}
]
}
}
```
A notification is not published to the `jobs/notify-next` topic because the next job in the queue is still `job1`.
When `job1` is complete, its status changes to `SUCCEEDED`, and this notification is published to the `jobs/notify` topic:
```
{
"timestamp": 1517186269,
"jobs": {
"QUEUED": [
{
"jobId": "job2",
"queuedAt": 1517017191,
"lastUpdatedAt": 1517017191,
"executionNumber": 1,
"versionNumber": 1
},
{
"jobId": "job3",
"queuedAt": 1517017905,
"lastUpdatedAt": 1517017905,
"executionNumber": 1,
"versionNumber": 1
}
]
}
}
```
At this point, `job1` has been removed from the queue, and the next job to be executed is `job2`. This notification is published to the `jobs/notify-next` topic:
```
{
"timestamp": 1517186269,
"execution": {
"jobId": "job2",
"status": "QUEUED",
"queuedAt": 1517017191,
"lastUpdatedAt": 1517017191,
"versionNumber": 1,
"executionNumber": 1,
"jobDocument": {
"operation": "test"
}
}
}
```
If `job3` must begin executing before `job2` (which is not recommended), the status of `job3` can be changed to `IN_PROGRESS`. If this happens, `job2` is no longer next in the queue, and this notification is published to the `jobs/notify-next` topic:
```
{
"timestamp": 1517186779,
"execution": {
"jobId": "job3",
"status": "IN_PROGRESS",
"queuedAt": 1517017905,
"startedAt": 1517186779,
"lastUpdatedAt": 1517186779,
"versionNumber": 2,
"executionNumber": 1,
"jobDocument": {
"operation": "test"
}
}
}
```
No notification is published to the `jobs/notify` topic because no job has been added or removed.
If the device rejects `job2` and updates its status to `REJECTED`, this notification is published to the `jobs/notify` topic:
```
{
"timestamp": 1517189392,
"jobs": {
"IN_PROGRESS": [
{
"jobId": "job3",
"queuedAt": 1517017905,
"lastUpdatedAt": 1517186779,
"startedAt": 1517186779,
"executionNumber": 1,
"versionNumber": 2
}
]
}
}
```
If `job3` (which is still in progress) is force deleted, this notification is published to the `jobs/notify` topic:
```
{
"timestamp": 1517189551,
"jobs": {}
}
```
At this point, the queue is empty. This notification is published to the `jobs/notify-next` topic:
```
{
"timestamp": 1517189551
}
```