# Device Advisor workflow
This tutorial explains how to create a custom test suite and run tests against the device you want to test in the console. After the tests are complete, you can view the test results and detailed logs.
## Prerequisites
Before you begin this tutorial this tutorial, complete the steps outlined in [Setting up](device-advisor-setting-up.md).
## Create a test suite definition
First, [install an AWS SDK](https://docs.aws.amazon.com/iot/latest/developerguide/iot-connect-service.html#iot-service-sdks).
### `rootGroup` syntax
A root group is a JSON string that specifies which test cases to include in your test suite. It also specifies any necessary configurations for those test cases. Use the root group to structure and order your test suite based on your needs. The hierarchy of a test suite is:
```
test suite → test group(s) → test case(s)
```
A test suite must have at least one test group, and each test group must have at least one test case. Device Advisor runs tests in the order in which you define the test groups and test cases.
Each root group follows this basic structure:
```
{
"configuration": { // for all tests in the test suite
"": ""
}
"tests": [{
"name": ""
"configuration": { // for all sub-groups in this test group
"": ""
},
"tests": [{
"name": ""
"configuration": { // for all test cases in this test group
"": ""
},
"test": {
"id": ""
"version": ""
}
}]
}]
}
```
In the root group, you define the test suite with a `name`, `configuration`, and the `tests` that the group contains. The `tests` group contains the definitions of individual tests. You define each test with a `name`, `configuration`, and a `test` block that defines the test cases for that test. Finally, each test case is defined with an `id` and `version`.
For information on how to use the `"id"` and `"version"` fields for each test case (`test` block), see [Device Advisor test cases](device-advisor-tests.md). That section also contains information on the available `configuration` settings.
The following block is an example of a root group configuration. This configurations specifies the *MQTT Connect Happy Case* and *MQTT Connect Exponential Backoff Retries* test cases, along with descriptions of the configuration fields.
```
{
"configuration": {}, // Suite-level configuration
"tests": [ // Group definitions should be provided here
{
"name": "My_MQTT_Connect_Group", // Group definition name
"configuration": {} // Group definition-level configuration,
"tests": [ // Test case definitions should be provided here
{
"name": "My_MQTT_Connect_Happy_Case", // Test case definition name
"configuration": {
"EXECUTION_TIMEOUT": 300 // Test case definition-level configuration, in seconds
},
"test": {
"id": "MQTT_Connect", // test case id
"version": "0.0.0" // test case version
}
},
{
"name": "My_MQTT_Connect_Jitter_Backoff_Retries", // Test case definition name
"configuration": {
"EXECUTION_TIMEOUT": 600 // Test case definition-level configuration, in seconds
},
"test": {
"id": "MQTT_Connect_Jitter_Backoff_Retries", // test case id
"version": "0.0.0" // test case version
}
}]
}]
}
```
You must supply the root group configuration when you create your test suite definition. Save the `suiteDefinitionId` that is returned in the response object. You can use this ID to retrieve your test suite definition information and run your test suite.
Here is a Java SDK example:
```
response = iotDeviceAdvisorClient.createSuiteDefinition(
CreateSuiteDefinitionRequest.builder()
.suiteDefinitionConfiguration(SuiteDefinitionConfiguration.builder()
.suiteDefinitionName("your-suite-definition-name")
.devices(
DeviceUnderTest.builder()
.thingArn("your-test-device-thing-arn")
.certificateArn("your-test-device-certificate-arn")
.deviceRoleArn("your-device-role-arn") //if using SigV4 for MQTT over WebSocket
.build()
)
.rootGroup("your-root-group-configuration")
.devicePermissionRoleArn("your-device-permission-role-arn")
.protocol("MqttV3_1_1 || MqttV5 || MqttV3_1_1_OverWebSocket || MqttV5_OverWebSocket")
.build()
)
.build()
)
```
## Get a test suite definition
After you create your test suite definition, you receive the `suiteDefinitionId` in the response object of the `CreateSuiteDefinition` API operation.
When the operation returns the `suiteDefinitionId`, you may see new `id` fields within each group and test case definition within the root group. You can use these IDs to run a subset of your test suite definition.
Java SDK example:
```
response = iotDeviceAdvisorClient.GetSuiteDefinition(
GetSuiteDefinitionRequest.builder()
.suiteDefinitionId("your-suite-definition-id")
.build()
)
```
## Get a test endpoint
Use the `GetEndpoint` API operation to get the test endpoint used by your device. Select the endpoint that best fits your test. To simultaneously run multiple test suites, use the Device-level endpoint by providing a `thing ARN`, `certificate ARN`, or `device role ARN`. To run a single test suite, provide no arguments to the GetEndpoint operation to choose the Account-level endpoint.
SDK example:
```
response = iotDeviceAdvisorClient.getEndpoint(GetEndpointRequest.builder()
.certificateArn("your-test-device-certificate-arn")
.thingArn("your-test-device-thing-arn")
.deviceRoleArn("your-device-role-arn") //if using SigV4 for MQTT over WebSocket
.build())
```
## Start a test suite run
After you create a test suite definition and configureyour test device to connect to your Device Advisor test endpoint, run your test suite with the `StartSuiteRun` API.
For MQTT customers, use either `certificateArn` or `thingArn` to run the test suite. If both are configured, the certificate is used if it belongs to the thing.
For MQTT over WebSocket customer, use `deviceRoleArn` to run the test suite. If the specified role is different from the role specified in the test suite definition, the specified role overrides the defined role.
For `.parallelRun()`, use `true` if you use a Device-level endpoint to run multiple test suites in parallel using one AWS account.
SDK example:
```
response = iotDeviceAdvisorClient.startSuiteRun(StartSuiteRunRequest.builder()
.suiteDefinitionId("your-suite-definition-id")
.suiteRunConfiguration(SuiteRunConfiguration.builder()
.primaryDevice(DeviceUnderTest.builder()
.certificateArn("your-test-device-certificate-arn")
.thingArn("your-test-device-thing-arn")
.deviceRoleArn("your-device-role-arn") //if using SigV4 for MQTT over WebSocket
.build())
.parallelRun(true | false)
.build())
.build())
```
Save the `suiteRunId` from the response. You will use this to retrieve the results of this test suite run.
## Get a test suite run
After you start a test suite run, you can check its progress and its results with the `GetSuiteRun` API.
SDK example:
```
// Using the SDK, call the GetSuiteRun API.
response = iotDeviceAdvisorClient.GetSuiteRun(
GetSuiteRunRequest.builder()
.suiteDefinitionId("your-suite-definition-id")
.suiteRunId("your-suite-run-id")
.build())
```
## Stop a test suite run
To stop a test suite run that is still in progress, you can call the `StopSuiteRun` API operation. After you call the `StopSuiteRun` operation, the service starts the cleanup process. While the service runs the cleanup process, the test suite run status updates to `Stopping`. The cleanup process can take several minutes. Once the process is complete, the test suite run status updates to `Stopped`. After a test run has completely stopped, you n start another test suite run. You can periodically check the suite run status using the `GetSuiteRun` API operation, as shown in the previous section.
SDK example:
```
// Using the SDK, call the StopSuiteRun API.
response = iotDeviceAdvisorClient.StopSuiteRun(
StopSuiteRun.builder()
.suiteDefinitionId("your-suite-definition-id")
.suiteRunId("your-suite-run-id")
.build())
```
## Get a qualification report for a successful qualification test suite run
If you run a qualification test suite that completes successfully, you can retrieve a qualification report with the `GetSuiteRunReport` API operation. You use this qualification report to qualify your device with the AWS IoT Core qualification program. To determine whether your test suite is a qualification test suite, check whether the `intendedForQualification` parameter is set to `true`. After you call the `GetSuiteRunReport` API operation, you can download the report from the returned URL for up to 90 seconds. If more than 90 seconds elapse from the previous time you called the `GetSuiteRunReport` operation, call the operation again to retrieve a new, valid URL.
SDK example:
```
// Using the SDK, call the getSuiteRunReport API.
response = iotDeviceAdvisorClient.getSuiteRunReport(
GetSuiteRunReportRequest.builder()
.suiteDefinitionId("your-suite-definition-id")
.suiteRunId("your-suite-run-id")
.build()
)
```