# Test reports in AWS CodeBuild You can create reports in CodeBuild that contain details about tests that are run during builds. You can create tests such as unit tests, configuration tests, and functional tests. The following test report file formats are supported: + Cucumber JSON (.json) + JUnit XML (.xml) + NUnit XML (.xml) + NUnit3 XML (.xml) + TestNG XML (.xml) + Visual Studio TRX (.trx) + Visual Studio TRX XML (.xml) **Note** The latest supported version of `cucumber-js` is 7.3.2. Create your test cases with any test framework that can create report files in one of these formats (for example, Surefire JUnit plugin, TestNG, or Cucumber). To create a test report, you add a report group name to the buildspec file of a build project with information about your test cases. When you run the build project, the test cases are run and a test report is created. A new test report is created in the report group each time the test cases run. You do not need to create a report group before you run your tests. If you specify a report group name, CodeBuild creates a report group for you when you run your reports. If you want to use a report group that already exists, you specify its ARN in the buildspec file. You can use a test report to help troubleshoot a problem during a build run. If you have many test reports from multiple builds of a build project, you can use your test reports to view trends and test and failure rates to help you optimize builds. A report expires 30 days after it was created. You cannot view an expired test report. If you want to keep test reports for more than 30 days, you can export your test results' raw data files to an Amazon S3 bucket. Exported test files do not expire. Information about the S3 bucket is specified when you create the report group. **Note** The CodeBuild service role specified in the project is used for permissions to upload to the S3 bucket. **Topics** + [Create test reports](report-create.md) + [Create code coverage reports](code-coverage-report.md) + [Auto-discover reports in CodeBuild](report-auto-discover.md) + [Report groups](test-report-group.md) + [Test frameworks](test-framework-reporting.md) + [View test reports](test-view-reports.md) + [Test report permissions](test-permissions.md) + [Test report statuses](test-report.md) # Create test reports To create a test report, you run a build project that is configured with one to five report groups in its buildspec file. A test report is created during the run. It contains the results of the test cases that are specified for the report groups. A new test report is generated for each subsequent build that uses the same buildspec file. **To create a test report** 1. Create a build project. For information, see [Create a build project in AWS CodeBuild](create-project.md). 1. Configure the buildspec file of your project with test report informaton: 1. Add a `reports:` section and specify either the ARN of an existing report group, or the name of a report group. If you specify an ARN, CodeBuild uses that report group. If you specify a name, CodeBuild creates a report group for you using your project name, and the name you specified, in the format **-**. If the named report group already exists, CodeBuild uses that report group. 1. Under the report group, specify the location of the files that contain the test results. If you use more than one report group, specify test result file locations for each one. A new test report is created each time your build project runs. For more information, see [Specify test files](report-group-test-cases.md). 1. In the `commands` section of the `build` or `post_build` sequence, specify the commands that run the tests cases you specified for your report groups. For more information, see [Specify test commands](report-group-test-case-commands.md). The following is an example of a buildspec `reports` section: ``` reports: php-reports: files: - "reports/php/*.xml" file-format: "JUNITXML" nunit-reports: files: - "reports/nunit/*.xml" file-format: "NUNITXML" ``` 1. Run a build of the build project. For more information, see [Run AWS CodeBuild builds manually](run-build.md). 1. When the build is complete, choose the new build run from **Build history** on your project page. Choose **Reports** to view the test report. For more information, see [View test reports for a build](test-view-project-reports.md). # Create code coverage reports CodeBuild allows you to generate code coverage reports for your tests. The following code coverage reports are provided: Line coverage Line coverage measures how many statements your tests cover. A statement is a single instruction, not including comments or conditionals. `line coverage = (total lines covered)/(total number of lines)` Branch coverage Branch coverage measures how many branches your tests cover out of every possible branch of a control structure, such as an `if` or `case` statement. `branch coverage = (total branches covered)/(total number of branches)` The following code coverage report file formats are supported: + JaCoCo XML + SimpleCov JSON¹ + Clover XML + Cobertura XML + LCOV INFO ¹ CodeBuild accepts JSON code coverage reports generated by [simplecov](https://github.com/simplecov-ruby/simplecov), not [simplecov-json](https://github.com/vicentllongo/simplecov-json). ## Create a code coverage report To create a code coverage report, you run a build project that is configured with at least one code coverage report group in its buildspec file. CodeBuild will interpret the code coverage results and provide a code coverage report for the run. A new test report is generated for each subsequent build that uses the same buildspec file. **To create a test report** 1. Create a build project. For information, see [Create a build project in AWS CodeBuild](create-project.md). 1. Configure the buildspec file of your project with test report information: 1. Add a `reports:` section and specify the name for your report group. CodeBuild creates a report group for you using your project name and the name you specified in the format `project-name`-`report-group-name-in-buildspec`. If you already have a report group you want to use, specify its ARN. If you use the name instead of the ARN, CodeBuild creates a new report group. For more information, see [Reports syntax in the buildspec file](build-spec-ref.md#reports-buildspec-file). 1. Under the report group, specify the location of the files that contain the code coverage results. If you use more than one report group, specify result file locations for each report group. A new code coverage report is created each time your build project runs. For more information, see [Specify test files](report-group-test-cases.md). This is an example that generates a code coverage report for a JaCoCo XML results file located in test-`results/jacoco-coverage-report.xml`. ``` reports: jacoco-report: files: - 'test-results/jacoco-coverage-report.xml' file-format: 'JACOCOXML' ``` 1. In the `commands` section of the `build` or `post_build` sequence, specify the commands that run the code coverage analysis. For more information, see [Specify test commands](report-group-test-case-commands.md). 1. Run a build of the build project. For more information, see [Run AWS CodeBuild builds manually](run-build.md). 1. When the build is complete, choose the new build run from **Build history** on your project page. Choose **Reports** to view the code coverage report. For more information, see [View test reports for a build](test-view-project-reports.md). # Auto-discover reports in CodeBuild With auto-discovery, CodeBuild searches through all your build files after the build phase has completed, searches for any supported report file types, and automatically creates new test and code coverage report groups and reports. For any discovered report types, CodeBuild creates new report groups with the following pattern: ``` --AutoDiscovered ``` **Note** If the discovered report files have the same format type, they will be placed in to the same report group or report. Report auto-discover is configured by your project environment variables: `CODEBUILD_CONFIG_AUTO_DISCOVER` This variable determines whether report auto-discover is disabled during the build. By default, report auto-discover is enabled for all builds. To disable this feature, set `CODEBUILD_CONFIG_AUTO_DISCOVER` to `false`. `CODEBUILD_CONFIG_AUTO_DISCOVER_DIR` (Optional) This variable determines where CodeBuild searches for potential report files. Note that by default, CodeBuild searches in `**/*` by default. These environment variables can be modified during the build phase. For example, if you only want to enable report auto-discover for builds on the `main` git branch, you can check the git branch during the build process and set `CODEBUILD_CONFIG_AUTO_DISCOVER` to false if the build is not on the `main` branch. Report auto-discover can be disabled using the console or using project environment variables. **Topics** + [Configure report auto-discover using the console](#report-auto-discover-configure-console) + [Configure report auto-discover using project environment variables](#report-auto-discover-configure-variable) ## Configure report auto-discover using the console Use the following procedure to configure report auto-discovery using the console. **To configure report auto-discover using the console** 1. Create a build project or choose a build project to edit. For information, see [Create a build project in AWS CodeBuild](create-project.md) or [Change build project settings in AWS CodeBuild](change-project.md). 1. In **Environment**, select **Additional configuration**. 1. To disable report auto-discover, in **Report auto-discover**, select **Disable report auto-discover**. 1. (Optional) In **Auto-discover directory - optional**, enter a directory pattern for CodeBuild to search for supported report format files. Note that CodeBuild searches in `**/*` by default. ## Configure report auto-discover using project environment variables Use the following procedure to configure report auto-discovery using project environment variables. **To configure report auto-discover using project environment variables** 1. Create a build project or choose a build project to edit. For information, see [Create a build project in AWS CodeBuild](create-project.md) or [Change build project settings in AWS CodeBuild](change-project.md). 1. In **Environment variables**, do the following: 1. To disable report auto-discover, for **Name**, enter **CODEBUILD\$1CONFIG\$1AUTO\$1DISCOVER** and for **Value**, enter **false**. This disables report auto-discover. 1. (Optional) For **Name**, enter **CODEBUILD\$1CONFIG\$1AUTO\$1DISCOVER\$1DIR** and for **Value**, enter the directory where CodeBuild should search for supported report format files. For example, `output/*xml` searches for `.xml` files in the `output` directory # Report groups A *report group* contains test reports and specifies shared settings. You use the buildspec file to specify the test cases to run and the commands to run them when it builds. For each report group configured in a build project, a run of the build project creates a test report. Multiple runs of a build project configured with a report group create multiple test reports in that report group, each with results of the same test cases specified for that report group. The test cases are specified for a report group in the buildspec file of a build project. You can specify up to five report groups in one build project. When you run a build, all the test cases run. A new test report is created with the results of each test case specified for a report group. Each time you run a new build, the test cases run and a new test report is created with the new test results. Report groups can be used in more than one build project. All test reports created with one report group share the same configuration, such as its export option and permissions, even if the test reports are created using different build projects. Test reports created with one report group in multiple build projects can contain the results from running different sets of test cases (one set of test cases for each build project). This is because you can specify different test case files for the report group in each project's buildspec file. You can also change the test case files for a report group in a build project by editing its buildspec file. Subsequent build runs create new test reports that contain the results of the test case files in the updated buildspec. **Topics** + [Create a report group](report-group-create.md) + [Report group naming](test-report-group-naming.md) + [Share report groups](report-groups-sharing.md) + [Specify test files](report-group-test-cases.md) + [Specify test commands](report-group-test-case-commands.md) + [Tag a report group in AWS CodeBuild](how-to-tag-report-group.md) + [Update a report group](report-group-export-settings.md) # Create a report group You can use the CodeBuild console, the AWS CLI, or a buildspec file to create a report group. Your IAM role must have the permissions required to create a report group. For more information, see [Test report permissions](test-permissions.md). **Topics** + [Create a report group (buildspec)](#test-report-group-create-buildspec) + [Create a report group (console)](#test-report-group-create-console) + [Create a report group (CLI)](#test-report-group-create-cli) + [Create a report group (CloudFormation)](#test-report-group-create-cfn) ## Create a report group (buildspec) A report group created using the buildspec does not export raw test result files. You can view your report group and specify export settings. For more information, see [Update a report group](report-group-export-settings.md). **To create a report group using a buildspec file** 1. Choose a report group name that is not associated with a report group in your AWS account. 1. Configure the `reports` section of the buildspec file with this name. In this example, the report group name is `new-report-group` and the use test cases are created with the JUnit framework: ``` reports: new-report-group: #surefire junit reports files: - '**/*' base-directory: 'surefire/target/surefire-reports' ``` The report group name can also be specified by using environment variables in the buildspec: ``` version: 0.2 env: variables: REPORT_GROUP_NAME: "new-report-group" phases: build: commands: - ... ... reports: $REPORT_GROUP_NAME: files: - '**/*' base-directory: 'surefire/target/surefire-reports' ``` For more information, see [Specify test files](report-group-test-cases.md) and [Reports syntax in the buildspec file](build-spec-ref.md#reports-buildspec-file). 1. In the `commands` section, specify the command to run your tests. For more information, see [Specify test commands](report-group-test-case-commands.md). 1. Run the build. When the build is complete, a new report group is created with a name that uses the format `project-name-report-group-name`. For more information, see [Report group naming](test-report-group-naming.md). ## Create a report group (console) Use the following procedure to create a report group using the AWS Management Console. **To create a report group** 1. Open the AWS CodeBuild console at [https://console.aws.amazon.com/codesuite/codebuild/home](https://console.aws.amazon.com/codesuite/codebuild/home). 1. In the navigation pane, choose **Report groups**. 1. Choose **Create report group**. 1. For **Report group name**, enter a name for your report group. 1. (Optional) For **Tags**, enter the name and value of any tags that you want supporting AWS services to use. Use **Add row** to add a tag. You can add up to 50 tags. 1. If you want to upload the raw data of your test report results to an Amazon S3 bucket: 1. Select **Export to Amazon S3**. 1. For **S3 bucket name**, enter the name of the S3 bucket. 1. (Optional) For **S3 bucket owner**, enter the AWS account identifier of the account that owns the S3 bucket. This allows report data to be exported to an Amazon S3 bucket that is owned by an account other than the account running the build. 1. For **Path prefix**, enter the path in your S3 bucket where you want to upload your test results. 1. Select **Compress test result data in a zip file** to compress your raw test result data files. 1. Expand **Additional configuration** to display encryption options. Choose one of the following: + **Default AWS managed key** to use a AWS managed key for Amazon S3. For more information, see [Customer managed CMKs](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk) in the *AWS Key Management Service User Guide*. This is the default encryption option. + **Choose a custom key** to use a customer managed key that you create and configure. For **AWS KMS encryption key**, enter the ARN of your encryption key. Its format is ` arn:aws:kms:: :key/ `. For more information, see [Creating KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html) in the *AWS Key Management Service User Guide*. + **Disable artifact encryption** to disable encryption. You might choose this if you want to share your test results, or publish them to a static website. (A dynamic website can run code to decrypt test results.) For more information about encryption of data at rest, see [Data encryption](security-encryption.md). **Note** The CodeBuild service role specified in the project is used for permissions to upload to the S3 bucket. 1. Choose **Create report group**. ## Create a report group (CLI) Use the following procedure to create a report group using the AWS CLI. **To create a report group** 1. Create a file named `CreateReportGroup.json`. 1. Depending on your requirements, copy one of the following JSON code snippets into `CreateReportGroup.json`: + Use the following JSON to specify that your test report group exports raw test result files to an Amazon S3 bucket. ``` { "name": "", "type": "TEST", "exportConfig": { "exportConfigType": "S3", "s3Destination": { "bucket": "", "bucketOwner": "", "path": "", "packaging": "NONE | ZIP", "encryptionDisabled": "false", "encryptionKey": "" }, "tags": [ { "key": "tag-key", "value": "tag-value" } ] } } ``` + Replace ** with your Amazon S3 bucket name and ** with the path in your bucket to where you want to export the files. + If you want to compress the exported files, for `packaging`, specify `ZIP`. Otherwise, specify `NONE`. + `bucketOwner` is optional and is only required if the Amazon S3 bucket is owned by an account other than the account running the build. + Use `encryptionDisabled` to specify whether to encrypt the exported files. If you encrypt the exported files, enter your customer managed key. For more information, see [Update a report group](report-group-export-settings.md). + Use the following JSON to specify that your test report does not export raw test files: ``` { "name": "", "type": "TEST", "exportConfig": { "exportConfigType": "NO_EXPORT" } } ``` **Note** The CodeBuild service role specified in the project is used for permissions to upload to the S3 bucket. 1. Run the following command: ``` aws codebuild create-report-group --cli-input-json file://CreateReportGroupInput.json ``` ## Create a report group (CloudFormation) Use the following instructions to create a report group using the CloudFormation template **To create a report group using the CloudFormation template** You can use an CloudFormation template file to create and provision a report group. For more information, see [CloudFormation User Guide](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html). The following CloudFormation YAML template creates a report group that does not export raw test result files. ``` Resources: CodeBuildReportGroup: Type: AWS::CodeBuild::ReportGroup Properties: Name: my-report-group-name Type: TEST ExportConfig: ExportConfigType: NO_EXPORT ``` The following CloudFormation YAML template creates a report group that exports raw test result files to an Amazon S3 bucket. ``` Resources: CodeBuildReportGroup: Type: AWS::CodeBuild::ReportGroup Properties: Name: my-report-group-name Type: TEST ExportConfig: ExportConfigType: S3 S3Destination: Bucket: amzn-s3-demo-bucket Path: path-to-folder-for-exported-files Packaging: ZIP EncryptionKey: my-KMS-encryption-key EncryptionDisabled: false ``` **Note** The CodeBuild service role specified in the project is used for permissions to upload to the S3 bucket. # Report group naming When you use the AWS CLI or the AWS CodeBuild console to create a report group, you specify a name for the report group. If you use the buildspec to create a new report group, it is named using the format `project-name-report-group-name-specified-in-buildspec`. All reports created by running builds of that build project belong to the new report group that has the new name. If you do not want CodeBuild to create a new report group, specify the ARN of the report group in a build project's buildspec file. You can specify a report group's ARN in multiple build projects. After each build project runs, the report group contains test reports created by each build project. For example, if you create one report group with the name `my-report-group`, and then use its name in two different build projects named `my-project-1` and `my-project-2` and create a build of both projects, two new report groups are created. The result is three report groups with the following names: + `my-report-group`: Does not have any test reports. + `my-project-1-my-report-group`: Contains reports with results of tests run by the build project named `my-project-1`. + `my-project-2-my-report-group`: Contains reports with results of tests run by the build project named `my-project-2`. If you use the ARN of the report group named `my-report-group` in both projects, and then run builds of each project, you still have one report group (`my-report-group`). That report group contains test reports with results of tests run by both build projects. If you choose a report group name that doesn't belong to a report group in your AWS account, and then use that name for a report group in a buildspec file and run a build of its build project, a new report group is created. The format of name of the new report group is `project-name-new-group-name`. For example, if there is not a report group in your AWS account with the name `new-report-group`, and specify it in a build project called `test-project`, a build run creates a new report group with the name `test-project-new-report-group`. # Share report groups Report group sharing allows multiple AWS accounts or users to view a report group, its unexpired reports, and the test results of its reports. In this model, the account that owns the report group (owner) shares a report group with other accounts (consumers). A consumer cannot edit a report group. A report expires 30 days after it is created. **Topics** + [Share a report group](#report-groups-sharing-share) + [Related services](#report-groups-sharing-related) + [Access report groups shared with you](report-groups-sharing-access-prereqs.md) + [Unshare a shared report group](report-groups-sharing-unshare.md) + [Identify a shared report group](report-groups-sharing-identify.md) + [Shared report group permissions](report-groups-sharing-perms.md) ## Share a report group When you share a report group, the consumer is granted read-only access to the report group and its reports. The consumer can use the AWS CLI to view the report group, its reports, and the test case results for each report. The consumer cannot: + View a shared report group or its reports in the CodeBuild console. + Edit a shared report group. + Use the ARN of the shared report group in a project to run a report. A project build that specifies a shared report group fails. You can use the CodeBuild console to add a report group to an existing resource share. If you want to add the report group to a new resource share, you must first create it in the [AWS RAM console](https://console.aws.amazon.com/ram). To share a report group with organizational units or an entire organization, you must enable sharing with AWS Organizations. For more information, see [Enable sharing with AWS Organizations](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html) in the *AWS RAM User Guide*. You can use the CodeBuild console, AWS RAM console, or AWS CLI to share report groups that you own. **Prerequisite** To share a report group, your AWS account must own it. You cannot share a report group that has been shared with you. **To share a report group that you own (CodeBuild console)** 1. Open the AWS CodeBuild console at [https://console.aws.amazon.com/codesuite/codebuild/home](https://console.aws.amazon.com/codesuite/codebuild/home). 1. In the navigation pane, choose **Report groups**. 1. Choose the project you want to share, and then choose **Share**. For more information, see [Create a resource share](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html#getting-started-sharing-create) in the *AWS RAM User Guide*. **To share report groups that you own (AWS RAM console)** See [Creating a resource share](https://docs.aws.amazon.com/ram/latest/userguide/working-with-sharing.html#working-with-sharing-create) in the *AWS RAM User Guide*. **To share report groups that you own (AWS RAM command)** Use the [create-resource-share](https://docs.aws.amazon.com/cli/latest/reference/ram/create-resource-share.html) command. **To share a report group that you own (CodeBuild command)** Use the [put-resource-policy](https://docs.aws.amazon.com/cli/latest/reference/codebuild/put-resource-policy.html) command: 1. Create a file named `policy.json` and copy the following into it. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement":[{ "Effect":"Allow", "Principal":{ "AWS":"111122223333" }, "Action":[ "codebuild:BatchGetReportGroups", "codebuild:BatchGetReports", "codebuild:ListReportsForReportGroup", "codebuild:DescribeTestCases"], "Resource":"arn:aws:iam::*:role/Service*" }] } ``` ------ 1. Update `policy.json` with the report group ARN and identifiers to share it with. The following example grants read-only access to the report group with the ARN `arn:aws:codebuild:us-west-2:123456789012:report-group/my-report-group` to Alice and the root user for the AWS account identified by 123456789012. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement":[{ "Effect":"Allow", "Principal":{ "AWS": [ "arn:aws:iam::123456789012:user/Alice", "123456789012" ] }, "Action":[ "codebuild:BatchGetReportGroups", "codebuild:BatchGetReports", "codebuild:ListReportsForReportGroup", "codebuild:DescribeTestCases"], "Resource":"arn:aws:codebuild:us-west-2:123456789012:report-group/my-report-group" }] } ``` ------ 1. Run the following command. ``` aws codebuild put-resource-policy --resource-arn report-group-arn --policy file://policy.json ``` ## Related services Report group sharing integrates with AWS Resource Access Manager (AWS RAM), a service that makes it possible for you to share your AWS resources with any AWS account or through AWS Organizations. With AWS RAM, you share resources that you own by creating a *resource share* that specifies the resources and the consumers to share them with. Consumers can be individual AWS accounts, organizational units in AWS Organizations, or an entire organization in AWS Organizations. For more information, see the *[AWS RAM User Guide](https://docs.aws.amazon.com/ram/latest/userguide/)*. # Access report groups shared with you To access a shared report group, a consumer's IAM role requires the `BatchGetReportGroups` permission. You can attach the following policy to their IAM role: ``` { "Effect": "Allow", "Resource": [ "*" ], "Action": [ "codebuild:BatchGetReportGroups" ] } ``` For more information, see [Using identity-based policies for AWS CodeBuild](auth-and-access-control-iam-identity-based-access-control.md). # Unshare a shared report group An unshared report group, including its reports and their test case results, can be accessed only by its owner. If you unshare a report group, any AWS account or user you previously shared it with cannot access the report group, its reports, or the results of test cases in the reports. To unshare a shared report group that you own, you must remove it from the resource share. You can use the AWS RAM console or AWS CLI to do this. **To unshare a shared report group that you own (AWS RAM console)** See [Updating a resource share](https://docs.aws.amazon.com/ram/latest/userguide/working-with-sharing.html#working-with-sharing-update) in the *AWS RAM User Guide*. **To unshare a shared report group that you own (AWS RAM command)** Use the [disassociate-resource-share](https://docs.aws.amazon.com/cli/latest/reference/ram/disassociate-resource-share.html) command. ** To unshare report group that you own CodeBuild command)** Run the [delete-resource-policy](https://docs.aws.amazon.com/cli/latest/reference/codebuild/delete-resource-policy.html) command and specify the ARN of the report group you want to unshare: ``` aws codebuild delete-resource-policy --resource-arn report-group-arn ``` # Identify a shared report group Owners and consumers can use the AWS CLI to identify shared report groups. To identify and get information about a shared report group and its reports, use the following commands: + To see the ARNs of report groups shared with you, run `[list-shared-report-groups](https://docs.aws.amazon.com/cli/latest/reference/codebuild/list-shared-report-groups.html)`: ``` aws codebuild list-shared-report-groups ``` + To see the ARNs of the reports in a report group, run `[list-reports-for-report-group](https://docs.aws.amazon.com/cli/latest/reference/codebuild/list-reports-for-report-group.html)` using the report group ARN: ``` aws codebuild list-reports-for-report-group --report-group-arn report-group-arn ``` + To see information about test cases in a report, run `[describe-test-cases](https://docs.aws.amazon.com/cli/latest/reference/codebuild/describe-test-cases.html)` using the report ARN: ``` aws codebuild describe-test-cases --report-arn report-arn ``` The output looks like the following: ``` { "testCases": [ { "status": "FAILED", "name": "Test case 1", "expired": 1575916770.0, "reportArn": "report-arn", "prefix": "Cucumber tests for agent", "message": "A test message", "durationInNanoSeconds": 1540540, "testRawDataPath": "path-to-output-report-files" }, { "status": "SUCCEEDED", "name": "Test case 2", "expired": 1575916770.0, "reportArn": "report-arn", "prefix": "Cucumber tests for agent", "message": "A test message", "durationInNanoSeconds": 1540540, "testRawDataPath": "path-to-output-report-files" } ] } ``` # Shared report group permissions ## Permissions for owners A report group owner can edit the report group and specify it in a project to run reports. ## Permissions for consumers A report group consumer can view a report group, its reports, and the test case results for its reports. A consumer cannot edit a report group or its reports, and cannot use it to create reports. # Specify test files You specify the test result files and their location for each report group in the `reports` section of your build project's buildspec file. For more information, see [Reports syntax in the buildspec file](build-spec-ref.md#reports-buildspec-file). The following is a sample `reports` section that specifies two report groups for a build project. One is specified with its ARN, the other with a name. The `files` section specifies the files that contain the test case results. The optional `base-directory` section specifies the directory where the test case files are located. The optional `discard-paths` section specifies whether paths to test result files uploaded to an Amazon S3 bucket are discarded. ``` reports: arn:aws:codebuild:your-region:your-aws-account-id:report-group/report-group-name-1: #surefire junit reports files: - '**/*' base-directory: 'surefire/target/surefire-reports' discard-paths: false sampleReportGroup: #Cucumber reports from json plugin files: - 'cucumber-json/target/cucumber-json-report.json' file-format: CUCUMBERJSON #Type of the report, defaults to JUNITXML ``` # Specify test commands You specify the commands that run your test cases in the `commands` section of your buildspec file. These commands run the test cases specified for your report groups in the `reports` section of your buildspec file. The following is a sample `commands` section that includes commands to run the tests in test files: ``` commands: - echo Running tests for surefire junit - mvn test -f surefire/pom.xml -fn - echo - echo Running tests for cucumber with json plugin - mvn test -Dcucumber.options="--plugin json:target/cucumber-json-report.json" -f cucumber-json/pom.xml -fn ``` For more information, see [Buildspec syntax](build-spec-ref.md#build-spec-ref-syntax). # Tag a report group in AWS CodeBuild A *tag* is a custom attribute label that you or AWS assigns to an AWS resource. Each AWS tag has two parts: + A *tag key* (for example, `CostCenter`, `Environment`, `Project`, or `Secret`). Tag keys are case sensitive. + An optional field known as a *tag value* (for example, `111122223333`, `Production`, or a team name). Omitting the tag value is the same as using an empty string. Like tag keys, tag values are case sensitive. Together these are known as key-value pairs. For limits on the number of tags you can have on a report group and restrictions on tag keys and values, see [Tags](limits.md#tag-limits). Tags help you identify and organize your AWS resources. Many AWS services support tagging, so you can assign the same tag to resources from different services to indicate that the resources are related. For example, you can assign the same tag to a CodeBuild report group that you assign to an Amazon S3 bucket. For more information about using tags, see the [Tagging best practices](https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf) whitepaper. In CodeBuild, the primary resources are the report group and the project. You can use the CodeBuild console, the AWS CLI, CodeBuild APIs, or AWS SDKs to add, manage, and remove tags for a report group. In addition to identifying, organizing, and tracking your report group with tags, you can use tags in IAM policies to help control who can view and interact with your report group. For examples of tag-based access policies, see [Using tags to control access to AWS CodeBuild resources](auth-and-access-control-using-tags.md). **Topics** + [Add tags to a report group](how-to-tag-report-group-add.md) + [View tags for a report group](how-to-tag-report-group-list.md) + [Edit tags for a report group](how-to-tag-report-group-update.md) + [Remove tags from a report group](how-to-tag-report-group-delete.md) # Add tags to a report group Adding tags to a report group can help you identify and organize your AWS resources and manage access to them. First, you add one or more tags (key-value pairs) to a report group. Keep in mind that there are limits on the number of tags you can have on a report group. There are restrictions on the characters you can use in the key and value fields. For more information, see [Tags](limits.md#tag-limits). After you have tags, you can create IAM policies to manage access to the report group based on these tags. You can use the CodeBuild console or the AWS CLI to add tags to a report group. **Important** Adding tags to a report group can impact access to that report group. Before you add a tag to a report group, make sure to review any IAM policies that might use tags to control access to resources such as report groups. For examples of tag-based access policies, see [Using tags to control access to AWS CodeBuild resources](auth-and-access-control-using-tags.md). For more information about adding tags to a report group when you create it, see [Create a report group (console)](report-group-create.md#test-report-group-create-console). **Topics** + [Add a tag to a report group (console)](#how-to-tag-report-group-add-console) + [Add a tag to a report group (AWS CLI)](#how-to-tag-report-group-add-cli) ## Add a tag to a report group (console) You can use the CodeBuild console to add one or more tags to a CodeBuild report group. 1. Open the CodeBuild console at [https://console.aws.amazon.com/codebuild/](https://console.aws.amazon.com/codebuild/). 1. In **Report groups**, choose the name of the report group where you want to add tags. 1. In the navigation pane, choose **Settings**. 1. If no tags have been added to the report group, choose **Add tag**. You can also choose **Edit**, and then choose **Add tag**. 1. In **Key**, enter a name for the tag. You can add an optional value for the tag in **Value**. 1. (Optional) To add another tag, choose **Add tag** again. 1. When you have finished adding tags, choose **Submit**. ## Add a tag to a report group (AWS CLI) To add a tag to a report group when you create it, see [Create a report group (CLI)](report-group-create.md#test-report-group-create-cli). In `CreateReportGroup.json`, add your tags. To add tags to an existing report group, see [Update a report group (CLI)](report-group-export-settings.md#update-report-group-cli) and add your tags in `UpdateReportGroupInput.json`. In these steps, we assume that you have already installed a recent version of the AWS CLI or updated to the current version. For more information, see [Installing the AWS Command Line Interface](https://docs.aws.amazon.com/cli/latest/userguide/installing.html). # View tags for a report group Tags can help you identify and organize your AWS resources and manage access to them. For more information about using tags, see the [Tagging best practices](https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf) whitepaper. For examples of tag-based access policies, see [Deny or allow actions on report groups based on resource tags](auth-and-access-control-using-tags.md#report-group-tag-policy-example). ## View tags for a report group (console) You can use the CodeBuild console to view the tags associated with a CodeBuild report group. 1. Open the CodeBuild console at [https://console.aws.amazon.com/codebuild/](https://console.aws.amazon.com/codebuild/). 1. In **Report groups**, choose the name of the report group where you want to view tags. 1. In the navigation pane, choose **Settings**. ## View tags for a report group (AWS CLI) Follow these steps to use the AWS CLI to view the AWS tags for a report group. If no tags have been added, the returned tags list is empty. 1. Use the console or the AWS CLI to locate the ARN of your report group. Make a note of it. ------ #### [ AWS CLI ] Run the following command. ``` aws list-report-groups ``` This command returns JSON-formatted information similar to the following: ``` { "reportGroups": [ "arn:aws:codebuild:region:123456789012:report-group/report-group-1", "arn:aws:codebuild:region:123456789012:report-group/report-group-2", "arn:aws:codebuild:region:123456789012:report-group/report-group-3" ] } ``` A report group ARN ends with its name, which you can use to identify the ARN for your report group. ------ #### [ Console ] 1. Open the CodeBuild console at [https://console.aws.amazon.com/codebuild/](https://console.aws.amazon.com/codebuild/). 1. In **Report groups**, choose the name of your report group with the tags you want to view. 1. In **Configuration** locate your report group's ARN. ------ 1. Run the following command. Use the ARN you made a note of for the `--report-group-arns` parameter. ``` aws codebuild batch-get-report-groups --report-group-arns arn:aws:codebuild:region:123456789012:report-group/report-group-name ``` If successful, this command returns JSON-formatted information that contains a `tags` section similar to the following: ``` { ... "tags": { "Status": "Secret", "Project": "TestBuild" } ... } ``` # Edit tags for a report group You can change the value for a tag associated with a report group. You can also change the name of the key, which is equivalent to removing the current tag and adding a different one with the new name and the same value as the other key. Keep in mind that there are restrictions on the characters you can use in the key and value fields. For more information, see [Tags](limits.md#tag-limits). **Important** Editing tags for a report group can impact access to that report group. Before you edit the name (key) or value of a tag for a report group, make sure to review any IAM policies that might use the key or value for a tag to control access to resources such as report groups. For examples of tag-based access policies, see [Deny or allow actions on report groups based on resource tags](auth-and-access-control-using-tags.md#report-group-tag-policy-example). ## Edit a tag for a report group (console) You can use the CodeBuild console to edit the tags associated with a CodeBuild report group. 1. Open the CodeBuild console at [https://console.aws.amazon.com/codebuild/](https://console.aws.amazon.com/codebuild/). 1. In **Report groups**, choose the name of the report group where you want to edit tags. 1. In the navigation pane, choose **Settings**. 1. Choose **Edit**. 1. Do one of the following: + To change the tag, enter a new name in **Key**. Changing the name of the tag is the equivalent of removing a tag and adding a new tag with the new key name. + To change the value of a tag, enter a new value. If you want to change the value to nothing, delete the current value and leave the field blank. 1. When you have finished editing tags, choose **Submit**. ## Edit tags for a report group (AWS CLI) To add, change, or delete tags from a report group, see [Update a report group (CLI)](report-group-export-settings.md#update-report-group-cli). Update the tags in `UpdateReportGroupInput.json`. # Remove tags from a report group You can remove one or more tags associated with a report group. Removing a tag does not delete the tag from other AWS resources that are associated with that tag. **Important** Removing tags for a report group can impact access to that report group. Before you remove a tag from a report group, make sure to review any IAM policies that might use the key or value for a tag to control access to resources such as report groups. For examples of tag-based access policies, see [Using tags to control access to AWS CodeBuild resources](auth-and-access-control-using-tags.md). ## Remove a tag from a report group (console) You can use the CodeBuild console to remove the association between a tag and a CodeBuild report group. 1. Open the CodeBuild console at [https://console.aws.amazon.com/codebuild/](https://console.aws.amazon.com/codebuild/). 1. In **Report groups**, choose the name of the report group where you want to remove tags. 1. In the navigation pane, choose **Settings**. 1. Choose **Edit**. 1. Find the tag you want to remove, and then choose **Remove tag**. 1. When you have finished removing tags, choose **Submit**. ## Remove a tag from a report group (AWS CLI) Follow these steps to use the AWS CLI to remove a tag from a CodeBuild report group. Removing a tag does not delete it, but simply removes the association between the tag and the report group. **Note** If you delete a CodeBuild report group, all tag associations are removed from the deleted report group. You do not have to remove tags before you delete a report group. To delete one or more tags from a report group, see [Edit tags for a report group (AWS CLI)](how-to-tag-report-group-update.md#how-to-tag-report-group-update-cli). Update the `tags` section in the JSON-formatted data with an updated list of tags that does not contain the ones you want to delete. If you want to delete all tags, update the `tags` section to: ``` "tags: []" ``` # Update a report group When you update a report group, you can specify information about whether to export the raw test result data to files in an Amazon S3 bucket. If you choose to export to an S3 bucket, you can specify the following for your report group: + Whether the raw test results files are compressed in a ZIP file. + Whether the raw test result files are encrypted. You can specify encryption with one of the following: + An AWS managed key for Amazon S3. + A customer managed key that you create and configure. For more information, see [Data encryption](security-encryption.md). If you use the AWS CLI to update a report group, you can also update or add tags. For more information, see [Tag a report group in AWS CodeBuildTag a report group](how-to-tag-report-group.md). **Note** The CodeBuild service role specified in the project is used for permissions to upload to the S3 bucket. **Topics** + [Update a report group (console)](#update-report-group-console) + [Update a report group (CLI)](#update-report-group-cli) ## Update a report group (console) Use the following procedure to update a report group using the AWS Management Console. **To update a report group** 1. Open the AWS CodeBuild console at [https://console.aws.amazon.com/codesuite/codebuild/home](https://console.aws.amazon.com/codesuite/codebuild/home). 1. In the navigation pane, choose **Report groups**. 1. Choose the report group you want to update. 1. Choose **Edit**. 1. Select or clear **Backup to Amazon S3**. If you selected this option, specify your export settings: 1. For **S3 bucket name**, enter the name of the S3 bucket. 1. For **Path prefix**, enter the path in your S3 bucket where you want to upload your test results. 1. Select **Compress test result data in a zip file** to compress your raw test result data files. 1. Expand **Additional configuration** to display encryption options. Choose one of the following: + **Default AWS managed key** to use a AWS managed key for Amazon S3. For more information, see [Customer managed CMKs](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk) in the *AWS Key Management Service User Guide*. This is the default encryption option. + **Choose a custom key** to use a customer managed key that you create and configure. For **AWS KMS encryption key**, enter the ARN of your encryption key. Its format is ` arn:aws:kms:: :key/ `. For more information, see [Creating KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html) in the *AWS Key Management Service User Guide*. + **Disable artifact encryption** to disable encryption. You might choose this if you want to share your test results, or publish them to a static website. (A dynamic website can run code to decrypt test results.) ## Update a report group (CLI) Use the following procedure to update a report group using the AWS CLI. **To update a report group** 1. Create a file named `UpdateReportGroupInput.json`. 1. Copy the following into `UpdateReportGroupInput.json`: ``` { "arn": "", "exportConfig": { "exportConfigType": "S3", "s3Destination": { "bucket": "bucket-name", "path": "path", "packaging": "NONE | ZIP", "encryptionDisabled": "false", "encryptionKey": "your-key" } }, "tags": [ { "key": "tag-key", "value": "tag-value" } ] } ``` 1. Enter the ARN of your report group in the `arn` line (for example, `"arn":"arn:aws:codebuild:region:123456789012:report-group/report-group-1")`. 1. Update `UpdateReportGroupInput.json` with the updates you want to apply to your report group. + If you want to update your report group to export raw test result files to an S3 bucket, update the `exportConfig` section. Replace `bucket-name` with your S3 bucket name and `path` with the path in your S3 bucket that you want to export the files to. If you want to compress the exported files, for `packaging`, specify `ZIP`. Otherwise, specify `NONE`. Use `encryptionDisabled` to specify whether to encrypt the exported files. If you encrypt the exported files, enter your customer managed key. + If you want to update your report group so that it does not export raw test result files to an S3 bucket, update the `exportConfig` section with the following JSON: ``` { "exportConfig": { "exportConfigType": "NO_EXPORT" } } ``` + If you want to update the report group's tags, update the `tags` section. You can change, add, or remove tags. If you want to remove all tags, update it with the following JSON: ``` "tags": [] ``` 1. Run the following command: ``` aws codebuild update-report-group \ --cli-input-json file://UpdateReportGroupInput.json ``` # Test frameworks The topics in this section demonstrate how to set up test reporting in AWS CodeBuild for various test frameworks. **Topics** + [Set up test reporting with Jasmine](test-report-jasmine.md) + [Set up test reporting with Jest](test-report-jest.md) + [Set up test reporting with pytest](test-report-pytest.md) + [Set up test reporting with RSpec](test-report-rspec.md) # Set up test reporting with Jasmine The following procedure demonstrates how to set up test reporting in AWS CodeBuild with the [JasmineBDD testing framework](http://jasmine.github.io/). The procedure requires the following prerequisites: + You have an existing CodeBuild project. + Your project is a Node.js project that is set up to use the Jasmine testing framework. Add the [https://www.npmjs.com/package/jasmine-reporters](https://www.npmjs.com/package/jasmine-reporters) package to the `devDependencies` section of your project's `package.json` file. This package has a collection of JavaScript reporter classes that can be used with Jasmine. ``` npm install --save-dev jasmine-reporters ``` If it's not already present, add the `test` script to your project's `package.json` file. The `test` script ensures that Jasmine is called when **npm test** is run. ``` { "scripts": { "test": "npx jasmine" } } ``` CodeBuild supports the following Jasmine test reporters: **JUnitXmlReporter** Used to generate reports in the `JunitXml` format. **NUnitXmlReporter** Used to generate reports in the `NunitXml` format. A Node.js project with Jasmine will, by default, have a `spec` sub-directory, which contains the Jasmine configuration and test scripts. To configure Jasmine to generate reports in the `JunitXML` format, instantiate the `JUnitXmlReporter` reporter by adding the following code to your tests. ``` var reporters = require('jasmine-reporters'); var junitReporter = new reporters.JUnitXmlReporter({ savePath: , filePrefix: , consolidateAll: true }); jasmine.getEnv().addReporter(junitReporter); ``` To configure Jasmine to generate reports in the `NunitXML` format, instantiate the `NUnitXmlReporter` reporter by adding the following code to your tests. ``` var reporters = require('jasmine-reporters'); var nunitReporter = new reporters.NUnitXmlReporter({ savePath: , filePrefix: , consolidateAll: true }); jasmine.getEnv().addReporter(nunitReporter) ``` The test reports are exported to the file specified by **/**. In your `buildspec.yml` file, add/update the following sections. ``` version: 0.2 phases: pre_build: commands: - npm install build: commands: - npm build - npm test reports: jasmine_reports: files: - file-format: JUNITXML base-directory: ``` If you are using the the `NunitXml` report format, change the `file-format` value to the following. ``` file-format: NUNITXML ``` # Set up test reporting with Jest The following procedure demonstrates how to set up test reporting in AWS CodeBuild with the [Jest testing framework](https://jestjs.io/). The procedure requires the following prerequisites: + You have an existing CodeBuild project. + Your project is a Node.js project that is set up to use the Jest testing framework. Add the [https://www.npmjs.com/package/jest-junit](https://www.npmjs.com/package/jest-junit) package to the `devDependencies` section of your project's `package.json` file. CodeBuild uses this package to generate reports in the `JunitXml` format. ``` npm install --save-dev jest-junit ``` If it's not already present, add the `test` script to your project's `package.json` file. The `test` script ensures that Jest is called when **npm test** is run. ``` { "scripts": { "test": "jest" } } ``` Configure Jest to use the `JunitXml` reporter by adding the following to your Jest configuration file. If your project does not have a Jest configuration file, create a file named `jest.config.js` in the root of your project and add the following. The test reports are exported to the file specified by **/**. ``` module.exports = { reporters: [ 'default', [ 'jest-junit', { outputDirectory: , outputName: , } ] ] }; ``` In your `buildspec.yml` file, add/update the following sections. ``` version: 0.2 phases: pre_build: commands: - npm install build: commands: - npm build - npm test reports: jest_reports: files: - file-format: JUNITXML base-directory: ``` # Set up test reporting with pytest The following procedure demonstrates how to set up test reporting in AWS CodeBuild with the [pytest testing framework](https://docs.pytest.org/). The procedure requires the following prerequisites: + You have an existing CodeBuild project. + Your project is a Python project that is set up to use the pytest testing framework. Add the following entry to either the `build` or `post_build` phase of your `buildspec.yml` file. This code automatically discovers tests in the current directory and exports the test reports to the file specified by **/**. The report uses the `JunitXml` format. ``` - python -m pytest --junitxml=/ ``` In your `buildspec.yml` file, add/update the following sections. ``` version: 0.2 phases: install: runtime-versions: python: 3.7 commands: - pip3 install pytest build: commands: - python -m pytest --junitxml=/ reports: pytest_reports: files: - base-directory: file-format: JUNITXML ``` # Set up test reporting with RSpec The following procedure demonstrates how to set up test reporting in AWS CodeBuild with the [RSpec testing framework](https://rspec.info/). The procedure requires the following prerequisites: + You have an existing CodeBuild project. + Your project is a Ruby project that is set up to use the RSpec testing framework. Add/update the following in your `buildspec.yml` file. This code runs the tests in the ** directory and exports the test reports to the file specified by **/**. The report uses the `JunitXml` format. ``` version: 0.2 phases: install: runtime-versions: ruby: 2.6 pre_build: commands: - gem install rspec - gem install rspec_junit_formatter build: commands: - rspec /* --format RspecJunitFormatter --out / reports: rspec_reports: files: - base-directory: file-format: JUNITXML ``` # View test reports You can view details about a test report, such as information about its test cases, pass and fail numbers, and how long it took for it to run. You can view test reports grouped by build run, report group, or your AWS account. Choose a test report in the console to see its details and results of its test cases. You can see view test reports that are not expired. Test reports expire 30 days after they are created. You cannot view an expired report in CodeBuild. **Topics** + [View test reports for a build](test-view-project-reports.md) + [View test reports for a report group](test-view-report-group-reports.md) + [View test reports in your AWS account](test-view-account-reports.md) # View test reports for a build **To view test reports for a build** 1. Open the AWS CodeBuild console at [https://console.aws.amazon.com/codesuite/codebuild/home](https://console.aws.amazon.com/codesuite/codebuild/home). 1. Locate the build you want to view. If you know the project that ran the build that created the test report: 1. In the navigation pane, choose **Build projects**, and then choose the project with the build that ran the test report you want to view. 1. Choose **Build history**, and then choose the build that ran created the reports you want to view. You can also locate the build in the build history for your AWS account: 1. In the navigation pane, choose **Build history**, and then choose the build that created the test reports you want to view. 1. In the build page, choose **Reports**, and then choose a test report to see its details. # View test reports for a report group **To view test reports in a report group** 1. Open the AWS CodeBuild console at [https://console.aws.amazon.com/codesuite/codebuild/home](https://console.aws.amazon.com/codesuite/codebuild/home). 1. In the navigation pane, choose **Report groups**. 1. Choose the report group that contains the test reports you want to view. 1. Choose a test report to see its details. # View test reports in your AWS account **To view test reports in your AWS account** 1. Open the AWS CodeBuild console at [https://console.aws.amazon.com/codesuite/codebuild/home](https://console.aws.amazon.com/codesuite/codebuild/home). 1. In the navigation pane, choose **Report history**. 1. Choose a test report to see its details. # Test report permissions This topic describes important information about permissions related to test reporting. **Topics** + [IAM role for test reports](#test-permissions-required) + [Permissions for test reporting operations](#test-permissions-related-to-reporting) + [Test reporting permissions examples](#test-permissions-examples) ## IAM role for test reports To run a test report, and to update a project to include test reports, your IAM role requires the following permissions. These permissions are included in the predefined AWS managed policies. If you want to add test reporting to an existing build project, you must add these permissions yourself. + `CreateReportGroup` + `CreateReport` + `UpdateReport` + `BatchPutTestCases` To run a code coverage report, your IAM role must also include the `BatchPutCodeCoverages` permission. **Note** `BatchPutTestCases`, `CreateReport`, `UpdateReport`, and `BatchPutCodeCoverages` are not public permissions. You cannot call a corresponding AWS CLI command or SDK method for these permissions. To make sure you have these permissions, you can attach the following policy to your IAM role: ``` { "Effect": "Allow", "Resource": [ "*" ], "Action": [ "codebuild:CreateReportGroup", "codebuild:CreateReport", "codebuild:UpdateReport", "codebuild:BatchPutTestCases", "codebuild:BatchPutCodeCoverages" ] } ``` We recommend that you restrict this policy to only those report groups you must use. The following restricts permissions to only the report groups with the two ARNs in the policy: ``` { "Effect": "Allow", "Resource": [ "arn:aws:codebuild:your-region:your-aws-account-id:report-group/report-group-name-1", "arn:aws:codebuild:your-region:your-aws-account-id:report-group/report-group-name-2" ], "Action": [ "codebuild:CreateReportGroup", "codebuild:CreateReport", "codebuild:UpdateReport", "codebuild:BatchPutTestCases", "codebuild:BatchPutCodeCoverages" ] } ``` The following restricts permissions to only report groups created by running builds of a project named `my-project`: ``` { "Effect": "Allow", "Resource": [ "arn:aws:codebuild:your-region:your-aws-account-id:report-group/my-project-*" ], "Action": [ "codebuild:CreateReportGroup", "codebuild:CreateReport", "codebuild:UpdateReport", "codebuild:BatchPutTestCases", "codebuild:BatchPutCodeCoverages" ] } ``` **Note** The CodeBuild service role specified in the project is used for permissions to upload to the S3 bucket. ## Permissions for test reporting operations You can specify permissions for the following test reporting CodeBuild API operations: + `BatchGetReportGroups` + `BatchGetReports` + `CreateReportGroup` + `DeleteReportGroup` + `DeleteReport` + `DescribeTestCases` + `ListReportGroups` + `ListReports` + `ListReportsForReportGroup` + `UpdateReportGroup` For more information, see [AWS CodeBuild permissions reference](auth-and-access-control-permissions-reference.md). ## Test reporting permissions examples For information about sample policies related to test reporting, see the following: + [Allow a user to change a report group](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-change-report-group) + [Allow a user to create a report group](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-create-report-group) + [Allow a user to delete a report](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-delete-report) + [Allow a user to delete a report group](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-delete-report-group) + [Allow a user to get information about report groups](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-get-information-about-report-group) + [Allow a user to get information about reports](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-get-information-about-reports) + [Allow a user to get a list of report groups](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-get-list-of-report-groups) + [Allow a user to get a list of reports](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-get-list-of-reports) + [Allow a user to get a list of reports for a report group](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-get-list-of-reports-for-report-group) + [Allow a user to get a list of test cases for a report](auth-and-access-control-iam-identity-based-access-control.md#customer-managed-policies-example-get-list-of-test-cases-for-report) # Test report statuses The status of a test report can be one of the following: + `GENERATING`: The run of the test cases is still in progress. + `DELETING`: The test report is being deleted. When a test report is deleted, its test cases are also deleted. Raw test result data files exported to an S3 bucket are not deleted. + `INCOMPLETE`: The test report was not completed. This status might be returned for one of the following reasons: + A problem with the configuration of the report group that specifies this report's test cases. For example, the path to the test cases under the report group in the buildspec file might be incorrect. + The IAM user that ran the build does not have permissions to run tests. For more information, see [Test report permissions](test-permissions.md). + The build was not completed because of an error that is not related to the tests. + `SUCCEEDED`: All test cases were successful. + `FAILED`: Some of the test cases were not successful. Each test case returns a status. The status for a test case can be one of the following: + `SUCCEEDED`: The test case passed. + `FAILED`: The test case failed. + `ERROR`: The test case resulted in an unexpected error. + `SKIPPED`: The test case did not run. + `UNKNOWN`: The test case returned a status other than `SUCCEEDED`, `FAILED`, `ERROR`, or `SKIPPED`. A test report can have a maximum of 500 test case results. If more than 500 test cases are run, CodeBuild prioritizes tests with the status `FAILED` and truncates the test case results.