Experiment report configurations for AWS FIS

You can enable AWS Fault Injection Service (FIS) to generate reports for experiments, making it easier to produce evidence of resilience testing. The experiment report is a PDF document that summarizes experiment actions and optionally captures application response from a CloudWatch dashboard that you specify. To see an example experiment report, download the zip file here.

To enable and configure the contents of the report generated for the experiment, you define the experiment report configuration for the experiment template. When you specify a CloudWatch dashboard, AWS FIS includes a snapshot graph of all the widgets in the given dashboard annotated with experiment start and end time over a duration that you specify, as shown in the example below. This example demonstrates the impact of a packet loss experiment in an Availability Zone (AZ). When packet loss is introduced in AZ use1-az6, traffic shifts away from use1-az6 and towards use1-az4, so that the number of bytes processed by the load balancer in that AZ declines.

When the experiment ends, the report can be downloaded from the AWS FIS console and is also stored in an Amazon S3 bucket. If you include a CloudWatch dashboard in your report configuration, images of each widget are also delivered. Reports are not generated for experiments that are cancelled or run as part of target preview (with actionsMode set to skip-all). Once the experiment exceeds the experiment data retention limit, the report will only be available from the Amazon S3 bucket. AWS FIS charges apply for each delivered report, except for those that fail with internal errors. For more information, see AWS Fault Injection Service pricing and Quotas and limitations for AWS Fault Injection Service. Ingest and storage charges for Amazon S3 and CloudWatch API charges for getMetricWidgetImage and GetDashboard requests may apply. For more information, see Amazon S3 pricing and CloudWatch pricing.

Experiment report configuration syntax

The following is the syntax for the experiment report configuration, an optional section of the experiment template.

{
    "experimentReportConfiguration": {
        "outputs": {
            "s3Configuration": {
                "bucketName": "my-bucket-name",
                "prefix": "report-storage-prefix" 
            }
        },
        "dataSources": {
            "cloudWatchDashboards": [
                {
                    "dashboardIdentifier": "arn:aws:cloudwatch::123456789012:dashboard/MyDashboard"
                }
            ]
        },
        "preExperimentDuration": "PT20M",
        "postExperimentDuration": "PT20M" 
    }
}
        

Using the experimentReportConfiguration, you can customize the output destination, input data, and time windows for the data to include in the experiment report, which can help you better understand the impact and results of your AWS FIS experiments. When you define the experiment report configuration, you provide the following:

Experiment report permissions

To enable AWS FIS to generate and store the experiment report, you need to allow the following operations from your AWS FIS experiment IAM role:

We recommend you follow AWS security best practices and restrict the experiment role to the bucket and prefix. The following is an example policy statement that restricts the experiment role access.

{
    "Version": "2012-10-17", 
    "Statement": 
        [ 
            {
                "Action": [
                    "s3:putObject",
                    "s3:getObject"
                    ],
                "Resource": "arn:aws:s3:::my-experiment-report-bucket/my-prefix/*", 
                "Effect": "Allow"
            },
            {
                "Action": [
                    "cloudwatch:getDashboard"
                    ],
                "Resource": "arn:aws:cloudwatch::012345678912:dashboard/my-experiment-report-dashboard",
                "Effect": "Allow"
            },
            {
                "Action": [
                    "cloudwatch:getMetricWidgetImage"
                    ],
                "Resource": "*",
                "Effect": "Allow"
            }
         ] 
 }
        

Experiment report best practices

The following are best practices for using the AWS FIS experiment report configuration: