Logging Insights events - AWS CloudTrail
Understanding Insights events deliveryLogging Insights events with the AWS Management ConsoleLogging Insights events with the AWS Command Line InterfaceLogging Insights events with the AWS SDKs

Logging Insights events

AWS CloudTrail Insights help AWS users identify and respond to unusual activity associated with API calls and API error rates by continuously analyzing CloudTrail management events. CloudTrail Insights analyzes your normal patterns of API call volume and API error rates, also called the baseline, and generates Insights events when the call volume or error rates are outside normal patterns. Insights events on API call volume are generated for write management APIs, and Insights events on API error rate are generated for both read and write management APIs.

Note

To log Insights events on API call volume, the trail or event data store must log write management events. To log Insights events on API error rate, the trail or event data store must log read or write management events.

CloudTrail Insights analyzes management events that occur in a single Region, not globally. A CloudTrail Insights event is generated in the same Region as its supporting management events are generated.

Additional charges apply for Insights events. You will be charged separately if you enable Insights for both trails and event data stores. For more information, see AWS CloudTrail Pricing.

Understanding Insights events delivery

Unlike other types of events that CloudTrail captures, Insights events are logged only when CloudTrail detects changes in your account's API usage that differ significantly from the account's typical usage patterns.

Where CloudTrail delivers events and how long it takes to receive Insights events differs between trails and event data stores.

Insights events delivery for trails

If you've enabled Insights events on a trail and CloudTrail detects unusual activity, CloudTrail delivers Insights events to the /CloudTrail-Insight folder in the chosen destination S3 bucket for your trail. After you enable CloudTrail Insights for the first time on a trail, it can take up to 36 hours for CloudTrail to deliver the first Insights event, if unusual activity is detected.

If you turn off Insights events logging on a trail and then re-enable Insights events, or stop and restart logging on a trail, it can take up to 36 hours for CloudTrail to restart delivery of Insights events, if unusual activity is detected.

Insights events delivery for event data stores

If you've enabled Insights events on a source event data store, CloudTrail delivers Insights events to the destination event data store. After you enable CloudTrail Insights for the first time on the source event data store, it can take up to 7 days for CloudTrail to deliver the first Insights event to the destination event data store, if unusual activity is detected.

If you turn off Insights events logging on a source event data store and then re-enable Insights events, or stop and restart event ingestion on a source event data store, it can take up to 7 days for CloudTrail to restart delivery of Insights events, if unusual activity is detected. Additional charges apply for ingesting Insights events in CloudTrail Lake. You will be charged separately if you enable Insights for both trails and event data stores. For information about CloudTrail pricing, see AWS CloudTrail Pricing.

Logging Insights events with the AWS Management Console

You can enable Insights events on a trail or event data store using the console.

Enabling CloudTrail Insights events on an existing trail

Use the following procedure to enable CloudTrail Insights events on an existing trail. By default, Insights events are not enabled.

  1. In the left navigation pane of the CloudTrail console, open the Trails page, and choose a trail name.

  2. In Insights events choose Edit.

    Note

    Additional charges apply for logging Insights events. For CloudTrail pricing, see AWS CloudTrail Pricing.

  3. In Event type, choose Insights events.

  4. In Insights events, under Choose Insights types, choose API call rate, API error rate, or both. Your trail must be logging Write management events to log Insights events for API call rate. Your trail must be logging Read or Write management events to log Insights events for API error rate.

  5. Choose Save changes to save your changes.

It can take up to 36 hours for CloudTrail to deliver the first Insights events, if unusual activity is detected.

Enabling CloudTrail Insights events on an existing event data store

Use the following procedure to enable CloudTrail Insights events on an existing event data store. By default, Insights events are not enabled.

Additional charges apply for ingesting Insights events in CloudTrail Lake. You will be charged separately if you enable Insights for both trails and event data stores. For information about CloudTrail pricing, see AWS CloudTrail Pricing.

Note

You can only enable CloudTrail Insights events on event data stores containing CloudTrail management events. You cannot enable CloudTrail Insights events on other event data store types.

  1. In the left navigation pane of the CloudTrail console, under Lake, choose Event data stores.

  2. Choose the event data store name.

  3. In Management events, choose Edit.

  4. Choose Enable Insights.

  5. Choose the destination event data store where CloudTrail will deliver Insights events. The destination event data store will collect Insights events based upon the management event activity in this event data store. For information about how to create the destination event data store, see To create a destination event data store that logs Insights events.

  6. Under Choose Insights types, choose API call rate, API error rate, or both. Your event data store must be logging Write management events to log Insights events for API call rate. Your event data store must be logging Read or Write management events to log Insights events for API error rate.

  7. Choose Save changes to save your changes.

It can take up to 7 days for CloudTrail to deliver the first Insights events, if unusual activity is detected.

Logging Insights events with the AWS Command Line Interface

You can configure your trails and event data stores to log Insights events using the AWS CLI.

Note

To log Insights events on API call volume, the trail or event data store must log write management events. To log Insights events on API error rate, the trail or event data store must log read or write management events.

Logging Insights events for a trail using the AWS CLI

To view whether your trail is logging Insights events, run the get-insight-selectors command.

aws cloudtrail get-insight-selectors --trail-name TrailName

The following result shows the default settings for a trail. By default, trails don't log Insights events. The InsightType attribute value is empty, and no Insight event selectors are specified, because Insights event collection is not enabled.

If you do not add Insights selectors, the get-insight-selectors command returns the following error message: "An error occurred (InsightNotEnabledException) when calling the GetInsightSelectors operation: Trail name does not have Insights enabled. Edit the trail settings to enable Insights, and then try the operation again."

{
  "InsightSelectors": [ ],
  "TrailARN": "arn:aws:cloudtrail:us-east-1:123456789012:trail/TrailName"
}

To configure your trail to log Insights events, run the put-insight-selectors command. The following example shows how to configure your trail to include Insights events. Insights selector values can be ApiCallRateInsight, ApiErrorRateInsight, or both.

aws cloudtrail put-insight-selectors --trail-name TrailName --insight-selectors '[{"InsightType": "ApiCallRateInsight"},{"InsightType": "ApiErrorRateInsight"}]'

The following result shows the Insights event selector that is configured for the trail.

{
  "InsightSelectors":
      [
         {
            "InsightType": "ApiErrorRateInsight"
         },
         {
            "InsightType": "ApiCallRateInsight"
         }
      ],
  "TrailARN": "arn:aws:cloudtrail:us-east-1:123456789012:trail/TrailName"
}

Logging Insights events for an event data store using the AWS CLI

To enable Insights on an event data store, you must have a source event data store that logs management events and a destination event data store that logs Insights events.

To view whether Insights events are enabled on an event data store, run the get-insight-selectors command.

aws cloudtrail get-insight-selectors --event-data-store arn:aws:cloudtrail:us-east-1:123456789012:eventdatastore/EXAMPLE-f852-4e8f-8bd1-bcf6cEXAMPLE

To view whether an event data store is configured to receive Insights events or management events, run the get-event-data-store command.

aws cloudtrail get-event-data-store --event-data-store arn:aws:cloudtrail:us-east-1:123456789012:eventdatastore/EXAMPLE-d483-5c7d-4ac2-adb5dEXAMPLE

The following procedure shows you how to create the destination and source event data stores and then enable Insights events.

  1. Run the aws cloudtrail create-event-data-store command to create a destination event data store that collects Insights events. The value for eventCategory must be Insight. Replace retention-period-days with the number of days you would like to retain events in your event data store.

    If you are signed in with the management account for an AWS Organizations organization, include the --organization-enabled parameter if you want to give your delegated administrator access to the event data store.

    aws cloudtrail create-event-data-store \
--name insights-event-data-store \
--no-multi-region-enabled \
--retention-period retention-period-days \
--advanced-event-selectors '[
    {
      "Name": "Select Insights events",
      "FieldSelectors": [
          { "Field": "eventCategory", "Equals": ["Insight"] }
        ]
    }
  ]'

    The following is an example response.

    {
    "Name": "insights-event-data-store",
    "ARN": "arn:aws:cloudtrail:us-east-1:111122223333:eventdatastore/EXAMPLEf852-4e8f-8bd1-bcf6cEXAMPLE",
    "AdvancedEventSelectors": [
        {
           "Name": "Select Insights events",
           "FieldSelectors": [
              {
                  "Field": "eventCategory",
                  "Equals": [
                      "Insight"
                    ]
                }
            ]
        }
    ],
    "MultiRegionEnabled": false,
    "OrganizationEnabled": false,
    "BillingMode": "EXTENDABLE_RETENTION_PRICING",
    "RetentionPeriod": "90",
    "TerminationProtectionEnabled": true,
    "CreatedTimestamp": "2023-11-08T15:22:33.578000+00:00",
    "UpdatedTimestamp": "2023-11-08T15:22:33.714000+00:00"
}

    You will use the ARN (or ID suffix of the ARN) from the response as the value for the --insights-destination parameter in step 3.

  2. Run the aws cloudtrail create-event-data-store command to create a source event data store that logs management events. By default, event data stores log all management events. You don't need to specify the advanced event selectors if you want to log all management events. Replace retention-period-days with the number of days you would like to retain events in your event data store. If you are creating an organization event data store, include the --organization-enabled parameter.

    aws cloudtrail create-event-data-store --name source-event-data-store --retention-period retention-period-days

    The following is an example response.

    {
    "EventDataStoreArn": "arn:aws:cloudtrail:us-east-1:111122223333:eventdatastore/EXAMPLE9952-4ab9-49c0-b788-f4f3EXAMPLE",
    "Name": "source-event-data-store",
    "Status": "CREATED",
    "AdvancedEventSelectors": [
        {
            "Name": "Default management events",
            "FieldSelectors": [
                {
                    "Field": "eventCategory",
                    "Equals": [
                        "Management"
                    ]
                }
            ]
        }
    ],
    "MultiRegionEnabled": true,
    "OrganizationEnabled": false,
    "BillingMode": "EXTENDABLE_RETENTION_PRICING",
    "RetentionPeriod": 90,
    "TerminationProtectionEnabled": true,
    "CreatedTimestamp": "2023-11-08T15:25:35.578000+00:00",
    "UpdatedTimestamp": "2023-11-08T15:25:35.714000+00:00"
}

    You will use the ARN (or ID suffix of the ARN) from the response as the value for the --event-data-store parameter in step 3.

  3. Run the put-insight-selectors command to enable Insights events. Insights selector values can be ApiCallRateInsight, ApiErrorRateInsight, or both. For the --event-data-store parameter, specify the ARN (or ID suffix of the ARN) of the source event data store that logs management events and will enable Insights. For the --insights-destination parameter, specify the ARN (or ID suffix of the ARN) of the destination event data store that will log Insights events.

    aws cloudtrail put-insight-selectors --event-data-store arn:aws:cloudtrail:us-east-1:111122223333:eventdatastore/EXAMPLE9952-4ab9-49c0-b788-f4f3EXAMPLE --insights-destination arn:aws:cloudtrail:us-east-1:111122223333:eventdatastore/EXAMPLEf852-4e8f-8bd1-bcf6cEXAMPLE --insight-selectors '[{"InsightType": "ApiCallRateInsight"},{"InsightType": "ApiErrorRateInsight"}]'

    The following result shows the Insights event selector that is configured for the event data store.

    {
  "EventDataStoreARN": "arn:aws:cloudtrail:us-east-1:111122223333:eventdatastore/EXAMPLE9952-4ab9-49c0-b788-f4f3EXAMPLE",
  "InsightsDestination": "arn:aws:cloudtrail:us-east-1:111122223333:eventdatastore/EXAMPLEf852-4e8f-8bd1-bcf6cEXAMPLE",
  "InsightSelectors":
      [
         {
            "InsightType": "ApiErrorRateInsight"
         },
         {
            "InsightType": "ApiCallRateInsight"
         }
      ]
}

    After you enable CloudTrail Insights for the first time on an event data store, it can take up to 7 days for CloudTrail to deliver the first Insights event, if unusual activity is detected.

    CloudTrail Insights analyzes management events that occur in a single Region, not globally. A CloudTrail Insights event is generated in the same Region as its supporting management events are generated.

    For an organization event data store, CloudTrail analyzes management events from each member's account instead of analyzing the aggregation of all management events for the organization.

Additional charges apply for ingesting Insights events in CloudTrail Lake. You will be charged separately if you enable Insights for both trails and event data stores. For information about CloudTrail pricing, see AWS CloudTrail Pricing.

Logging Insights events with the AWS SDKs

Run the GetInsightSelectors operation to see whether your trail or event data store enables Insights events. You can configure your trails or event data stores to enable Insights events with the PutInsightSelectors operation. For more information, see the AWS CloudTrail API Reference.