Use real-time logs - Amazon CloudFront
Create and use real-time log configurationsUnderstand real-time log configurationsCreate a Kinesis Data Streams consumerTroubleshoot real-time logs

Use real-time logs

With CloudFront real-time logs, you can get information about requests made to a distribution in real time (logs are delivered within seconds of receiving the requests). You can use real-time logs to monitor, analyze, and take action based on content delivery performance.

CloudFront real-time logs are configurable. You can choose:

  • The sampling rate for your real-time logs—that is, the percentage of requests for which you want to receive real-time log records.

  • The specific fields that you want to receive in the log records.

  • The specific cache behaviors (path patterns) that you want to receive real-time logs for.

CloudFront real-time logs are delivered to the data stream of your choice in Amazon Kinesis Data Streams. You can build your own Kinesis data stream consumer, or use Amazon Data Firehose to send the log data to Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon OpenSearch Service (OpenSearch Service), or a third-party log processing service.

CloudFront charges for real-time logs, in addition to the charges you incur for using Kinesis Data Streams. For more information about pricing, see Amazon CloudFront Pricing and Amazon Kinesis Data Streams pricing.

Important

We recommend that you use the logs to understand the nature of the requests for your content, not as a complete accounting of all requests. CloudFront delivers real-time logs on a best-effort basis. The log entry for a particular request might be delivered long after the request was actually processed and, in rare cases, a log entry might not be delivered at all. When a log entry is omitted from real-time logs, the number of entries in the real-time logs won't match the usage that appears in the AWS billing and usage reports.

Create and use real-time log configurations

To get information about requests made to a distribution in real time. you can use a real-time log configurations. Logs are delivered within seconds of receiving the requests. You can create a real-time log configuration in the CloudFront console, with the AWS Command Line Interface (AWS CLI), or with the CloudFront API.

To use a real-time log configuration, you attach it to one or more cache behaviors in a CloudFront distribution.

Console
To create a real-time log configuration

  1. Sign in to the AWS Management Console and open the Logs page in the CloudFront console at https://console.aws.amazon.com/cloudfront/v4/home?#/logs.

  2. Choose the Real-time configurations tab.

  3. Choose Create configuration.

  4. For Name, enter a name for the configuration.

  5. For Sampling rate, enter the percentage of requests for which you want to receive log records.

  6. For Fields, choose the fields to receive in the real-time logs.

    • To include all CMCD fields for your logs, choose CMCD all keys.

  7. For Endpoint, choose one or more Kinesis data streams to receive real-time logs.

    Note

    CloudFront real-time logs are delivered to the data stream that you specify in Kinesis Data Streams. To read and analyze your real-time logs, you can build your own Kinesis data stream consumer. You can also use Firehose to send the log data to Amazon S3, Amazon Redshift, Amazon OpenSearch Service, or a third-party log processing service.

  8. For IAM role, choose Create new service role or choose an existing role. You must have permission to create IAM roles.

  9. (Optional) For Distribution, choose a CloudFront distribution and cache behavior to attach to the real-time log configuration.

  10. Choose Create configuration.

If successful, the console shows the details of the real-time log configuration that you just created.

For more information, see Understand real-time log configurations.

AWS CLI

To create a real-time log configuration with the AWS CLI, use the aws cloudfront create-realtime-log-config command. You can use an input file to provide the command's input parameters, rather than specifying each individual parameter as command line input.

To create a real-time log configuration (CLI with input file)

  1. Use the following command to create a file named rtl-config.yaml that contains all of the input parameters for the create-realtime-log-config command.

    
aws cloudfront create-realtime-log-config --generate-cli-skeleton yaml-input > rtl-config.yaml

  2. Open the file named rtl-config.yaml that you just created. Edit the file to specify the real-time log configuration settings that you want, then save the file. Note the following:

    • For StreamType, the only valid value is Kinesis.

    For more information about the real-time long configuration settings, see Understand real-time log configurations.

  3. Use the following command to create the real-time log configuration using input parameters from the rtl-config.yaml file.

    
aws cloudfront create-realtime-log-config --cli-input-yaml file://rtl-config.yaml

If successful, the command's output shows the details of the real-time log configuration that you just created.

To attach a real-time log configuration to an existing distribution (CLI with input file)

  1. Use the following command to save the distribution configuration for the CloudFront distribution that you want to update. Replace distribution_ID with the distribution's ID.

    
aws cloudfront get-distribution-config --id distribution_ID --output yaml > dist-config.yaml

  2. Open the file named dist-config.yaml that you just created. Edit the file, making the following changes to each cache behavior that you are updating to use a real-time log configuration.

    • In the cache behavior, add a field named RealtimeLogConfigArn. For the field's value, use the ARN of the real-time log configuration that you want to attach to this cache behavior.

    • Rename the ETag field to IfMatch, but don't change the field's value.

    Save the file when finished.

  3. Use the following command to update the distribution to use the real-time log configuration. Replace distribution_ID with the distribution's ID.

    
aws cloudfront update-distribution --id distribution_ID --cli-input-yaml file://dist-config.yaml

If successful, the command's output shows the details of the distribution that you just updated.

API

To create a real-time log configuration with the CloudFront API, use the CreateRealtimeLogConfig API operation. For more information about the parameters that you specify in this API call, see Understand real-time log configurations and the API reference documentation for your AWS SDK or other API client.

After you create a real-time log configuration, you can attach it to a cache behavior, by using one of the following API operations:

For both of these API operations, provide the ARN of the real-time log configuration in the RealtimeLogConfigArn field, inside a cache behavior. For more information about the other fields that you specify in these API calls, see Distribution settings reference and the API reference documentation for your AWS SDK or other API client.

Understand real-time log configurations

To use CloudFront real-time logs, you start by creating a real-time log configuration. The real-time log configuration contains information about which log fields you want to receive, the sampling rate for log records, and the Kinesis data stream where you want to deliver the logs.

Specifically, a real-time log configuration contains the following settings:

Name

A name to identify the real-time log configuration.

Sampling rate

The sampling rate is a whole number between 1 and 100 (inclusive) that determines the percentage of viewer requests that are sent to Kinesis Data Streams as real-time log records. To include every viewer request in your real-time logs, specify 100 for the sampling rate. You might choose a lower sampling rate to reduce costs while still receiving a representative sample of request data in your real-time logs.

Fields

A list of the fields that are included in each real-time log record. Each log record can contain up to 40 fields, and you can choose to receive all of the available fields, or only the fields that you need for monitoring and analyzing performance.

The following list contains each field name and a description of the information in that field. The fields are listed in the order in which they appear in the log records that are delivered to Kinesis Data Streams.

Fields 46-63 are common media client data (CMCD) that media player clients can send to CDNs with each request. You can use this data to understand each request, such as the media type (audio, video), playback rate, and streaming length. These fields will only appear in your real-time logs if they're sent to CloudFront.

  1. timestamp

    The date and time at which the edge server finished responding to the request.

  2. c-ip

    The IP address of the viewer that made the request, for example, 192.0.2.183 or 2001:0db8:85a3::8a2e:0370:7334. If the viewer used an HTTP proxy or a load balancer to send the request, the value of this field is the IP address of the proxy or load balancer. See also the x-forwarded-for field.

  3. s-ip

    The IP address of the CloudFront server that served the request, for example, 192.0.2.183 or 2001:0db8:85a3::8a2e:0370:7334.

  4. time-to-first-byte

    The number of seconds between receiving the request and writing the first byte of the response, as measured on the server.

  5. sc-status

    The HTTP status code of the server's response (for example, 200).

  6. sc-bytes

    The total number of bytes that the server sent to the viewer in response to the request, including headers. For WebSocket and gRPC connections, this is the total number of bytes sent from the server to the client through the connection.

  7. cs-method

    The HTTP request method received from the viewer.

  8. cs-protocol

    The protocol of the viewer request (http, https, grpcs, ws, or wss).

  9. cs-host

    The value that the viewer included in the Host header of the request. If you're using the CloudFront domain name in your object URLs (such as d111111abcdef8.cloudfront.net), this field contains that domain name. If you're using alternate domain names (CNAMEs) in your object URLs (such as www.example.com), this field contains the alternate domain name.

  10. cs-uri-stem

    The entire request URL, including the query string (if one exists), but without the domain name. For example, /images/cat.jpg?mobile=true.

    Note

    In standard logs, the cs-uri-stem value doesn't include the query string.

  11. cs-bytes

    The total number of bytes of data that the viewer included in the request, including headers. For WebSocket and gRPC connections, this is the total number of bytes sent from the client to the server on the connection.

  12. x-edge-location

    The edge location that served the request. Each edge location is identified by a three-letter code and an arbitrarily assigned number (for example, DFW3). The three-letter code typically corresponds with the International Air Transport Association (IATA) airport code for an airport near the edge location's geographic location. (These abbreviations might change in the future.)

  13. x-edge-request-id

    An opaque string that uniquely identifies a request. CloudFront also sends this string in the x-amz-cf-id response header.

  14. x-host-header

    The domain name of the CloudFront distribution (for example, d111111abcdef8.cloudfront.net).

  15. time-taken

    The number of seconds (to the thousandth of a second, for example, 0.082) from when the server receives the viewer's request to when the server writes the last byte of the response to the output queue, as measured on the server. From the perspective of the viewer, the total time to get the full response will be longer than this value because of network latency and TCP buffering.

  16. cs-protocol-version

    The HTTP version that the viewer specified in the request. Possible values include HTTP/0.9, HTTP/1.0, HTTP/1.1, HTTP/2.0, and HTTP/3.0.

  17. c-ip-version

    The IP version of the request (IPv4 or IPv6).

  18. cs-user-agent

    The value of the User-Agent header in the request. The User-Agent header identifies the source of the request, such as the type of device and browser that submitted the request or, if the request came from a search engine, which search engine.

  19. cs-referer

    The value of the Referer header in the request. This is the name of the domain that originated the request. Common referrers include search engines, other websites that link directly to your objects, and your own website.

  20. cs-cookie

    The Cookie header in the request, including name—value pairs and the associated attributes.

    Note

    This field is truncated to 800 bytes.

  21. cs-uri-query

    The query string portion of the request URL, if any.

  22. x-edge-response-result-type

    How the server classified the response just before returning the response to the viewer. See also the x-edge-result-type field. Possible values include:

    • Hit – The server served the object to the viewer from the cache.

    • RefreshHit – The server found the object in the cache but the object had expired, so the server contacted the origin to verify that the cache had the latest version of the object.

    • Miss – The request could not be satisfied by an object in the cache, so the server forwarded the request to the origin server and returned the result to the viewer.

    • LimitExceeded – The request was denied because a CloudFront quota (formerly referred to as a limit) was exceeded.

    • CapacityExceeded – The server returned a 503 error because it didn't have enough capacity at the time of the request to serve the object.

    • Error – Typically, this means the request resulted in a client error (the value of the sc-status field is in the 4xx range) or a server error (the value of the sc-status field is in the 5xx range).

      If the value of the x-edge-result-type field is Error and the value of this field is not Error, the client disconnected before finishing the download.

    • Redirect – The server redirected the viewer from HTTP to HTTPS according to the distribution settings.

  23. x-forwarded-for

    If the viewer used an HTTP proxy or a load balancer to send the request, the value of the c-ip field is the IP address of the proxy or load balancer. In that case, this field is the IP address of the viewer that originated the request. This field can contain multiple comma-separated IP addresses. Each IP address can be an IPv4 address (for example, 192.0.2.183) or an IPv6 address (for example, 2001:0db8:85a3::8a2e:0370:7334).

  24. ssl-protocol

    When the request used HTTPS, this field contains the SSL/TLS protocol that the viewer and server negotiated for transmitting the request and response. For a list of possible values, see the supported SSL/TLS protocols in Supported protocols and ciphers between viewers and CloudFront.

  25. ssl-cipher

    When the request used HTTPS, this field contains the SSL/TLS cipher that the viewer and server negotiated for encrypting the request and response. For a list of possible values, see the supported SSL/TLS ciphers in Supported protocols and ciphers between viewers and CloudFront.

  26. x-edge-result-type

    How the server classified the response after the last byte left the server. In some cases, the result type can change between the time that the server is ready to send the response and the time that it finishes sending the response. See also the x-edge-response-result-type field.

    For example, in HTTP streaming, suppose the server finds a segment of the stream in the cache. In that scenario, the value of this field would ordinarily be Hit. However, if the viewer closes the connection before the server has delivered the entire segment, the final result type (and the value of this field) is Error.

    WebSocket and gRPC connections will have a value of Miss for this field because the content is not cacheable and is proxied directly to the origin.

    Possible values include:

    • Hit – The server served the object to the viewer from the cache.

    • RefreshHit – The server found the object in the cache but the object had expired, so the server contacted the origin to verify that the cache had the latest version of the object.

    • Miss – The request could not be satisfied by an object in the cache, so the server forwarded the request to the origin and returned the result to the viewer.

    • LimitExceeded – The request was denied because a CloudFront quota (formerly referred to as a limit) was exceeded.

    • CapacityExceeded – The server returned an HTTP 503 status code because it didn't have enough capacity at the time of the request to serve the object.

    • Error – Typically, this means the request resulted in a client error (the value of the sc-status field is in the 4xx range) or a server error (the value of the sc-status field is in the 5xx range). If the value of the sc-status field is 200, or if the value of this field is Error and the value of the x-edge-response-result-type field is not Error, it means the HTTP request was successful but the client disconnected before receiving all of the bytes.

    • Redirect – The server redirected the viewer from HTTP to HTTPS according to the distribution settings.

  27. fle-encrypted-fields

    The number of field-level encryption fields that the server encrypted and forwarded to the origin. CloudFront servers stream the processed request to the origin as they encrypt data, so this field can have a value even if the value of fle-status is an error.

  28. fle-status

    When field-level encryption is configured for a distribution, this field contains a code that indicates whether the request body was successfully processed. When the server successfully processes the request body, encrypts values in the specified fields, and forwards the request to the origin, the value of this field is Processed. The value of x-edge-result-type can still indicate a client-side or server-side error in this case.

    Possible values for this field include:

    • ForwardedByContentType – The server forwarded the request to the origin without parsing or encryption because no content type was configured.

    • ForwardedByQueryArgs – The server forwarded the request to the origin without parsing or encryption because the request contains a query argument that wasn't in the configuration for field-level encryption.

    • ForwardedDueToNoProfile – The server forwarded the request to the origin without parsing or encryption because no profile was specified in the configuration for field-level encryption.

    • MalformedContentTypeClientError – The server rejected the request and returned an HTTP 400 status code to the viewer because the value of the Content-Type header was in an invalid format.

    • MalformedInputClientError – The server rejected the request and returned an HTTP 400 status code to the viewer because the request body was in an invalid format.

    • MalformedQueryArgsClientError – The server rejected the request and returned an HTTP 400 status code to the viewer because a query argument was empty or in an invalid format.

    • RejectedByContentType – The server rejected the request and returned an HTTP 400 status code to the viewer because no content type was specified in the configuration for field-level encryption.

    • RejectedByQueryArgs – The server rejected the request and returned an HTTP 400 status code to the viewer because no query argument was specified in the configuration for field-level encryption.

    • ServerError – The origin server returned an error.

    If the request exceeds a field-level encryption quota (formerly referred to as a limit), this field contains one of the following error codes, and the server returns HTTP status code 400 to the viewer. For a list of the current quotas on field-level encryption, see Quotas on field-level encryption.

    • FieldLengthLimitClientError – A field that is configured to be encrypted exceeded the maximum length allowed.

    • FieldNumberLimitClientError – A request that the distribution is configured to encrypt contains more than the number of fields allowed.

    • RequestLengthLimitClientError – The length of the request body exceeded the maximum length allowed when field-level encryption is configured.

  29. sc-content-type

    The value of the HTTP Content-Type header of the response.

  30. sc-content-len

    The value of the HTTP Content-Length header of the response.

  31. sc-range-start

    When the response contains the HTTP Content-Range header, this field contains the range start value.

  32. sc-range-end

    When the response contains the HTTP Content-Range header, this field contains the range end value.

  33. c-port

    The port number of the request from the viewer.

  34. x-edge-detailed-result-type

    This field contains the same value as the x-edge-result-type field, except in the following cases:

    • When the object was served to the viewer from the Origin Shield layer, this field contains OriginShieldHit.

    • When the object was not in the CloudFront cache and the response was generated by an origin request Lambda@Edge function, this field contains MissGeneratedResponse.

    • When the value of the x-edge-result-type field is Error, this field contains one of the following values with more information about the error:

      • AbortedOrigin – The server encountered an issue with the origin.

      • ClientCommError – The response to the viewer was interrupted due to a communication problem between the server and the viewer.

      • ClientGeoBlocked – The distribution is configured to refuse requests from the viewer's geographic location.

      • ClientHungUpRequest – The viewer stopped prematurely while sending the request.

      • Error – An error occurred for which the error type doesn't fit any of the other categories. This error type can occur when the server serves an error response from the cache.

      • InvalidRequest – The server received an invalid request from the viewer.

      • InvalidRequestBlocked – Access to the requested resource is blocked.

      • InvalidRequestCertificate – The distribution doesn't match the SSL/TLS certificate for which the HTTPS connection was established.

      • InvalidRequestHeader – The request contained an invalid header.

      • InvalidRequestMethod – The distribution is not configured to handle the HTTP request method that was used. This can happen when the distribution supports only cacheable requests.

      • OriginCommError – The request timed out while connecting to the origin, or reading data from the origin.

      • OriginConnectError – The server couldn't connect to the origin.

      • OriginContentRangeLengthError – The Content-Length header in the origin's response doesn't match the length in the Content-Range header.

      • OriginDnsError – The server couldn't resolve the origin's domain name.

      • OriginError – The origin returned an incorrect response.

      • OriginHeaderTooBigError – A header returned by the origin is too big for the edge server to process.

      • OriginInvalidResponseError – The origin returned an invalid response.

      • OriginReadError – The server couldn't read from the origin.

      • OriginWriteError – The server couldn't write to the origin.

      • OriginZeroSizeObjectError – A zero size object sent from the origin resulted in an error.

      • SlowReaderOriginError – The viewer was slow to read the message that caused the origin error.

  35. c-country

    A country code that represents the viewer's geographic location, as determined by the viewer's IP address. For a list of country codes, see ISO 3166-1 alpha-2.

  36. cs-accept-encoding

    The value of the Accept-Encoding header in the viewer request.

  37. cs-accept

    The value of the Accept header in the viewer request.

  38. cache-behavior-path-pattern

    The path pattern that identifies the cache behavior that matched the viewer request.

  39. cs-headers

    The HTTP headers (names and values) in the viewer request.

    Note

    This field is truncated to 800 bytes.

  40. cs-header-names

    The names of the HTTP headers (not values) in the viewer request.

    Note

    This field is truncated to 800 bytes.

  41. cs-headers-count

    The number of HTTP headers in the viewer request.

  42. origin-fbl

    The number of seconds of first-byte latency between CloudFront and your origin.

  43. origin-lbl

    The number of seconds of last-byte latency between CloudFront and your origin.

  44. asn

    The autonomous system number (ASN) of the viewer.

  45. primary-distribution-id

    When continuous deployment is enabled, this ID identifies which distribution is the primary in the current distribution.

  46. primary-distribution-dns-name

    When continuous deployment is enabled, this value shows the primary domain name that is related to the current CloudFront distribution (for example, d111111abcdef8.cloudfront.net).

    CMCD fields in real-time logs

    For more information about these fields, see the CTA Specification Web Application Video Ecosystem - Common Media Client Data CTA-5004 document.

  47. cmcd-encoded-bitrate

    The encoded bitrate of the requested audio or video object.

  48. cmcd-buffer-length

    The buffer length of the requested media object.

  49. cmcd-buffer-starvation

    Whether the buffer was starved at some point between the prior request and the object request. This can cause the player to be in a rebuffering stat, which can stall the video or audio playback.

  50. cmcd-content-id

    A unique string that identifies the current content.

  51. cmcd-object-duration

    The playback duration of the requested object (in milliseconds).

  52. cmcd-deadline

    The deadline from the request time that the first sample of this object must be available, so that a buffer underrun state or other playback problems are avoided.

  53. cmcd-measured-throughput

    The throughput between the client and server, as measured by the client.

  54. cmcd-next-object-request

    The relative path of the next requested object.

  55. cmcd-next-range-request

    If the next request is a partial object request, this string denotes the byte range to be requested.

  56. cmcd-object-type

    The media type of the current object being requested.

  57. cmcd-playback-rate

    1 if real-time, 2 if double-speed, 0 if not playing.

  58. cmcd-requested-maximum-throughput

    The requested maximum throughput that the client considers sufficient for asset delivery.

  59. cmcd-streaming-format

    The streaming format that defines the current request.

  60. cmcd-session-id

    A GUID identifying the current playback session.

  61. cmcd-stream-type

    Token identifying segment availability. v = all segments are available. l = segments become available over time.

  62. cmcd-startup

    Key is included without a value if the object is needed urgently during startup, seeking, or recovery after a buffer-empty event.

  63. cmcd-top-bitrate

    The highest bitrate rendition that the client can play.

  64. cmcd-version

    The version of this specification used for interpreting the defined key names and values. If this key is omitted, the client and server must interpret the values as being defined by version 1.

Endpoint (Kinesis Data Streams)

The endpoint contains information about the Kinesis Data Streams where you want to send real-time logs. You provide the Amazon Resource Name (ARN) of the data stream.

For more information about creating a Kinesis Data Streams, see the following topics in the Amazon Kinesis Data Streams Developer Guide.

When you create a data stream, you need to specify the number of shards. Use the following information to help you estimate the number of shards you need.

To estimate the number of shards for your Kinesis data stream

  1. Calculate (or estimate) the number of requests per second that your CloudFront distribution receives.

    You can use the CloudFront usage reports (in the CloudFront console) and the CloudFront metrics (in the CloudFront and Amazon CloudWatch consoles) to help you calculate your requests per second.

  2. Determine the typical size of a single real-time log record.

    In general, a single log record is around 500 bytes. A large record that includes all available fields is generally around 1 KB.

    If you're not sure what your log record size is, you can enable real-time logs with a low sampling rate (for example, 1%), and then calculate the average record size using monitoring data in Kinesis Data Streams (total incoming bytes divided by total number of records).

  3. On the Amazon Kinesis Data Streams pricing page, under AWS Pricing Calculator, choose Create your custom estimate now.

    • In the calculator, enter the number of requests (records) per second.

    • Enter the average record size of a single log record.

    • Choose Show calculations.

    The pricing calculator shows you the number of shards you need and the estimated cost.

IAM role

The AWS Identity and Access Management (IAM) role that gives CloudFront permission to deliver real-time logs to your Kinesis data stream.

When you create a real-time log configuration with the CloudFront console, you can choose Create new service role to let the console create the IAM role for you.

When you create a real-time log configuration with AWS CloudFormation or the CloudFront API (AWS CLI or SDK), you must create the IAM role yourself and provide the role ARN. To create the IAM role yourself, use the following policies.

IAM role trust policy

To use the following IAM role trust policy, replace 111122223333 with your AWS account number. The Condition element in this policy helps to prevent the confused deputy problem because CloudFront can only assume this role on behalf of a distribution in your AWS account.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "111122223333"
                }
            }
        }
    ]
}

IAM role permissions policy for an unencrypted data stream

To use the following policy, replace arn:aws:kinesis:us-east-2:123456789012:stream/StreamName with the ARN of your Kinesis data stream.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:DescribeStreamSummary",
                "kinesis:DescribeStream",
                "kinesis:PutRecord",
                "kinesis:PutRecords"
            ],
            "Resource": [
                "arn:aws:kinesis:us-east-2:123456789012:stream/StreamName"
            ]
        }
    ]
}

IAM role permissions policy for an encrypted data stream

To use the following policy, replace arn:aws:kinesis:us-east-2:123456789012:stream/StreamName with the ARN of your Kinesis data stream and arn:aws:kms:us-east-2:123456789012:key/e58a3d0b-fe4f-4047-a495-ae03cc73d486 with the ARN of your AWS KMS key.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:DescribeStreamSummary",
                "kinesis:DescribeStream",
                "kinesis:PutRecord",
                "kinesis:PutRecords"
            ],
            "Resource": [
                "arn:aws:kinesis:us-east-2:123456789012:stream/StreamName"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:GenerateDataKey"
            ],
            "Resource": [
                "arn:aws:kms:us-east-2:123456789012:key/e58a3d0b-fe4f-4047-a495-ae03cc73d486"
            ]
        }
    ]
}

Create a Kinesis Data Streams consumer

To read and analyze your real-time logs, you build or use a Kinesis Data Streams consumer. When you build a consumer for CloudFront real-time logs, it's important to know that the fields in every real-time log record are always delivered in the same order, as listed in the Fields section. Make sure that you build your consumer to accommodate this fixed order.

For example, consider a real-time log configuration that includes only these three fields: time-to-first-byte, sc-status, and c-country. In this scenario, the last field, c-country, is always field number 3 in every log record. However, if you later add fields to the real-time log configuration, the placement of each field in a record can change.

For example, if you add the fields sc-bytes and time-taken to the real-time log configuration, these fields are inserted into each log record according to the order shown in the Fields section. The resulting order of all five fields is time-to-first-byte, sc-status, sc-bytes, time-taken, and c-country. The c-country field was originally field number 3, but is now field number 5. Make sure that your consumer application can handle fields that change position in a log record, in case you add fields to your real-time log configuration.

Troubleshoot real-time logs

After you create a real-time log configuration, you might find that no records (or not all records) are delivered to Kinesis Data Streams. In this case, you should first verify that your CloudFront distribution is receiving viewer requests. If it is, you can check the following setting to continue troubleshooting.

IAM role permissions

To deliver real-time log records to your Kinesis data stream, CloudFront uses the IAM role in the real-time log configuration. Make sure that the role trust policy and the role permissions policy match the policies shown in IAM role.

Kinesis Data Streams throttling

If CloudFront writes real-time log records to your Kinesis data stream faster than the stream can handle, Kinesis Data Streams might throttle the requests from CloudFront. In this case, you can increase the number of shards in your Kinesis data stream. Each shard can support writes up to 1,000 records per second, up to a maximum data write of 1 MB per second.