# Data protection in Amazon Kinesis Data Streams Server-side encryption using AWS Key Management Service (AWS KMS) keys makes it easy for you to meet strict data management requirements by encrypting your data at rest within Amazon Kinesis Data Streams. **Note** If you require FIPS 140-2 validated cryptographic modules when accessing AWS through a command line interface or an API, use a FIPS endpoint. For more information about the available FIPS endpoints, see [Federal Information Processing Standard (FIPS) 140-2](https://aws.amazon.com/compliance/fips/). **Topics** + [ # What is server-side encryption for Kinesis Data Streams? ](what-is-sse.md) + [ # Costs, Regions, and performance considerations ](costs-performance.md) + [ # How do I get started with server-side encryption? ](getting-started-with-sse.md) + [ # Create and use user-generated KMS keys ](creating-using-sse-master-keys.md) + [ # Permissions to use user-generated KMS keys ](permissions-user-key-KMS.md) + [ # Verify and Troubleshoot KMS key permissions ](sse-troubleshooting.md) + [ # Use Amazon Kinesis Data Streams with interface VPC endpoints ](vpc.md) # What is server-side encryption for Kinesis Data Streams? Server-side encryption is a feature in Amazon Kinesis Data Streams that automatically encrypts data before it's at rest by using an AWS KMS customer master key (CMK) you specify. Data is encrypted before it's written to the Kinesis stream storage layer, and decrypted after it’s retrieved from storage. As a result, your data is encrypted at rest within the Kinesis Data Streams service. This allows you to meet strict regulatory requirements and enhance the security of your data. With server-side encryption, your Kinesis stream producers and consumers don't need to manage master keys or cryptographic operations. Your data is automatically encrypted as it enters and leaves the Kinesis Data Streams service, so your data at rest is encrypted. AWS KMS provides all the master keys that are used by the server-side encryption feature. AWS KMS makes it easy to use a CMK for Kinesis that is managed by AWS, a user-specified AWS KMS CMK, or a master key imported into the AWS KMS service. **Note** Server-side encryption encrypts incoming data only after encryption is enabled. Preexisting data in an unencrypted stream is not encrypted after server-side encryption is enabled. When encrypting your data streams and sharing access to other principals, you must grant permission in both the key policy for the AWS KMS key and the IAM policies in the external account. For more information, see [Allowing users in other accounts to use a KMS key](https://docs.aws.amazon.com/kms/latest/developerguide/key-policy-modifying-external-accounts.html). If you have enabled server-side encryption for a data stream with AWS managed KMS key and want to share access via a resource policy, you must switch to using customer-managed key (CMK), as shown following: ![\[Encryption settings interface with options for server-side encryption and customer-managed CMK.\]](http://docs.aws.amazon.com/streams/latest/dev/images/cmk2.png) In addition, you must allow your sharing principal entities to have access to your CMK, using KMS cross account sharing capabilities. Make sure to also make the change in the IAM policies for the sharing principal entities. For more information, see [Allowing users in other accounts to use a KMS key](https://docs.aws.amazon.com/kms/latest/developerguide/key-policy-modifying-external-accounts.html). # Costs, Regions, and performance considerations When you apply server-side encryption, you are subject to AWS KMS API usage and key costs. Unlike custom KMS master keys, the `(Default) aws/kinesis` customer master key (CMK) is offered free of charge. However, you still must pay for the API usage costs that Amazon Kinesis Data Streams incurs on your behalf. API usage costs apply for every CMK, including custom ones. Kinesis Data Streams calls AWS KMS approximately every five minutes when it is rotating the data key. In a 30-day month, the total cost of AWS KMS API calls that are initiated by a Kinesis stream should be less than a few dollars. This cost scales with the number of user credentials that you use on your data producers and consumers because each user credential requires a unique API call to AWS KMS. When you use an IAM role for authentication, each assume role call results in unique user credentials. To save KMS costs, you might want to cache user credentials that are returned by the assume role call. The following describes the costs by resource: **Keys** + The CMK for Kinesis that's managed by AWS (alias = `aws/kinesis`) is free. + User-generated KMS keys are subject to KMS key costs. For more information, see [AWS Key Management Service Pricing](http://aws.amazon.com/kms/pricing/#Keys). API usage costs apply for every CMK, including custom ones. Kinesis Data Streams calls KMS approximately every 5 minutes when it is rotating the data key. In a 30-day month, the total cost of KMS API calls initiated by a Kinesis data stream should be less than a few dollars. Please note that this cost scales with the number of user credentials you use on your data producers and consumers because each user credential requires a unique API call to AWS KMS. When you use IAM role for authentication, each assume-role-call will result in unique user credentials and you might want to cache user credentials returned by the assume-role-call to save KMS costs. ## KMS API usage For every encrypted stream, when reading from TIP and using a single IAM account/user access key across readers and writers, Kinesis service calls the AWS KMS service approximately 12 times every 5 minutes. Not reading from TIP could lead to higher calls to AWS KMS service. API requests to generate new data encryption keys are subject to AWS KMS usage costs. For more information, see [AWS Key Management Service Pricing: Usage](http://aws.amazon.com/kms/pricing/#Usage). ## Availability of server-side encryption by Region Currently, server-side encryption of Kinesis streams is available in all the Regions supported for Kinesis Data Streams, including AWS GovCloud (US-West), and the China Regions. For more information about supported Regions for Kinesis Data Streams see [https://docs.aws.amazon.com/general/latest/gr/ak.html](https://docs.aws.amazon.com/general/latest/gr/ak.html). ## Performance Considerations Due to the service overhead of applying encryption, applying server-side encryption increases the typical latency of `PutRecord`, `PutRecords`, and `GetRecords` by less than 100μs. # How do I get started with server-side encryption? The easiest way to get started with server-side encryption is to use the AWS Management Console and the Amazon Kinesis KMS Service Key, `aws/kinesis`. The following procedure demonstrates how to enable server-side encryption for a Kinesis stream. **To enable server-side encryption for a Kinesis stream** 1. Sign in to the AWS Management Console and open the [Amazon Kinesis Data Streams console](http://console.aws.amazon.com/kinesis/home?region=us-east-1#/streams/list). 1. Create or select a Kinesis stream in the AWS Management Console. 1. Choose the **details** tab. 1. In **Server-side encryption**, choose **edit**. 1. Unless you want to use a user-generated KMS master key, ensure the **(Default) aws/kinesis** KMS master key is selected. This is the KMS master key generated by the Kinesis service. Choose **Enabled**, and then choose **Save**. **Note** The default Kinesis service master key is free, however, the API calls made by Kinesis to the AWS KMS service are subject to KMS usage costs. 1. The stream transitions through a **pending** state. After the stream returns to an **active** state with encryption enabled, all incoming data written to the stream is encrypted using the KMS master key you selected. 1. To disable server-side encryption, choose **Disabled** in **Server-side encryption** in the AWS Management Console, and then choose **Save**. # Create and use user-generated KMS keys This section describes how to create and use your own KMS keys, instead of using the master key administered by Amazon Kinesis. ## Create user-generated KMS keys For instructions on creating your own keys, see [Creating Keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html) in the *AWS Key Management Service Developer Guide*. After you create keys for your account, the Kinesis Data Streams service returns these keys in the **KMS master key** list. ## Use user-generated KMS keys After the correct permissions are applied to your consumers, producers, and administrators, you can use custom KMS keys in your own AWS account or another AWS account. All KMS master keys in your account appear in the **KMS Master Key** list within the AWS Management Console. To use custom KMS master keys located in another account, you need permissions to use those keys. You must also specify the ARN of the KMS master key in the ARN input box in the AWS Management Console. # Permissions to use user-generated KMS keys Before you can use server-side encryption with a user-generated KMS key, you must configure AWS KMS key policies to allow encryption of streams and encryption and decryption of stream records. For examples and more information about AWS KMS permissions, see [AWS KMS API Permissions: Actions and Resources Reference](https://docs.aws.amazon.com/kms/latest/developerguide/kms-api-permissions-reference.html). **Note** The use of the default service key for encryption does not require application of custom IAM permissions. Before you use user-generated KMS master keys, ensure that your Kinesis stream producers and consumers (IAM principals) are users in the KMS master key policy. Otherwise, writes and reads from a stream will fail, which could ultimately result in data loss, delayed processing, or hung applications. You can manage permissions for KMS keys using IAM policies. For more information, see [Using IAM Policies with AWS KMS](http://docs.aws.amazon.com/kms/latest/developerguide/iam-policies.html). ## Kinesis Data Streams encryption context When Amazon Kinesis Data Streams calls AWS KMS on your behalf, it passes an encryption context to AWS KMS that can be used as a condition for authorization in key policies and grants. Kinesis Data Streams uses the stream ARN as the encryption context in all AWS KMS calls. ``` "encryptionContext": { "aws:kinesis:arn": "arn:aws:kinesis:region:account-id:stream/stream-name" } ``` You can use the encryption context to identify the use of your KMS key in audit records and logs. It also appears in plaintext in logs, such as AWS CloudTrail. To limit the use of your KMS key to requests from Kinesis Data Streams for a specific stream, use the `kms:EncryptionContext:aws:kinesis:arn` condition key in the KMS key policy or IAM policy. ## Example producer permissions Your Kinesis stream producers must have the `kms:GenerateDataKey` permission. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "kms:GenerateDataKey" ], "Resource": "arn:aws:kms:us-west-2:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab" }, { "Effect": "Allow", "Action": [ "kinesis:PutRecord", "kinesis:PutRecords" ], "Resource": "arn:aws:kinesis:*:123456789012:MyStream" } ] } ``` ------ ## Example consumer permissions Your Kinesis stream consumers must have the `kms:Decrypt` permission. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource": "arn:aws:kms:us-west-2:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab" }, { "Effect": "Allow", "Action": [ "kinesis:GetRecords", "kinesis:DescribeStream" ], "Resource": "arn:aws:kinesis:*:123456789012:MyStream" } ] } ``` ------ Amazon Managed Service for Apache Flink and AWS Lambda use roles to consume Kinesis streams. Make sure to add the `kms:Decrypt` permission to the roles that these consumers use. ## Stream administrator permissions Kinesis stream administrators must have authorization to call `kms:List*` and ```kms:DescribeKey*`. # Verify and Troubleshoot KMS key permissions After enabling encryption on a Kinesis stream, we recommend that you monitor the success of your `putRecord`, `putRecords`, and `getRecords` calls using the following Amazon CloudWatch metrics: + `PutRecord.Success` + `PutRecords.Success` + `GetRecords.Success` For more information, see [Monitor Kinesis Data Streams](monitoring.md) # Use Amazon Kinesis Data Streams with interface VPC endpoints You can use an interface VPC endpoint to prevent traffic between your Amazon VPC and Kinesis Data Streams from leaving the Amazon network. Interface VPC endpoints don't require an internet gateway, NAT device, VPN connection, or Direct Connect connection. Interface VPC endpoints are powered by AWS PrivateLink, an AWS technology that enables private communication between AWS services using an elastic network interface with private IPs in your Amazon VPC. For more information, see [Amazon Virtual Private Cloud](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Introduction.html) and [Interface VPC Endpoints (AWS PrivateLink)](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint). **Topics** + [ ## Use interface VPC endpoints for Kinesis Data Streams ](#using-interface-vpc-endpoints) + [ ## Control access to VPC endpoints for Kinesis Data Streams ](#interface-vpc-endpoints-policies) + [ ## Availability of VPC endpoint policies for Kinesis Data Streams ](#availability) ## Use interface VPC endpoints for Kinesis Data Streams To get started, you do not need to change the settings for your streams, producers, or consumers. Create an interface VPC endpoint for your Kinesis Data Streams to start traffic flowing from and to your Amazon VPC resources through the interface VPC endpoint. FIPS-enabled interface VPC endpoints are available for US Regions. For more information, see [Creating an Interface Endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint). The Amazon Kinesis Producer Library (KPL) and Kinesis Consumer Library (KCL) call AWS services like Amazon CloudWatch and Amazon DynamoDB using either public endpoints or private interface VPC endpoints, whichever are in use. For example, if your KCL application is running in a VPC with DynamoDB interface with VPC endpoints enabled, calls between DynamoDB and your KCL application flow through the interface VPC endpoint. ## Control access to VPC endpoints for Kinesis Data Streams VPC endpoint policies let you control access by either attaching a policy to a VPC endpoint or by using additional fields in a policy that is attached to an IAM user, group, or role to restrict access to occur only through the specified VPC endpoint. Use these policies to restrict access to specific streams to a specified VPC endpoint when using them together with the IAM policies to grant only access to Kinesis data stream actions through the specified VPC endpoint. The following are example endpoint policies for accessing Kinesis data streams. + **VPC policy example: read-only access** - this sample policy can be attached to a VPC endpoint. (For more information, see [Controlling Access to Amazon VPC Resources](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_IAM.html)). It restricts actions to only listing and describing a Kinesis data stream through the VPC endpoint to which it is attached. ``` { "Statement": [ { "Sid": "ReadOnly", "Principal": "*", "Action": [ "kinesis:List*", "kinesis:Describe*" ], "Effect": "Allow", "Resource": "*" } ] } ``` + **VPC policy example: restrict access to a specific Kinesis data stream** - this sample policy can be attached to a VPC endpoint. It restricts access to a specific data stream through the VPC endpoint to which it is attached. ``` { "Statement": [ { "Sid": "AccessToSpecificDataStream", "Principal": "*", "Action": "kinesis:*", "Effect": "Allow", "Resource": "arn:aws:kinesis:us-east-1:123456789012:stream/MyStream" } ] } ``` + **IAM policy example: restrict access to a specific stream from a specific VPC endpoint only** - this sample policy can be attached to an IAM user, role, or group. It restricts access to a specified Kinesis data stream to occur only from a specified VPC endpoint. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "AccessFromSpecificEndpoint", "Action": "kinesis:*", "Effect": "Deny", "Resource": "arn:aws:kinesis:us-east-1:123456789012:stream/MyStream", "Condition": { "StringNotEquals" : { "aws:sourceVpce": "vpce-11aa22bb" } } } ] } ``` ------ ## Availability of VPC endpoint policies for Kinesis Data Streams Kinesis Data Streams interface VPC endpoints with policies are supported in the following Regions: + Europe (Paris) + Europe (Ireland) + US East (N. Virginia) + Europe (Stockholm) + US East (Ohio) + Europe (Frankfurt) + South America (São Paulo) + Europe (London) + Asia Pacific (Tokyo) + US West (N. California) + Asia Pacific (Singapore) + Asia Pacific (Sydney) + China (Beijing) + China (Ningxia) + Asia Pacific (Hong Kong) + Middle East (Bahrain) + Middle East (UAE) + Europe (Milan) + Africa (Cape Town) + Asia Pacific (Mumbai) + Asia Pacific (Seoul) + Canada (Central) + US West (Oregon) except usw2-az4 + AWS GovCloud (US-East) + AWS GovCloud (US-West) + Asia Pacific (Osaka) + Europe (Zurich) + Asia Pacific (Hyderabad)