Identify and alert when Amazon Data Firehose resources are not encrypted with an AWS KMS key
Created by Ram Kandaswamy (AWS)
Environment: Production | Technologies: Management & governance; Analytics; Big data; CloudNative; Infrastructure; Security, identity, compliance | AWS services: AWS CloudTrail; Amazon CloudWatch; AWS Identity and Access Management; Amazon Kinesis; AWS Lambda; Amazon SNS |
Summary
For compliance, some organizations must have encryption enabled on data delivery resources such as Amazon Data Firehose. This pattern shows a way to monitor, detect, and notify when resources are out of compliance.
To maintain the encryption requirement, this pattern can be used on Amazon Web Services (AWS) to provide automated monitoring and detection of Firehose delivery resources that are not encrypted with AWS Key Management Service (AWS KMS) key. The solution sends alert notifications, and it can be extended to perform automatic remediation. This solution can be applied to an individual account or a multiple-account environment, such as an environment using AWS Landing Zone or AWS Control Tower.
Prerequisites and limitations
Prerequisites
Firehose delivery stream
Sufficient permissions and familiarity with AWS CloudFormation, which is used in this infrastructure automation
Limitations
The solution is not real time because it uses AWS CloudTrail events for detection, and there is a delay between the time an unencrypted resource is created and the notification is sent.
Architecture
Target technology stack
The solution uses serverless technology and the following services:
AWS CloudTrail
Amazon CloudWatch
AWS Command Line Interface (AWS CLI)
AWS Identity and Access Management (IAM)
Amazon Data Firehose
AWS Lambda
Amazon Simple Notification Service (Amazon SNS)
Target architecture
A user creates or modifies Firehose.
A CloudTrail event is detected and matched.
Lambda is invoked.
Noncompliant resources are identified.
Email notification is sent.
Automation and scale
Using AWS CloudFormation StackSets, you can apply this solution to multiple AWS Regions or accounts with a single command.
Tools
AWS CloudTrail – AWS CloudTrail is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. Events include actions taken in the AWS Management Console, AWS Command Line Interface, and AWS SDKs and API operations.
Amazon CloudWatch Events – Amazon CloudWatch Events delivers a near-real-time stream of system events that describe changes in AWS resources.
AWS CLI – AWS Command Line Interface (AWS CLI) is an open source tool that enables you to interact with AWS services using commands in your command line shell.
IAM – AWS Identity and Access Management (IAM) is a web service that helps you securely control access to AWS resources. You use IAM to control who is authenticated (signed in) and authorized (has permissions) to use resources.
Amazon Data Firehose – Amazon Data Firehose is a fully managed service for delivering real-time streaming data. With Firehose, you don't need to write applications or manage resources. You configure your data producers to send data to Firehose, and it automatically delivers the data to the destination that you specified.
AWS Lambda – AWS Lambda is a compute service that supports running code without provisioning or managing servers. Lambda runs your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time that you consume—there is no charge when your code is not running.
Amazon SNS – Amazon Simple Notification Service (Amazon SNS) is a managed service that provides message delivery from publishers to subscribers (also known as producers and consumers).
Epics
Task | Description | Skills required |
---|---|---|
Deploy AWS CloudFormation StackSets. | In the AWS CLI, use the
| Cloud architect, Systems administrator |
Create stack instances. | Stacks need to be created in the AWS Regions of your choice as well as in one or more accounts. To create stack instances, run the following command, replacing the stack name, account numbers, and Regions with your own.
| Cloud architect, Systems administrator |
Related resources
Additional information
AWS Config does not support the Firehose delivery stream resource type, so an AWS Config rule cannot be used in the solution.
Attachments
To access additional content that is associated with this document, unzip the following file: attachment.zip