Create alarms for custom metrics using Amazon CloudWatch anomaly detection - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpicsRelated resourcesAttachments

Create alarms for custom metrics using Amazon CloudWatch anomaly detection

Created by Ram Kandaswamy (AWS) and Raheem Jiwani (AWS)

Summary

On the Amazon Web Services (AWS) Cloud, you can use Amazon CloudWatch to create alarms that monitor metrics and send notifications or automatically make changes if a threshold is breached.

To avoid being limited by static thresholds, you can create alarms based on past patterns and that notify you if specific metrics are outside the normal operating window. For example, you could monitor your API’s response times from Amazon API Gateway and receive notifications about anomalies that prevent you from meeting a service-level agreement (SLA).

This pattern describes how to use CloudWatch anomaly detection for custom metrics. The pattern shows you how to create a custom metric in Amazon CloudWatch Logs Insights or publish a custom metric with an AWS Lambda function, and then set up anomaly detection and create notifications using Amazon Simple Notification Service (Amazon SNS).

Prerequisites and limitations

Prerequisites 

  • An active AWS account.

  • An existing SNS topic, configured to send email notifications. For more information about this, see Getting started with Amazon SNS in the Amazon SNS documentation.

  • An existing application, configured with CloudWatch Logs.

Limitations 

  • CloudWatch metrics don't support millisecond time intervals. For more information about the granularity of regular and custom metrics, see the Amazon CloudWatch FAQs.

Architecture

The diagram shows the following workflow:

CloudWatch using an Amazon SNS topic to send an email notification when an alarm initiates.

  1. Logs that use metrics created and updated by CloudWatch Logs are streamed to CloudWatch.

  2. An alarm initiates based on thresholds and sends an alert to an SNS topic.

  3. Amazon SNS sends you an email notification.

Technology stack  

  • CloudWatch

  • AWS Lambda

  • Amazon SNS

Tools

Epics

TaskDescriptionSkills required

Option 1 - Create a custom metric with a Lambda function.

Download the lambda_function.py file (attached) and then replace the sample lambda_function.py file in the aws-lambda-developer-guide repository on the AWS Documentation GitHub. This provides you with a sample Lambda function that sends custom metrics to CloudWatch Logs. The Lambda function uses the Boto3 API to integrate with CloudWatch. 

After you run the Lambda function, you can sign in to the AWS Management Console, open the CloudWatch console, and the published metric is available under your published namespace.

DevOps engineer, AWS DevOps

Option 2 – Create custom metrics from CloudWatch log groups.

Sign in to the AWS Management Console, open the CloudWatch console, and then choose Log groups. Choose the log group that you want to create a metric for. 

Choose Actions and then choose Create metric filter. For Filter pattern, enter the filter pattern that you want to use. For more information, see Filter and pattern syntax in the CloudWatch documentation. 

To test your filter pattern, enter one or more log events under Test Pattern. Each log event must be within one line, because line breaks are used to separate log events in the Log event messages box. After you test the pattern, you can enter a name and value for your metric under Metric details. 

For more information and steps to create a custom metric, see Create a metric filter for a log group in the CloudWatch documentation.

DevOps engineer, AWS DevOps

Create an alarm for your custom metric.

On the CloudWatch console, choose Alarms and then choose Create Alarm. Choose Select metric and enter the name of the metric that you created earlier into the search box. Choose the Graphed metrics tab and configure the options according to your requirements.

Under Conditions, choose Anomaly detection instead of Static thresholds. This shows you a band based on two standard default deviations. You can set up thresholds and adjust them according to your requirements.

Choose Next.

Note

The band is dynamic and depends on the quality of the datapoints. When you begin aggregating more data, the band and thresholds are automatically updated. 

DevOps engineer, AWS DevOps

Set up SNS notifications.

Under Notification, choose the SNS topic to notify when the alarm is in ALARM state, OK state, or INSUFFICIENT_DATA state.

To have the alarm send multiple notifications for the same alarm state or for different alarm states, choose Add notification. Choose Next. Enter a name and description for the alarm. The name must only contain ASCII characters. Then choose Next.

Under Preview and create, confirm that the information and conditions are correct, and then choose Create alarm.

DevOps engineer, AWS DevOps

Related resources

Attachments

To access additional content that is associated with this document, unzip the following file: attachment.zip