Analyzing, optimizing, and reducing CloudWatch costs

This section describes how Amazon CloudWatch features generate costs. It also provides methods that can help you analyze, optimize, and reduce CloudWatch costs. Throughout this section, we sometimes refer to pricing when describing CloudWatch features. For information about pricing, see Amazon CloudWatch pricing.

Analyze CloudWatch cost and usage data with Cost Explorer

With AWS Cost Explorer, you can visualize and analyze cost and usage data for AWS services over time, including CloudWatch. For more information, see Getting started with AWS Cost Explorer.

The following procedure describes how to use Cost Explorer to visualize and analyze CloudWatch cost and usage data.

To visualize and analyze CloudWatch cost and usage data

The following image shows an example of the costs that CloudWatch features generated over six months.

To see which CloudWatch features generated the most costs, look at the values for UsageType. For example, EU-CW:GMD-Metrics represents the costs that CloudWatch bulk API requests generated.

The following table includes the names of each CloudWatch feature, lists the names of each subfeature, and lists the strings for UsageType.

Analyze CloudWatch cost and usage data with AWS Cost and Usage Reports and Athena

Another way to analyze CloudWatch cost and usage data is by using AWS Cost and Usage Reports with Amazon Athena. AWS Cost and Usage Reports contain a comprehensive set of cost and usage data. You can create reports that track your costs and usage, and you can publish these reports to an S3 bucket of your choice. You also can download and delete your reports from your S3 bucket. For more information, see What are AWS Cost and Usage Reports? in the AWS Cost and Usage Reports User Guide.

Athena is a query service that you can use with AWS Cost and Usage Reports to analyze cost and usage data. You can query your reports in your S3 bucket without needing to download them first. For more information, see What is Amazon Athena? in the Amazon Athena User Guide. For more information, see What is Amazon Athena? in the Amazon Athena User Guide. For information about pricing, see Amazon Athena pricing.

The following procedure describes the process for enabling AWS Cost and Usage Reports and integrating the service with Athena. The procedure contains two example queries that you can use to analyze CloudWatch cost and usage data.

To analyze cost and usage data with AWS Cost and Usage Reports and Athena

SELECT
CASE
-- Metrics
WHEN line_item_usage_type LIKE '%%MetricMonitorUsage%%' THEN 'Metrics (Custom, Detailed monitoring management portal EMF)'
WHEN line_item_usage_type LIKE '%%Requests%%' THEN 'Metrics (API Requests)'
WHEN line_item_usage_type LIKE '%%GMD-Metrics%%' THEN 'Metrics (Bulk API Requests)'
WHEN line_item_usage_type LIKE '%%MetricStreamUsage%%' THEN 'Metric Streams'
-- Dashboard
WHEN line_item_usage_type LIKE '%%DashboardsUsageHour%%' THEN 'Dashboards'
-- Alarms
WHEN line_item_usage_type LIKE '%%AlarmMonitorUsage%%' THEN 'Alarms (Standard)'
WHEN line_item_usage_type LIKE '%%HighResAlarmMonitorUsage%%' THEN 'Alarms (High Resolution)'
WHEN line_item_usage_type LIKE '%%MetricInsightAlarmUsage%%' THEN 'Alarms (Metrics Insights)'
WHEN line_item_usage_type LIKE '%%CompositeAlarmMonitorUsage%%' THEN 'Alarms (Composite)'
-- Logs
WHEN line_item_usage_type LIKE '%%DataProcessing-Bytes%%' THEN 'Logs (Collect - Data Ingestion)'
WHEN line_item_usage_type LIKE '%%DataProcessingIA-Bytes%%' THEN 'Infrequent Access Logs (Collect - Data Ingestion)'
WHEN line_item_usage_type LIKE '%%DataProtection-Bytes%%' THEN 'Logs (Data Protection - Detect and Mask)'
WHEN line_item_usage_type LIKE '%%TimedStorage-ByteHrs%%' THEN 'Logs (Storage - Archival)'
WHEN line_item_usage_type LIKE '%%DataScanned-Bytes%%' THEN 'Logs (Analyze - Logs Insights queries)'
WHEN line_item_usage_type LIKE '%%Logs-LiveTail%%' THEN 'Logs (Analyze - Logs Live Tail)'
-- Vended Logs
WHEN line_item_usage_type LIKE '%%VendedLog-Bytes%%' THEN 'Vended Logs (Delivered to CW)'
WHEN line_item_usage_type LIKE '%%VendedLogIA-Bytes%%' THEN 'Vended Infrequent Access Logs (Delivered to CW)'
WHEN line_item_usage_type LIKE '%%FH-Egress-Bytes%%' THEN 'Vended Logs (Delivered to Data Firehose)'
WHEN (line_item_usage_type LIKE '%%S3-Egress-Bytes%%') THEN 'Vended Logs (Delivered to S3)'
-- Other
WHEN line_item_usage_type LIKE '%%Application-Signals%%' THEN 'Application Signals'
WHEN line_item_usage_type LIKE '%%Canary-runs%%' THEN 'Synthetics'
WHEN line_item_usage_type LIKE '%%Evidently%%' THEN 'Evidently' 
WHEN line_item_usage_type LIKE '%%RUM-event%%' THEN 'RUM'
ELSE 'Others'
END AS UsageType,
-- REGEXP_EXTRACT(line_item_resource_id,'^(?:.+?:){5}(.+)$',1) as ResourceID,
-- SUM(CAST(line_item_usage_amount AS double)) AS UsageQuantity,
SUM(CAST(line_item_unblended_cost AS decimal(16,8))) AS TotalSpend
FROM
costandusagereport 
WHERE
product_product_name = 'AmazonCloudWatch'
AND year='2022' 
AND month='4' 
AND line_item_line_item_type NOT IN ('Tax','Credit','Refund','EdpDiscount','Fee','RIFee')
-- AND line_item_usage_account_id = '123456789012' – If you want to filter on a specific account, you can
remove this comment at the beginning of the line and specify an AWS account.
GROUP BY
1
ORDER BY
TotalSpend DESC,
UsageType;

SELECT
bill_payer_account_id as Payer,
line_item_usage_account_id as LinkedAccount,
line_item_usage_type AS UsageType,
line_item_operation AS Operation,
line_item_resource_id AS ResourceID,
SUM(CAST(line_item_usage_amount AS double)) AS UsageQuantity,
SUM(CAST(line_item_unblended_cost AS decimal(16,8))) AS TotalSpend
FROM
costandusagereport 
WHERE
product_product_name = 'AmazonCloudWatch'
AND year='2022' 
AND month='4' 
AND line_item_line_item_type NOT IN ('Tax','Credit','Refund','EdpDiscount','Fee','RIFee')
GROUP BY
bill_payer_account_id,
line_item_usage_account_id,
line_item_usage_type,y
line_item_resource_id,
line_item_operation

Optimizing and reducing costs of CloudWatch metrics

Many AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) , Amazon S3, and Amazon Data Firehose, automatically send metrics to CloudWatch at no charge. However, metrics that are grouped in the following categories can incur additional costs:

For more information, see Using Amazon CloudWatch metrics.

Custom metrics

You can create custom metrics to organize data points in any order and at any rate.

All custom metrics are prorated by the hour. They're metered only when they're sent to CloudWatch. For information about how metrics are priced, see Amazon CloudWatch Pricing.

The following table lists the names of relevant subfeatures for CloudWatch metrics. The table includes the strings for UsageType and Operation, which can help you analyze and identify metric-related costs.

Detailed monitoring

CloudWatch has two types of monitoring:

Similar to custom metrics, detailed monitoring is prorated by the hour and metered only when data is sent to CloudWatch. Detailed monitoring generates costs by the number of metrics that are sent to CloudWatch. To reduce costs, only enable detailed monitoring when necessary. For information about how detailed monitoring is priced, see Amazon CloudWatch Pricing.

Example: Athena query

You can use the following query to show which EC2 instances have detailed monitoring enabled.

SELECT
bill_payer_account_id as Payer,
line_item_usage_account_id as LinkedAccount,
line_item_usage_type AS UsageType,
line_item_operation AS Operation,
line_item_resource_id AS ResourceID,
SUM(CAST(line_item_usage_amount AS double)) AS UsageQuantity,
SUM(CAST(line_item_unblended_cost AS decimal(16,8))) AS TotalSpend
FROM
costandusagereport 
WHERE
product_product_name = 'AmazonCloudWatch'
AND year='2022' 
AND month='4' 
AND line_item_operation='MetricStorage:AWS/EC2'
AND line_item_line_item_type NOT IN ('Tax','Credit','Refund','EdpDiscount','Fee','RIFee')
GROUP BY
bill_payer_account_id,
line_item_usage_account_id,
line_item_usage_type,
line_item_resource_id,
line_item_operation,
line_item_line_item_description
ORDER BY line_item_operation

Embedded metrics

With the CloudWatch embedded metric format, you can ingest application data as log data, so that you can generate actionable metrics. For more information, see Ingesting high-cardinality logs and generating metrics with the CloudWatch embedded metric format.

Embedded metrics generate costs by the number of logs ingested, number of logs archived, and number of custom metrics generated.

The following table lists the names of relevant subfeatures for the CloudWatch embedded metric format. The table includes the strings for UsageType and Operation, which can help you analyze and identify costs.

To analyze costs, use AWS Cost and Usage Reports with Athena so that you can identify which metrics are generating costs and determine how the costs are generated.

To make the most of costs generated by the CloudWatch embedded metric format, avoid creating metrics based on high-cardinality dimensions. This way, CloudWatch doesn't create a custom metric for each unique dimension combination. For more information, see Dimensions.

If you're using CloudWatch Container Insights to leverage the embedded metric format, you can use AWS Distro for Open Telemetry as an alternative to make the most of metric-related costs. With Container Insights, you can collect, aggregate, and summarize metrics and logs from your containerized applications and microservices. When you enable Container Insights, the CloudWatch agent sends your logs to CloudWatch, so it can use the logs to generate embedded metrics. However, the CloudWatch agent only sends a fixed number of metrics to CloudWatch, and you're charged for all available metrics, including any that you're not using. With AWS Distro for Open Telemetry, you can configure and customize which metrics and dimensions are sent to CloudWatch. This helps you reduce the volume of data and cost that Container Insights generates. For more information, see the following resources:

API requests

CloudWatch has the following types of API requests:

API requests generate costs by the request type and number of metrics requested.

The following table lists the types of API requests and includes the strings for UsageType and Operation, which can help you analyze and identify API-related costs.

To analyze costs, use Cost Explorer, and group your results by API Operation.

Costs for API requests vary, and you incur costs when you exceed the number of API calls provided to you under the AWS Free Tier limit.

The API requests that typically drive cost are Put and Get requests.

PutMetricData

PutMetricData generates costs every time that it's called and can incur significant costs depending on the use case. For more information, see PutMetricData in the Amazon CloudWatch API Reference.

To make the most of costs that are generated by PutMetricData, batch more data into your API calls. Depending on your use case, consider using CloudWatch Logs or the CloudWatch embedded metric format to inject metric data. For more information, see the following resources:

GetMetricData

GetMetricData can also generate significant costs. Common use cases that drive cost involve third-party monitoring tools that pull data to generate insights. For more information, see GetMetricData in the Amazon CloudWatch API Reference.

To reduce costs generated by GetMetricData, consider only pulling data that's monitored and used, or consider pulling data less often. Depending on your use case, you might consider using metric streams instead of GetMetricData, so that you can push data in near real time to third parties at a lower cost. For more information, see the following resources:

GetMetricStatistics

Depending on your use case, you might consider using GetMetricStatistics instead of GetMetricData. With GetMetricData, you can retrieve data quickly and at scale. However, GetMetricStatistics is included under the AWS Free Tier limit for up to one million API requests, which can help you reduce costs if you don't need to retrieve as many metrics and data points per call. For more information, see the following resources:

CloudWatch metric streams

With CloudWatch metric streams, you can send metrics continuously to AWS destinations and third-party service provider destinations.

Metric streams generate costs by the number of metric updates. Metric updates always include values for the following statistics:

For more information, see Statistics that can be streamed.

To analyze costs that are generated by CloudWatch metric streams, use AWS Cost and Usage Reports with Athena. This way, you can identify which metric streams are generating costs and determine how the costs are generated.

Example: Athena query

You can use the following query to track which metric streams generate costs by Amazon Resource Name (ARN).

SELECT
SPLIT_PART(line_item_resource_id,'/',2) AS "Stream Name",
line_item_resource_id as ARN,
SUM(CAST(line_item_unblended_cost AS decimal(16,2))) AS TotalSpend
FROM
costandusagereport 
WHERE
product_product_name = 'AmazonCloudWatch'
AND year='2022' 
AND month='4' 
AND line_item_line_item_type NOT IN ('Tax','Credit','Refund','EdpDiscount','Fee','RIFee')
-- AND line_item_usage_account_id = '123456789012' – If you want to filter on a specific account, you can
remove this comment at the beginning of the line and specify an AWS account.
AND line_item_usage_type LIKE '%%MetricStreamUsage%%'
GROUP BY line_item_resource_id
ORDER BY TotalSpend DESC

To reduce costs generated by CloudWatch metric streams, stream only the metrics that bring your business value. You also can stop or pause any metric stream that you're not using.

Optimizing and reducing costs of CloudWatch alarms

With CloudWatch alarms, you can create alarms based on a single metric, alarms based on a Metrics Insights query, and composite alarms which watch other alarms.

Metric alarms

Metric alarms have the following resolution settings:

When you create a metric alarm, your costs are based on your alarm’s resolution setting and the number of metrics that your alarm references. For example, a metric alarm that references one metric incurs one alarm-metric cost per hour. For more information, see Using Amazon CloudWatch alarms.

If you create a metric alarm that contains a metric math expression, which references multiple metrics, you incur a cost for each alarm-metric that’s referenced in the metric math expression. For information about how to create a metric alarm that contains a metric math expression, see Creating a CloudWatch alarm based on a metric math expression.

If you create an anomaly detection alarm, where your alarm analyzes past metric data to create a model of expected values, you incur a cost for each alarm-metric that's referenced in your alarm plus two additional metrics, one for the upper and lower band metrics that the anomaly detection model creates. For information about how to create an anomaly detection alarm, see Creating a CloudWatch alarm based on anomaly detection.

Metrics Insights query alarms

Metric Insights query alarms are a specific type of metric alarm, only available with standard resolution (evaluated every 60 seconds).

When you create a Metric Insights query alarm, your costs are based on the number of metrics analyzed by the query that your alarm references. For example, a Metric Insights query alarm that references a query whose filter matches ten metrics incurs ten metrics analyzed cost per hour. For more information, see the pricing example on Amazon CloudWatch Pricing.

If you create an alarm that contains both a Metrics Insights query and a metric math expression, it is reported as a Metrics Insights query alarm. If your alarm contains a metric math expression which references other metrics in addition to the metrics analyzed by the Metrics Insights query, you incur an additional cost for each alarm-metric that’s referenced in the metric math expression. For information about how to create a metric alarm that contains a metric math expression, see Creating a CloudWatch alarm based on a metric math expression.

Composite alarms

Composite alarms contain rule expressions that specify how they should evaluate the states of other alarms to determine their own states. Composite alarms incur a standard cost per hour, regardless of how many other alarms they evaluate. Alarms that composite alarms reference in rule expressions incur separate costs. For more information, see Creating a composite alarm.

Alarm usage types

The following table lists the names of relevant subfeatures for CloudWatch alarms. The table includes the strings for UsageType, which can help you analyze and identify alarm-related costs.

Composite alarms contain rule expressions that specify how they should evaluate the states of other alarms to determine their own states. Composite alarms incur a standard cost per hour, regardless of how many other alarms they evaluate. Alarms that composite alarms reference in rule expressions incur separate costs. For more information, see Creating a composite alarm.

Alarm usage types

The following table lists the names of relevant subfeatures for CloudWatch alarms. The table includes the strings for UsageType, which can help you analyze and identify alarm-related costs.

Reducing alarm costs

To optimize costs generated by metric math alarms that aggregate four or more metrics, you can aggregate data before the data is sent to CloudWatch. This way, you can create an alarm for a single metric instead of an alarm that aggregates data for multiple metrics. For more information, see Publishing custom metrics.

To optimize costs generated by Metrics Insights query alarms, you can ensure that the filter used for the query matches only the metrics that you want to monitor.

The best way to reduce costs is to remove all unnecessary or unused alarms. For example, you can delete alarms that evaluate metrics emitted by AWS resources that no longer exist.

aws cloudwatch describe-alarms –state-value INSUFFICIENT_DATA

Other ways to reduce costs include the following:

Optimizing and reducing costs of CloudWatch Logs

Amazon CloudWatch Logs has the following log types:

For more information about vended logs, see Enabling logging from certain AWS services in the Amazon CloudWatch Logs User Guide.

Custom and vended logs generate costs based on the number of logs that are collected, stored, and analyzed. Separately, vended logs generate costs for delivery to Amazon S3 and Firehose.

The following table lists the names of the CloudWatch Logs features and names of relevant subfeatures. The table includes the strings for UsageType and Operation, which can help you analyze and identify log-related costs.

To analyze costs, use AWS Cost Explorer Service or AWS Cost and Usage Reports with Athena. With either method, you can identify which logs are generating costs and determine how the costs are generated..

Using AWS Cost Explorer Service

Select CloudWatch for the Service filter and select Resource for the Dimension. When you select Resource as the dimension in Cost Explorer Service you can see only the past 14 days of usage.

Using Amazon Athena query to track logs that generate costs

You can use the following query to track which logs generate costs by resource ID.

SELECT
bill_payer_account_id as Payer,
line_item_usage_account_id as LinkedAccount,
line_item_resource_id AS ResourceID,
line_item_usage_type AS UsageType,
SUM(CAST(line_item_unblended_cost AS decimal(16,8))) AS TotalSpend,
SUM(CAST(line_item_usage_amount AS double)) AS UsageQuantity
FROM
costandusagereport 
WHERE
product_product_name = 'AmazonCloudWatch'
AND year='2022'
AND month='4'
AND line_item_operation IN ('PutLogEvents','HourlyStorageMetering','StartQuery','LogDelivery','StartLiveTail','ParquetConversion')
AND line_item_line_item_type NOT IN ('Tax','Credit','Refund','EdpDiscount','Fee','RIFee')
GROUP BY
bill_payer_account_id,
line_item_usage_account_id,
line_item_usage_type,
line_item_resource_id,
line_item_operation
ORDER BY
TotalSpend DESC

To make the most of costs that are generated by CloudWatch Logs, consider the following:

To reduce costs for vended logs, consider your use case, and then determine whether your logs should be sent to CloudWatch or Amazon S3. For more information, see Logs sent to Amazon S3 in the Amazon CloudWatch Logs User Guide.

For additional information about how to make the most of costs that are generated by CloudWatch Logs, see Which log group is causing a sudden increase in my CloudWatch Logs bill?.