# Creating metrics from log events using filters
You can search and filter the log data coming into CloudWatch Logs by creating one or more *metric filters*. Metric filters define the terms and patterns to look for in log data as it is sent to CloudWatch Logs. CloudWatch Logs uses these metric filters to turn log data into numerical CloudWatch metrics that you can graph or set an alarm on.
When you create a metric from a log filter, you can also choose to assign dimensions and a unit to the metric. If you specify a unit, be sure to specify the correct one when you create the filter. Changing the unit for the filter later will have no effect.
If you have configured AWS Organizations and are working with member accounts you can use log centralization to collect log data from source accounts into a central monitoring account.
When working with centralized log groups you can use these system fields dimensions when creating metric filters.
+ `@aws.account` - This dimension represents the AWS account ID from which the log event originated.
+ `@aws.region` - This dimension represents the AWS region where the log event was generated.
These dimensions help in identifying the source of log data, allowing for more granular filtering and analysis of metrics derived from centralized logs. For more information, see [Cross-account cross-Region log centralization](CloudWatchLogs_Centralization.md).
If a log group with a subscription uses log transformation, the filter pattern is applied to the transformed versions of the log events. For more information, see [Transform logs during ingestion](CloudWatch-Logs-Transformation.md).
**Note**
Metric filters are supported only for log groups in the Standard log class. For more information about log classes, see [Log classes](CloudWatch_Logs_Log_Classes.md).
You can use any type of CloudWatch statistic, including percentile statistics, when viewing these metrics or setting alarms.
**Note**
Percentile statistics are supported for a metric only if none of the metric's values are negative. If you set up your metric filter so that it can report negative numbers, percentile statistics will not be available for that metric when it has negative numbers as values. For more information, see [Percentiles](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Percentiles).
Filters do not retroactively filter data. Filters only publish the metric data points for events that happen after the filter was created. When testing a filter pattern, the **Filter results** preview shows up to the first 50 matching log lines for validation purposes. If the timestamp on the filtered results is earlier than the metric creation time, no logs are displayed.
**Topics**
+ [
## Concepts
](#search-filter-concepts)
+ [
# Filter pattern syntax for metric filters
](FilterAndPatternSyntaxForMetricFilters.md)
+ [
# Creating metric filters
](MonitoringPolicyExamples.md)
+ [
# Listing metric filters
](ListingMetricFilters.md)
+ [
# Deleting a metric filter
](DeletingMetricFilter.md)
## Concepts
Each metric filter is made up of the following key elements:
**default value**
The value reported to the metric filter during a period when logs are ingested but no matching logs are found. By setting this to 0, you ensure that data is reported during every such period, preventing "spotty" metrics with periods of no matching data. If no logs are ingested during a one-minute period, then no value is reported.
If you assign dimensions to a metric created by a metric filter, you can't assign a default value for that metric.
**dimensions**
Dimensions are the key-value pairs that further define a metric. You can assign dimensions to the metric created from a metric filter. Because dimensions are part of the unique identifier for a metric, whenever a unique name/value pair is extracted from your logs, you are creating a new variation of that metric.
**filter pattern**
A symbolic description of how CloudWatch Logs should interpret the data in each log event. For example, a log entry may contain timestamps, IP addresses, strings, and so on. You use the pattern to specify what to look for in the log file.
**metric name**
The name of the CloudWatch metric to which the monitored log information should be published. For example, you may publish to a metric called ErrorCount.
**metric namespace**
The destination namespace of the new CloudWatch metric.
**metric value**
The numerical value to publish to the metric each time a matching log is found. For example, if you're counting the occurrences of a particular term like "Error", the value will be "1" for each occurrence. If you're counting the bytes transferred, you can increment by the actual number of bytes found in the log event.
# Filter pattern syntax for metric filters
**Note**
**How metric filters differ from CloudWatch Logs Insights queries**
Metric filters differ from CloudWatch Logs Insights queries in that a specified numerical value is added to a metric filter each time a matching log is found. For more information, see [Configuring metric values for a metric filter](#changing-default-increment-value).
For information about how to query your log groups with the Amazon CloudWatch Logs Insights query language, see [CloudWatch Logs Insights language query syntax](CWL_QuerySyntax.md).
**Generic filter pattern examples**
For more information on generic filter pattern syntax applicable to metric filters as well as [subscription filters](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html) and [filter log events](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SearchDataFilterPattern.html), see [Filter pattern syntax for metric filters, subscription filters, and filter log events](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html), which includes the following examples:
Supported regular expressions (regex) syntax
Matching terms in unstructured log events
Matching terms in JSON log events
Matching terms in space-delimited log events
*Metric filters* allow you to search and filter log data coming into CloudWatch Logs, extract metric observations from the filtered log data, and transform the data points into a CloudWatch Logs metric. You define the terms and patterns to look for in log data as it is sent to CloudWatch Logs. Metric filters are assigned to log groups, and all of the filters assigned to a log group are applied to their log streams.
When a metric filter matches a term, it increments the metric's count by a specified numerical value. For example, you can create a metric filter that counts the number of times the word ***ERROR*** occurs in your log events.
You can assign units of measure and dimensions to metrics. For example, if you create a metric filter that counts the number of times the word ***ERROR*** occurs in your log events, you can specify a dimension that's called `ErrorCode` to show the total number of log events that contain the word ***ERROR*** and filter data by reported error codes.
**Tip**
When you assign a unit of measure to a metric, make sure to specify the correct one. If you change the unit later, your change might not take effect. For the complete list of the units that CloudWatch supports, see [MetricDatum](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_MetricDatum.html) in the Amazon CloudWatch API Reference.
**Topics**
+ [
## Configuring metric values for a metric filter
](#changing-default-increment-value)
+ [
## Publishing dimensions with metrics from values in JSON or space-delimited log events
](#logs-metric-filters-dimensions)
+ [
## Using values in log events to increment a metric's value
](#publishing-values-found-in-logs)
## Configuring metric values for a metric filter
When you create a metric filter, you define your filter pattern and specify your metric's value and default value. You can set metric values to numbers, named identifiers, or numeric identifiers. If you don't specify a default value, CloudWatch won't report data when your metric filter doesn't find a match. We recommend that you specify a default value, even if the value is 0. Setting a default value helps CloudWatch report data more accurately and prevents CloudWatch from aggregating spotty metrics. CloudWatch aggregates and reports metric values every minute.
When your metric filter finds a match in your log events, it increments your metric's count by your metric's value. If your metric filter doesn't find a match, CloudWatch reports the metric's default value. For example, your log group publishes two records every minute, the metric value is 1, and the default value is 0. If your metric filter finds matches in both log records within the first minute, the metric value for that minute is 2. If your metric filter doesn't find matches in either records during the second minute, the default value for that minute is 0. If you assign dimensions to metrics that metric filters generate, you can't specify default values for those metrics.
You also can set up a metric filter to increment a metric with a value extracted from a log event, instead of a static value. For more information, see [Using values in log events to increment a metric's value](#publishing-values-found-in-logs).
## Publishing dimensions with metrics from values in JSON or space-delimited log events
You can use the CloudWatch console or AWS CLI to create metric filters that publish dimensions with metrics that JSON and space-delimited log events generate. Dimensions are name/value value pairs and only available for JSON and space-delimited filter patterns. You can create JSON and space-delimited metric filters with up to three dimensions. For more information about dimensions and information about how to assign dimensions to metrics, see the following sections:
+ [Dimensions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Dimension) in the *Amazon CloudWatch User guide*
+ [Example: Extract fields from an Apache log and assign dimensions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/ExtractBytesExample.html) in the *Amazon CloudWatch Logs User Guide*
**Important**
Dimensions contain values that gather charges the same as custom metrics. To prevent unexpected charges, don't specify high-cardinality fields, such as `IPAddress` or `requestID`, as dimensions.
If you extract metrics from log events, you're charged for custom metrics. To prevent you from collecting accidental high charges, Amazon might disable your metric filter if it generates 1000 different name/value pairs for specified dimensions over a certain amount of time.
You can create billing alarms that notify you of your estimated charges. For more information, see [ Creating a billing alarm to monitor your estimated AWS charges](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/monitor_estimated_charges_with_cloudwatch.html).
### Publishing dimensions with metrics from JSON log events
The following examples contain code snippets that describe how to specify dimensions in a JSON metric filter.
------
#### [ Example: JSON log event ]
```
{
"eventType": "UpdateTrail",
"sourceIPAddress": "111.111.111.111",
"arrayKey": [
"value",
"another value"
],
"objectList": [
{"name": "a",
"id": 1
},
{"name": "b",
"id": 2
}
]
}
```
**Note**
If you test the example metric filter with the example JSON log event, you must enter the example JSON log on a single line.
------
#### [ Example: Metric filter ]
The metric filter increments the metric whenever a JSON log event contain the properties `eventType` and `"sourceIPAddress"`.
```
{ $.eventType = "*" && $.sourceIPAddress != 123.123.* }
```
When you create a JSON metric filter, you can specify any of the properties in the metric filter as a dimension. For example, to set `eventType` as a dimension, use the following:
```
"eventType" : $.eventType
```
The example metric contains a dimension that's named `"eventType"`, and the dimension's value in the example log event is `"UpdateTrail"`.
------
### Publishing dimensions with metrics from space-delimited log events
The following examples contain code snippets that describe how to specify dimensions in a space-delimited metric filter.
------
#### [ Example: Space-delimited log event ]
```
127.0.0.1 Prod frank [10/Oct/2000:13:25:15 -0700] "GET /index.html HTTP/1.0" 404 1534
```
------
#### [ Example: Metric filter ]
```
[ip, server, username, timestamp, request, status_code, bytes > 1000]
```
The metric filter increments the metric when a space-delimited log event includes any of the fields that are specified in the filter. For example, the metric filter finds following fields and values in the example space-delimited log event.
```
{
"$bytes": "1534",
"$status_code": "404",
"$request": "GET /index.html HTTP/1.0",
"$timestamp": "10/Oct/2000:13:25:15 -0700",
"$username": "frank",
"$server": "Prod",
"$ip": "127.0.0.1"
}
```
When you create a space-delimited metric filter, you can specify any of the fields in the metric filter as a dimension. For example, to set `server` as a dimension, use the following:
```
"server" : $server
```
The example metric filter has a dimension that's named `server`, and the dimension's value in the example log event is `"Prod"`.
------
#### [ Example: Match terms with AND (&&) and OR (\$1\$1) ]
You can use the logical operators AND ("&&") and OR ("\$1\$1") to create space-delimited metric filters that contain conditions. The following metric filter returns log events where the first word in the events is ERROR or any superstring of WARN.
```
[w1=ERROR || w1=%WARN%, w2]
```
------
## Using values in log events to increment a metric's value
You can create metric filters that publish numeric values found in your log events. The procedure in this section uses the following example metric filter to show how you can publish a numeric value in a JSON log event to a metric.
```
{ $.latency = * } metricValue: $.latency
```
**To create a metric filter that publishes a value in a log event**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Logs**, and then choose **Log groups**.
1. Select or create a log group.
For information about how to create a log group, see [Create a log group in CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Working-with-log-groups-and-streams.html) in the *Amazon CloudWatch Logs User Guide*.
1. Choose **Actions**, and then choose **Create metric filter**.
1. For **Filter Pattern**, enter **\$1 \$1.latency = \$1 \$1**, and then choose **Next**.
1. For **Metric Name**, enter **myMetric**.
1. For **Metric Value**, enter **\$1.latency**.
1. (Optional) For **Default Value**, enter **0**, and then choose **Next**.
We recommend that you specify a default value, even if the value is 0. Setting a default value helps CloudWatch report data more accurately and prevents CloudWatch from aggregating spotty metrics. CloudWatch aggregates and reports metric values every minute.
1. Choose **Create metric filter**.
The example metric filter matches the term `"latency"` in the example JSON log event and publishes a numeric value of 50 to the metric **myMetric**.
```
{
"latency": 50,
"requestType": "GET"
}
```
# Creating metric filters
The following procedure and examples show how to create metric filters.
**Topics**
+ [
# Create a metric filter for a log group
](CreateMetricFilterProcedure.md)
+ [
# Example: Count log events
](CountingLogEventsExample.md)
+ [
# Example: Count occurrences of a term
](CountOccurrencesExample.md)
+ [
# Example: Count HTTP 404 codes
](Counting404Responses.md)
+ [
# Example: Count HTTP 4xx codes
](FindCountMetric.md)
+ [
# Example: Extract fields from an Apache log and assign dimensions
](ExtractBytesExample.md)
# Create a metric filter for a log group
To create a metric filter for a log group, follow these steps. The metric won't be visible until there are some data points for it.
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Logs**, and then choose **Log groups**.
1. Choose the name of the log group.
1. Choose `Actions`, and then choose **Create metric filter**.
1. For **Filter pattern**, enter a filter pattern. For more information, see [Filter pattern syntax for metric filters, subscription filters, filter log events, and Live Tail](FilterAndPatternSyntax.md).
1. (Optional) If you are using centralized log groups, under **Filter selection criteria**, you can specify filters based on source account (`@aws.account`), source region (`@aws.region`), or both conditions.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to test the pattern. Each log event must be formatted on one line. Line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then enter a name for your metric filter.
1. Under **Metric details**, for **Metric namespace**, enter a name for the CloudWatch namespace where the metric will be published. If the namespace doesn't already exist, make sure that **Create new** is selected.
1. For **Metric name**, enter a name for the new metric.
1. For **Metric value**, if your metric filter is counting occurrences of the keywords in the filter, enter 1. This increments the metric by 1 for each log event that includes one of the keywords.
Alternatively, enter a token, such as **\$1size**. This increments the metric by the value of the number in the `size` field for every log event that contains a `size` field.
1. (Optional) For **Unit**, select a unit to assign to the metric. If you do not specify a unit, the unit is set as `None`.
1. (Optional) Enter the names and tokens for as many as three dimensions for the metric. If you assign dimensions to metrics that metric filters create, you cannot assign default values for those metrics.
**Note**
Dimensions are supported only in JSON or space-delimited metric filters.
1. Choose **Create metric filter**. You can find the metric filter that you created from the navigation pane. Choose **Logs**, and then choose **Log groups**. Choose the name of the log group that you created your metric filter for, and then select the **Metric filters** tab.
# Example: Count log events
The simplest type of log event monitoring is to count the number of log events that occur. You might want to do this to keep a count of all events, to create a "heartbeat" style monitor or just to practice creating metric filters.
In the following CLI example, a metric filter called MyAppAccessCount is applied to the log group MyApp/access.log to create the metric EventCount in the CloudWatch namespace MyNamespace. The filter is configured to match any log event content and to increment the metric by "1".
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of a log group.
1. Choose `Actions`, **Create metric filter**.
1. Leave **Filter Pattern** and **Select Log Data to Test** blank.
1. Choose **Next**, and then for **Filter Name**, type **EventCount**.
1. Under **Metric Details**, for **Metric Namespace**, type **MyNameSpace**.
1. For **Metric Name**, type **MyAppEventCount**.
1. Confirm that **Metric Value** is 1. This specifies that the count is incremented by 1 for every log event.
1. For **Default Value** enter 0, and then choose **Next**. Specifying a default value ensures that data is reported even during periods when no log events occur, preventing spotty metrics where data sometimes does not exist.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name EventCount \
--filter-pattern " " \
--metric-transformations \
metricName=MyAppEventCount,metricNamespace=MyNamespace,metricValue=1,defaultValue=0
```
You can test this new policy by posting any event data. You should see data points published to the metric MyAppAccessEventCount.
**To post event data using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-log-events \
--log-group-name MyApp/access.log --log-stream-name TestStream1 \
--log-events \
timestamp=1394793518000,message="Test event 1" \
timestamp=1394793518000,message="Test event 2" \
timestamp=1394793528000,message="This message also contains an Error"
```
# Example: Count occurrences of a term
Log events frequently include important messages that you want to count, maybe about the success or failure of operations. For example, an error may occur and be recorded to a log file if a given operation fails. You may want to monitor these entries to understand the trend of your errors.
In the example below, a metric filter is created to monitor for the term Error. The policy has been created and added to the log group **MyApp/message.log**. CloudWatch Logs publishes a data point to the CloudWatch custom metric ErrorCount in the **MyApp/message.log** namespace with a value of "1" for every event containing Error. If no event contains the word Error, then a value of 0 is published. When graphing this data in the CloudWatch console, be sure to use the sum statistic.
After you create a metric filter, you can view the metric in the CloudWatch console. When you are selecting the metric to view, select the metric namespace that matches the log group name. For more information, see [ Viewing Available Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/viewing_metrics_with_cloudwatch.html).
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of the log group.
1. Choose **Actions**, **Create metric filter**.
1. For **Filter Pattern**, enter **Error**.
**Note**
All entries in **Filter Pattern** are case-sensitive.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then on the **Assign metric** page, for **Filter Name**, type **MyAppErrorCount**.
1. Under **Metric Details**, for **Metric Namespace**, type **MyNameSpace**.
1. For **Metric Name**, type **ErrorCount**.
1. Confirm that **Metric Value** is 1. This specifies that the count is incremented by 1 for every log event containing "Error".
1. For **Default Value** type 0, and then choose **Next**.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/message.log \
--filter-name MyAppErrorCount \
--filter-pattern 'Error' \
--metric-transformations \
metricName=ErrorCount,metricNamespace=MyNamespace,metricValue=1,defaultValue=0
```
You can test this new policy by posting events containing the word "Error" in the message.
**To post events using the AWS CLI**
At a command prompt, run the following command. Note that patterns are case-sensitive.
```
aws logs put-log-events \
--log-group-name MyApp/access.log --log-stream-name TestStream1 \
--log-events \
timestamp=1394793518000,message="This message contains an Error" \
timestamp=1394793528000,message="This message also contains an Error"
```
# Example: Count HTTP 404 codes
Using CloudWatch Logs, you can monitor how many times your Apache servers return a HTTP 404 response, which is the response code for page not found. You might want to monitor this to understand how often your site visitors do not find the resource they are looking for. Assume that your log records are structured to include the following information for each log event (site visit):
+ Requestor IP Address
+ RFC 1413 Identity
+ Username
+ Timestamp
+ Request method with requested resource and protocol
+ HTTP response code to request
+ Bytes transferred in request
An example of this might look like the following:
```
127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 404 2326
```
You could specify a rule which attempts to match events of that structure for HTTP 404 errors, as shown in the following example:
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose `Actions`, **Create metric filter**.
1. For **Filter Pattern**, type **[IP, UserInfo, User, Timestamp, RequestInfo, StatusCode=404, Bytes]**.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then for **Filter Name**, type **HTTP404Errors**.
1. Under **Metric Details**, for **Metric Namespace**, enter **MyNameSpace**.
1. For **Metric Name**, enter **ApacheNotFoundErrorCount**.
1. Confirm that **Metric Value** is 1. This specifies that the count is incremented by 1 for every 404 Error event.
1. For **Default Value** enter 0, and then choose **Next**.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name HTTP404Errors \
--filter-pattern '[ip, id, user, timestamp, request, status_code=404, size]' \
--metric-transformations \
metricName=ApacheNotFoundErrorCount,metricNamespace=MyNamespace,metricValue=1
```
In this example, literal characters such as the left and right square brackets, double quotes and character string 404 were used. The pattern needs to match with the entire log event message for the log event to be considered for monitoring.
You can verify the creation of the metric filter by using the **describe-metric-filters** command. You should see output that looks like this:
```
aws logs describe-metric-filters --log-group-name MyApp/access.log
{
"metricFilters": [
{
"filterName": "HTTP404Errors",
"metricTransformations": [
{
"metricValue": "1",
"metricNamespace": "MyNamespace",
"metricName": "ApacheNotFoundErrorCount"
}
],
"creationTime": 1399277571078,
"filterPattern": "[ip, id, user, timestamp, request, status_code=404, size]"
}
]
}
```
Now you can post a few events manually:
```
aws logs put-log-events \
--log-group-name MyApp/access.log --log-stream-name hostname \
--log-events \
timestamp=1394793518000,message="127.0.0.1 - bob [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 404 2326" \
timestamp=1394793528000,message="127.0.0.1 - bob [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb2.gif HTTP/1.0\" 200 2326"
```
Soon after putting these sample log events, you can retrieve the metric named in the CloudWatch console as ApacheNotFoundErrorCount.
# Example: Count HTTP 4xx codes
As in the previous example, you might want to monitor your web service access logs and monitor the HTTP response code levels. For example, you might want to monitor all of the HTTP 400-level errors. However, you might not want to specify a new metric filter for every return code.
The following example demonstrates how to create a metric that includes all 400-level HTTP code responses from an access log using the Apache access log format from the [Example: Count HTTP 404 codes](Counting404Responses.md) example.
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of the log group for the Apache server.
1. Choose `Actions`, **Create metric filter**.
1. For **Filter pattern**, enter **[ip, id, user, timestamp, request, status\$1code=4\$1, size]**.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then for **Filter name**, type **HTTP4xxErrors**.
1. Under **Metric details**, for **Metric namespace**, enter **MyNameSpace**.
1. For **Metric name**, enter **HTTP4xxErrors**.
1. For **Metric value**, enter 1. This specifies that the count is incremented by 1 for every log containing a 4xx error.
1. For **Default value** enter 0, and then choose **Next**.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name HTTP4xxErrors \
--filter-pattern '[ip, id, user, timestamp, request, status_code=4*, size]' \
--metric-transformations \
metricName=HTTP4xxErrors,metricNamespace=MyNamespace,metricValue=1,defaultValue=0
```
You can use the following data in put-event calls to test this rule. If you did not remove the monitoring rule in the previous example, you will generate two different metrics.
```
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /~test/ HTTP/1.1" 200 3
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:51:34 -0700] "GET /~test/index.html HTTP/1.1" 200 3
```
# Example: Extract fields from an Apache log and assign dimensions
Sometimes, instead of counting, it is helpful to use values within individual log events for metric values. This example shows how you can create an extraction rule to create a metric that measures the bytes transferred by an Apache webserver.
This example also shows how to assign dimensions to the metric that you are creating.
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of the log group for the Apache server.
1. Choose `Actions`, **Create metric filter**.
1. For **Filter pattern**, enter **[ip, id, user, timestamp, request, status\$1code, size]**.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then for **Filter name**, type **size**.
1. Under **Metric details**, for **Metric namespace**, enter **MyNameSpace**. Because this is a new namespace, be sure that **Create new** is selected.
1. For **Metric name**, enter **BytesTransferred**
1. For **Metric value**, enter **\$1size**.
1. For **Unit**, select **Bytes**.
1. For **Dimension Name**, type **IP**.
1. For **Dimension Value**, type **\$1ip** and then choose **Next**.
1. Choose **Create metric filter**.
**To create this metric filter using the AWS CLI**
At a command prompt, run the following command
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name BytesTransferred \
--filter-pattern '[ip, id, user, timestamp, request, status_code, size]' \
--metric-transformations \
metricName=BytesTransferred,metricNamespace=MyNamespace,metricValue='$size'
```
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name BytesTransferred \
--filter-pattern '[ip, id, user, timestamp, request, status_code, size]' \
--metric-transformations \
metricName=BytesTransferred,metricNamespace=MyNamespace,metricValue='$size',unit=Bytes,dimensions='{{IP=$ip}}'
```
**Note**
In this command, use this format to specify multiple dimensions.
```
aws logs put-metric-filter \
--log-group-name my-log-group-name \
--filter-name my-filter-name \
--filter-pattern 'my-filter-pattern' \
--metric-transformations \
metricName=my-metric-name,metricNamespace=my-metric-namespace,metricValue=my-token,unit=unit,dimensions='{dimension1=$dim,dimension2=$dim2,dim3=$dim3}'
```
You can use the following data in put-log-event calls to test this rule. This generates two different metrics if you did not remove monitoring rule in the previous example.
```
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /~test/ HTTP/1.1" 200 3
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:51:34 -0700] "GET /~test/index.html HTTP/1.1" 200 3
```
# Listing metric filters
You can list all metric filters in a log group.
**To list metric filters using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. In the contents pane, in the list of log groups, in the **Metric Filters** column, choose the number of filters.
The **Log Groups > Filters for** screen lists all metric filters associated with the log group.
**To list metric filters using the AWS CLI**
At a command prompt, run the following command:
```
aws logs describe-metric-filters --log-group-name MyApp/access.log
```
The following is example output:
```
{
"metricFilters": [
{
"filterName": "HTTP404Errors",
"metricTransformations": [
{
"metricValue": "1",
"metricNamespace": "MyNamespace",
"metricName": "ApacheNotFoundErrorCount"
}
],
"creationTime": 1399277571078,
"filterPattern": "[ip, id, user, timestamp, request, status_code=404, size]"
}
]
}
```
# Deleting a metric filter
A policy is identified by its name and the log group it belongs to.
**To delete a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. In the contents pane, in the **Metric Filter** column, choose the number of metric filters for the log group.
1. Under **Metric Filters** screen, select the check box to the right of the name of the filter that you want to delete. Then choose **Delete**.
1. When prompted for confirmation, choose **Delete**.
**To delete a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs delete-metric-filter --log-group-name MyApp/access.log \
--filter-name MyFilterName
```