# Creating metric filters
The following procedure and examples show how to create metric filters.
**Topics**
+ [
# Create a metric filter for a log group
](CreateMetricFilterProcedure.md)
+ [
# Example: Count log events
](CountingLogEventsExample.md)
+ [
# Example: Count occurrences of a term
](CountOccurrencesExample.md)
+ [
# Example: Count HTTP 404 codes
](Counting404Responses.md)
+ [
# Example: Count HTTP 4xx codes
](FindCountMetric.md)
+ [
# Example: Extract fields from an Apache log and assign dimensions
](ExtractBytesExample.md)
# Create a metric filter for a log group
To create a metric filter for a log group, follow these steps. The metric won't be visible until there are some data points for it.
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Logs**, and then choose **Log groups**.
1. Choose the name of the log group.
1. Choose `Actions`, and then choose **Create metric filter**.
1. For **Filter pattern**, enter a filter pattern. For more information, see [Filter pattern syntax for metric filters, subscription filters, filter log events, and Live Tail](FilterAndPatternSyntax.md).
1. (Optional) If you are using centralized log groups, under **Filter selection criteria**, you can specify filters based on source account (`@aws.account`), source region (`@aws.region`), or both conditions.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to test the pattern. Each log event must be formatted on one line. Line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then enter a name for your metric filter.
1. Under **Metric details**, for **Metric namespace**, enter a name for the CloudWatch namespace where the metric will be published. If the namespace doesn't already exist, make sure that **Create new** is selected.
1. For **Metric name**, enter a name for the new metric.
1. For **Metric value**, if your metric filter is counting occurrences of the keywords in the filter, enter 1. This increments the metric by 1 for each log event that includes one of the keywords.
Alternatively, enter a token, such as **\$1size**. This increments the metric by the value of the number in the `size` field for every log event that contains a `size` field.
1. (Optional) For **Unit**, select a unit to assign to the metric. If you do not specify a unit, the unit is set as `None`.
1. (Optional) Enter the names and tokens for as many as three dimensions for the metric. If you assign dimensions to metrics that metric filters create, you cannot assign default values for those metrics.
**Note**
Dimensions are supported only in JSON or space-delimited metric filters.
1. Choose **Create metric filter**. You can find the metric filter that you created from the navigation pane. Choose **Logs**, and then choose **Log groups**. Choose the name of the log group that you created your metric filter for, and then select the **Metric filters** tab.
# Example: Count log events
The simplest type of log event monitoring is to count the number of log events that occur. You might want to do this to keep a count of all events, to create a "heartbeat" style monitor or just to practice creating metric filters.
In the following CLI example, a metric filter called MyAppAccessCount is applied to the log group MyApp/access.log to create the metric EventCount in the CloudWatch namespace MyNamespace. The filter is configured to match any log event content and to increment the metric by "1".
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of a log group.
1. Choose `Actions`, **Create metric filter**.
1. Leave **Filter Pattern** and **Select Log Data to Test** blank.
1. Choose **Next**, and then for **Filter Name**, type **EventCount**.
1. Under **Metric Details**, for **Metric Namespace**, type **MyNameSpace**.
1. For **Metric Name**, type **MyAppEventCount**.
1. Confirm that **Metric Value** is 1. This specifies that the count is incremented by 1 for every log event.
1. For **Default Value** enter 0, and then choose **Next**. Specifying a default value ensures that data is reported even during periods when no log events occur, preventing spotty metrics where data sometimes does not exist.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name EventCount \
--filter-pattern " " \
--metric-transformations \
metricName=MyAppEventCount,metricNamespace=MyNamespace,metricValue=1,defaultValue=0
```
You can test this new policy by posting any event data. You should see data points published to the metric MyAppAccessEventCount.
**To post event data using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-log-events \
--log-group-name MyApp/access.log --log-stream-name TestStream1 \
--log-events \
timestamp=1394793518000,message="Test event 1" \
timestamp=1394793518000,message="Test event 2" \
timestamp=1394793528000,message="This message also contains an Error"
```
# Example: Count occurrences of a term
Log events frequently include important messages that you want to count, maybe about the success or failure of operations. For example, an error may occur and be recorded to a log file if a given operation fails. You may want to monitor these entries to understand the trend of your errors.
In the example below, a metric filter is created to monitor for the term Error. The policy has been created and added to the log group **MyApp/message.log**. CloudWatch Logs publishes a data point to the CloudWatch custom metric ErrorCount in the **MyApp/message.log** namespace with a value of "1" for every event containing Error. If no event contains the word Error, then a value of 0 is published. When graphing this data in the CloudWatch console, be sure to use the sum statistic.
After you create a metric filter, you can view the metric in the CloudWatch console. When you are selecting the metric to view, select the metric namespace that matches the log group name. For more information, see [ Viewing Available Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/viewing_metrics_with_cloudwatch.html).
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of the log group.
1. Choose **Actions**, **Create metric filter**.
1. For **Filter Pattern**, enter **Error**.
**Note**
All entries in **Filter Pattern** are case-sensitive.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then on the **Assign metric** page, for **Filter Name**, type **MyAppErrorCount**.
1. Under **Metric Details**, for **Metric Namespace**, type **MyNameSpace**.
1. For **Metric Name**, type **ErrorCount**.
1. Confirm that **Metric Value** is 1. This specifies that the count is incremented by 1 for every log event containing "Error".
1. For **Default Value** type 0, and then choose **Next**.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/message.log \
--filter-name MyAppErrorCount \
--filter-pattern 'Error' \
--metric-transformations \
metricName=ErrorCount,metricNamespace=MyNamespace,metricValue=1,defaultValue=0
```
You can test this new policy by posting events containing the word "Error" in the message.
**To post events using the AWS CLI**
At a command prompt, run the following command. Note that patterns are case-sensitive.
```
aws logs put-log-events \
--log-group-name MyApp/access.log --log-stream-name TestStream1 \
--log-events \
timestamp=1394793518000,message="This message contains an Error" \
timestamp=1394793528000,message="This message also contains an Error"
```
# Example: Count HTTP 404 codes
Using CloudWatch Logs, you can monitor how many times your Apache servers return a HTTP 404 response, which is the response code for page not found. You might want to monitor this to understand how often your site visitors do not find the resource they are looking for. Assume that your log records are structured to include the following information for each log event (site visit):
+ Requestor IP Address
+ RFC 1413 Identity
+ Username
+ Timestamp
+ Request method with requested resource and protocol
+ HTTP response code to request
+ Bytes transferred in request
An example of this might look like the following:
```
127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 404 2326
```
You could specify a rule which attempts to match events of that structure for HTTP 404 errors, as shown in the following example:
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose `Actions`, **Create metric filter**.
1. For **Filter Pattern**, type **[IP, UserInfo, User, Timestamp, RequestInfo, StatusCode=404, Bytes]**.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then for **Filter Name**, type **HTTP404Errors**.
1. Under **Metric Details**, for **Metric Namespace**, enter **MyNameSpace**.
1. For **Metric Name**, enter **ApacheNotFoundErrorCount**.
1. Confirm that **Metric Value** is 1. This specifies that the count is incremented by 1 for every 404 Error event.
1. For **Default Value** enter 0, and then choose **Next**.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name HTTP404Errors \
--filter-pattern '[ip, id, user, timestamp, request, status_code=404, size]' \
--metric-transformations \
metricName=ApacheNotFoundErrorCount,metricNamespace=MyNamespace,metricValue=1
```
In this example, literal characters such as the left and right square brackets, double quotes and character string 404 were used. The pattern needs to match with the entire log event message for the log event to be considered for monitoring.
You can verify the creation of the metric filter by using the **describe-metric-filters** command. You should see output that looks like this:
```
aws logs describe-metric-filters --log-group-name MyApp/access.log
{
"metricFilters": [
{
"filterName": "HTTP404Errors",
"metricTransformations": [
{
"metricValue": "1",
"metricNamespace": "MyNamespace",
"metricName": "ApacheNotFoundErrorCount"
}
],
"creationTime": 1399277571078,
"filterPattern": "[ip, id, user, timestamp, request, status_code=404, size]"
}
]
}
```
Now you can post a few events manually:
```
aws logs put-log-events \
--log-group-name MyApp/access.log --log-stream-name hostname \
--log-events \
timestamp=1394793518000,message="127.0.0.1 - bob [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 404 2326" \
timestamp=1394793528000,message="127.0.0.1 - bob [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb2.gif HTTP/1.0\" 200 2326"
```
Soon after putting these sample log events, you can retrieve the metric named in the CloudWatch console as ApacheNotFoundErrorCount.
# Example: Count HTTP 4xx codes
As in the previous example, you might want to monitor your web service access logs and monitor the HTTP response code levels. For example, you might want to monitor all of the HTTP 400-level errors. However, you might not want to specify a new metric filter for every return code.
The following example demonstrates how to create a metric that includes all 400-level HTTP code responses from an access log using the Apache access log format from the [Example: Count HTTP 404 codes](Counting404Responses.md) example.
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of the log group for the Apache server.
1. Choose `Actions`, **Create metric filter**.
1. For **Filter pattern**, enter **[ip, id, user, timestamp, request, status\$1code=4\$1, size]**.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then for **Filter name**, type **HTTP4xxErrors**.
1. Under **Metric details**, for **Metric namespace**, enter **MyNameSpace**.
1. For **Metric name**, enter **HTTP4xxErrors**.
1. For **Metric value**, enter 1. This specifies that the count is incremented by 1 for every log containing a 4xx error.
1. For **Default value** enter 0, and then choose **Next**.
1. Choose **Create metric filter**.
**To create a metric filter using the AWS CLI**
At a command prompt, run the following command:
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name HTTP4xxErrors \
--filter-pattern '[ip, id, user, timestamp, request, status_code=4*, size]' \
--metric-transformations \
metricName=HTTP4xxErrors,metricNamespace=MyNamespace,metricValue=1,defaultValue=0
```
You can use the following data in put-event calls to test this rule. If you did not remove the monitoring rule in the previous example, you will generate two different metrics.
```
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /~test/ HTTP/1.1" 200 3
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:51:34 -0700] "GET /~test/index.html HTTP/1.1" 200 3
```
# Example: Extract fields from an Apache log and assign dimensions
Sometimes, instead of counting, it is helpful to use values within individual log events for metric values. This example shows how you can create an extraction rule to create a metric that measures the bytes transferred by an Apache webserver.
This example also shows how to assign dimensions to the metric that you are creating.
**To create a metric filter using the CloudWatch console**
1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Log groups**.
1. Choose the name of the log group for the Apache server.
1. Choose `Actions`, **Create metric filter**.
1. For **Filter pattern**, enter **[ip, id, user, timestamp, request, status\$1code, size]**.
1. (Optional) To test your filter pattern, under **Test Pattern**, enter one or more log events to use to test the pattern. Each log event must be within one line, because line breaks are used to separate log events in the **Log event messages** box.
1. Choose **Next**, and then for **Filter name**, type **size**.
1. Under **Metric details**, for **Metric namespace**, enter **MyNameSpace**. Because this is a new namespace, be sure that **Create new** is selected.
1. For **Metric name**, enter **BytesTransferred**
1. For **Metric value**, enter **\$1size**.
1. For **Unit**, select **Bytes**.
1. For **Dimension Name**, type **IP**.
1. For **Dimension Value**, type **\$1ip** and then choose **Next**.
1. Choose **Create metric filter**.
**To create this metric filter using the AWS CLI**
At a command prompt, run the following command
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name BytesTransferred \
--filter-pattern '[ip, id, user, timestamp, request, status_code, size]' \
--metric-transformations \
metricName=BytesTransferred,metricNamespace=MyNamespace,metricValue='$size'
```
```
aws logs put-metric-filter \
--log-group-name MyApp/access.log \
--filter-name BytesTransferred \
--filter-pattern '[ip, id, user, timestamp, request, status_code, size]' \
--metric-transformations \
metricName=BytesTransferred,metricNamespace=MyNamespace,metricValue='$size',unit=Bytes,dimensions='{{IP=$ip}}'
```
**Note**
In this command, use this format to specify multiple dimensions.
```
aws logs put-metric-filter \
--log-group-name my-log-group-name \
--filter-name my-filter-name \
--filter-pattern 'my-filter-pattern' \
--metric-transformations \
metricName=my-metric-name,metricNamespace=my-metric-namespace,metricValue=my-token,unit=unit,dimensions='{dimension1=$dim,dimension2=$dim2,dim3=$dim3}'
```
You can use the following data in put-log-event calls to test this rule. This generates two different metrics if you did not remove monitoring rule in the previous example.
```
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:49:52 -0700] "GET /index.html HTTP/1.1" 404 287
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /~test/ HTTP/1.1" 200 3
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:50:51 -0700] "GET /favicon.ico HTTP/1.1" 404 308
127.0.0.1 - - [24/Sep/2013:11:51:34 -0700] "GET /~test/index.html HTTP/1.1" 200 3
```