Control which events Lambda sends to your function

You can use event filtering to control which records from a stream or queue Lambda sends to your function. For example, you can add a filter so that your function only processes Amazon SQS messages containing certain data parameters. Event filtering works only with certain event source mappings. You can add filters to event source mappings for the following AWS services:

For specific information about filtering with specific event sources, see Using filters with different AWS services. Lambda doesn't support event filtering for Amazon DocumentDB.

By default, you can define up to five different filters for a single event source mapping. Your filters are logically ORed together. If a record from your event source satisfies one or more of your filters, Lambda includes the record in the next event it sends to your function. If none of your filters are satisfied, Lambda discards the record.

Understanding event filtering basics

A filter criteria (FilterCriteria) object is a structure that consists of a list of filters (Filters). Each filter is a structure that defines an event filtering pattern (Pattern). A pattern is a string representation of a JSON filter rule. The structure of a FilterCriteria object is as follows.

{
   "Filters": [
        {
            "Pattern": "{ \"Metadata1\": [ rule1 ], \"data\": { \"Data1\": [ rule2 ] }}"
        }
    ]
}

For added clarity, here is the value of the filter's Pattern expanded in plain JSON.

{
    "Metadata1": [ rule1 ],
    "data": {
        "Data1": [ rule2 ]
    }
}

Your filter pattern can include metadata properties, data properties, or both. The available metadata parameters and the format of the data parameters vary according to the AWS service which is acting as the event source. For example, suppose your event source mapping receives the following record from an Amazon SQS queue:

{
    "messageId": "059f36b4-87a3-44ab-83d2-661975830a7d",
    "receiptHandle": "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...",
    "body": "{\n "City": "Seattle",\n "State": "WA",\n "Temperature": "46"\n}",
    "attributes": {
        "ApproximateReceiveCount": "1",
        "SentTimestamp": "1545082649183",
        "SenderId": "AIDAIENQZJOLO23YVJ4VO",
        "ApproximateFirstReceiveTimestamp": "1545082649185"
    },
    "messageAttributes": {},
    "md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
    "eventSource": "aws:sqs",
    "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
    "awsRegion": "us-east-2"
}

Different types of event source use different key values for their data fields. To filter on data properties, make sure that you use the correct key in your filter’s pattern. For a list of data filtering keys, and to see examples of filter patterns for each supported AWS service, refer to Using filters with different AWS services.

Event filtering can handle multi-level JSON filtering. For example, consider the following fragment of a record from a DynamoDB stream:

"dynamodb": {
    "Keys": {
        "ID": {
            "S": "ABCD"
        }
        "Number": {
            "N": "1234"
    },
    ...
}

Suppose you want to process only those records where the value of the sort key Number is 4567. In this case, your FilterCriteria object would look like this:

{
    "Filters": [
        {
            "Pattern": "{ \"dynamodb\": { \"Keys\": { \"Number\": { \"N\": [ "4567" ] } } } }"
        }
    ]
}

For added clarity, here is the value of the filter's Pattern expanded in plain JSON.

{
    "dynamodb": {
        "Keys": {
            "Number": {
                "N": [ "4567" ]
                }
            }
        }
}

Handling records that don't meet filter criteria

How Lambda handles records that don't meet your filter criteria depends on the event source.

Filter rule syntax

For filter rules, Lambda supports the Amazon EventBridge rules and uses the same syntax as EventBridge. For more information, see Amazon EventBridge event patterns in the Amazon EventBridge User Guide.

The following is a summary of all the comparison operators available for Lambda event filtering.

Note that the Exists operator only works on leaf nodes in your event source JSON. It doesn't match intermediate nodes. For example, with the following JSON, the filter pattern { "person": { "address": [ { "exists": true } ] } }" wouldn't find a match because "address" is an intermediate node.

{
  "person": {
    "name": "John Doe",
    "age": 30,
    "address": {
      "street": "123 Main St",
      "city": "Anytown",
      "country": "USA"
    }
  }
}

Attaching filter criteria to an event source mapping (console)

Follow these steps to create a new event source mapping with filter criteria using the Lambda console.

When you enter filter criteria using the console, you enter only the filter pattern and don't need to provide the Pattern key or escape quotes. In step 6 of the preceding instructions, { "Metadata" : [ 1, 2 ] } corresponds to the following FilterCriteria.

{
   "Filters": [
      {
          "Pattern": "{ \"Metadata\" : [ 1, 2 ] }"
      }
   ]
}

After creating your event source mapping in the console, you can see the formatted FilterCriteria in the trigger details. For more examples of creating event filters using the console, see Using filters with different AWS services.

Attaching filter criteria to an event source mapping (AWS CLI)

Suppose you want an event source mapping to have the following FilterCriteria:

{
   "Filters": [
      {
          "Pattern": "{ \"Metadata\" : [ 1, 2 ] }"
      }
   ]
}

To create a new event source mapping with these filter criteria using the AWS Command Line Interface (AWS CLI), run the following command.

aws lambda create-event-source-mapping \
    --function-name my-function \
    --event-source-arn arn:aws:sqs:us-east-2:123456789012:my-queue \
    --filter-criteria '{"Filters": [{"Pattern": "{ \"Metadata\" : [ 1, 2 ]}"}]}'

This create-event-source-mapping command creates a new Amazon SQS event source mapping for function my-function with the specified FilterCriteria.

To add these filter criteria to an existing event source mapping, run the following command.

aws lambda update-event-source-mapping \
    --uuid "a1b2c3d4-5678-90ab-cdef-11111EXAMPLE" \
    --filter-criteria '{"Filters": [{"Pattern": "{ \"Metadata\" : [ 1, 2 ]}"}]}'

Note that to update an event source mapping, you need its UUID. You can get the UUID from a list-event-source-mappings call. Lambda also returns the UUID in the create-event-source-mapping CLI response.

To remove filter criteria from an event source, you can run the following update-event-source-mapping command with an empty FilterCriteria object.

aws lambda update-event-source-mapping \
    --uuid "a1b2c3d4-5678-90ab-cdef-11111EXAMPLE" \
    --filter-criteria "{}"

For more examples of creating event filters using the AWS CLI, see Using filters with different AWS services.

Attaching filter criteria to an event source mapping (AWS SAM)

Suppose you want to configure an event source in AWS SAM to use the following filter criteria:

{
   "Filters": [
      {
          "Pattern": "{ \"Metadata\" : [ 1, 2 ] }"
      }
   ]
}

To add these filter criteria to your event source mapping, insert the following snippet into the YAML template for your event source.

FilterCriteria: 
  Filters: 
    - Pattern: '{"Metadata": [1, 2]}'

For more information on creating and configuring an AWS SAM template for an event source mapping, see the EventSource section of the AWS SAM Developer Guide. Fore more examples of creating event filters using AWS SAM templates, see Using filters with different AWS services.

Encryption of filter criteria

By default, Lambda doesn't encrypt your filter criteria object. For use cases where you may include sensitive information in your filter criteria object, you can use your own KMS key to encrypt it.

After you encrypt your filter criteria object, you can view its plaintext version using a GetEventSourceMapping API call. You must have kms:Decrypt permissions to be able to successfully view the filter criteria in plaintext.

You can specify your own KMS key via the console, API/CLI, or AWS CloudFormation.

If you use your own KMS key, the following API operations must be permitted in the key policy:

You can use AWS CloudTrail to track AWS KMS requests that Lambda makes on your behalf. For sample CloudTrail events, see Monitoring your encryption keys for Lambda.

We also recommend using the kms:ViaService condition key to limit the use of the KMS key to requests from Lambda only. The value of this key is the regional Lambda service principal (lambda.AWS_region.amazonaws.com). The following is a sample key policy that grants all the relevant permissions:

{
    "Version": "2012-10-17",
    "Id": "example-key-policy-1",
    "Statement": [
        {
            "Sid": "Allow Lambda to decrypt using the key",
            "Effect": "Allow",
            "Principal": {
                "Service": "lambda.us-east-1.amazonaws.com"
            },
            "Action": [
                "kms:Decrypt"
            ],
            "Resource": "*",
            "Condition": {
                "ArnEquals" : {
                    "aws:SourceArn": [
                        "arn:aws:lambda:us-east-1:123456789012:event-source-mapping:<esm_uuid>"
                    ]
                },
                "StringEquals": {
                    "kms:EncryptionContext:aws:lambda:FunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:test-function",
                    "kms:EncryptionContext:aws:lambda:EventSourceArn": "arn:aws:sqs:us-east-1:123456789012:test-queue"
                }
            }
        },
        {
            "Sid": "Allow actions by an AWS account on the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key to specific roles",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/ExampleRole"
            },
            "Action": [
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:GenerateDataKey"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals" : {
                    "kms:ViaService": "lambda.us-east-1.amazonaws.com"
                }
            }
        }
    ]
}

To use your own KMS key to encrypt filter criteria, you can also use the following CreateEventSourceMapping AWS CLI command. Specify the KMS key ARN with the --kms-key-arn flag.

aws lambda create-event-source-mapping --function-name my-function \
    --maximum-batching-window-in-seconds 60 \
    --event-source-arn arn:aws:sqs:us-east-1:123456789012:my-queue \
    --filter-criteria "{\"filters\": [{\"pattern\": \"{\"a\": [\"1\", \"2\"]}\" }]}" \
    --kms-key-arn arn:aws:kms:us-east-1:123456789012:key/055efbb4-xmpl-4336-ba9c-538c7d31f599

If you have an existing event source mapping, use the UpdateEventSourceMapping AWS CLI command instead. Specify the KMS key ARN with the --kms-key-arn flag.

aws lambda update-event-source-mapping --function-name my-function \
    --maximum-batching-window-in-seconds 60 \
    --event-source-arn arn:aws:sqs:us-east-1:123456789012:my-queue \
    --filter-criteria "{\"filters\": [{\"pattern\": \"{\"a\": [\"1\", \"2\"]}\" }]}" \
    --kms-key-arn arn:aws:kms:us-east-1:123456789012:key/055efbb4-xmpl-4336-ba9c-538c7d31f599

This operation overwrites any KMS key that was previously specified. If you specify the --kms-key-arn flag along with an empty argument, Lambda stops using your KMS key to encrypt filter criteria. Instead, Lambda defaults back to using an Amazon-owned key.

To specify your own KMS key in a AWS CloudFormation template, use the KMSKeyArn property of the AWS::Lambda::EventSourceMapping resource type. For example, you can insert the following snippet into the YAML template for your event source.

MyEventSourceMapping:
  Type: AWS::Lambda::EventSourceMapping
  Properties:
    ...
    FilterCriteria:
      Filters:
        - Pattern: '{"a": [1, 2]}'
    KMSKeyArn: "arn:aws:kms:us-east-1:123456789012:key/055efbb4-xmpl-4336-ba9c-538c7d31f599"
    ...

To be able to view your encrypted filter criteria in plaintext in a GetEventSourceMapping or DeleteEventSourceMapping API call, you must have kms:Decrypt permissions.

Starting August 6, 2024, the FilterCriteria field no longer shows up in AWS CloudTrail logs from CreateEventSourceMapping, UpdateEventSourceMapping, and DeleteEventSourceMapping API calls if your function doesn't use event filtering. If your function does use event filtering, the FilterCriteria field shows up as empty ({}). You can still view your filter criteria in plaintext in the response of GetEventSourceMapping API calls if you have kms:Decrypt permissions for the correct KMS key.

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "AROA123456789EXAMPLE:userid1",
        "arn": "arn:aws:sts::123456789012:assumed-role/Example/example-role",
        "accountId": "123456789012",
        "accessKeyId": "ASIAIOSFODNN7EXAMPLE",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "AROA987654321EXAMPLE",
                "arn": "arn:aws:iam::123456789012:role/User1",
                "accountId": "123456789012",
                "userName": "User1"
            },
            "webIdFederationData": {},
            "attributes": {
                "creationDate": "2024-05-09T20:35:01Z",
                "mfaAuthenticated": "false"
            }
        },
        "invokedBy": "AWS Internal"
    },
    "eventTime": "2024-05-09T21:05:41Z",
    "eventSource": "lambda.amazonaws.com",
    "eventName": "CreateEventSourceMapping20150331",
    "awsRegion": "us-east-2",
    "sourceIPAddress": "AWS Internal",
    "userAgent": "AWS Internal",
    "requestParameters": {
        "eventSourceArn": "arn:aws:sqs:us-east-2:123456789012:example-queue",
        "functionName": "example-function",
        "enabled": true,
        "batchSize": 10,
        "filterCriteria": {},
        "kMSKeyArn": "arn:aws:kms:us-east-2:123456789012:key/a1b2c3d4-5678-90ab-cdef-EXAMPLE11111",
        "scalingConfig": {},
        "maximumBatchingWindowInSeconds": 0,
        "sourceAccessConfigurations": []
    },
    "responseElements": {
        "uUID": "a1b2c3d4-5678-90ab-cdef-EXAMPLEaaaaa",
        "batchSize": 10,
        "maximumBatchingWindowInSeconds": 0,
        "eventSourceArn": "arn:aws:sqs:us-east-2:123456789012:example-queue",
        "filterCriteria": {},
        "kMSKeyArn": "arn:aws:kms:us-east-2:123456789012:key/a1b2c3d4-5678-90ab-cdef-EXAMPLE11111",
        "functionArn": "arn:aws:lambda:us-east-2:123456789012:function:example-function",
        "lastModified": "May 9, 2024, 9:05:41 PM",
        "state": "Creating",
        "stateTransitionReason": "USER_INITIATED",
        "functionResponseTypes": [],
        "eventSourceMappingArn": "arn:aws:lambda:us-east-2:123456789012:event-source-mapping:a1b2c3d4-5678-90ab-cdef-EXAMPLEbbbbb"
    },
    "requestID": "a1b2c3d4-5678-90ab-cdef-EXAMPLE33333",
    "eventID": "a1b2c3d4-5678-90ab-cdef-EXAMPLE22222",
    "readOnly": false,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "123456789012",
    "eventCategory": "Management",
    "sessionCredentialFromConsole": "true"
}

Using filters with different AWS services

Different types of event source use different key values for their data fields. To filter on data properties, make sure that you use the correct key in your filter’s pattern. The following table gives the filtering keys for each supported AWS service.

The following sections give examples of filter patterns for different types of event sources. They also provide definitions of supported incoming data formats and filter pattern body formats for each supported service.