Loading streaming data from Amazon Kinesis Data Streams - Amazon OpenSearch Service
Prerequisites

Loading streaming data from Amazon Kinesis Data Streams

You can load streaming data from Kinesis Data Streams to OpenSearch Service. New data that arrives in the data stream triggers an event notification to Lambda, which then runs your custom code to perform the indexing. This section includes some unsophisticated Python sample code.

Prerequisites

Before proceeding, you must have the following resources.

Prerequisite Description
Amazon Kinesis Data Stream The event source for your Lambda function. To learn more, see Kinesis Data Streams.
OpenSearch Service Domain The destination for data after your Lambda function processes it. For more information, see Creating OpenSearch Service domains
IAM Role

This role must have basic OpenSearch Service, Kinesis, and Lambda permissions, such as the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "es:ESHttpPost",
        "es:ESHttpPut",
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "kinesis:GetShardIterator",
        "kinesis:GetRecords",
        "kinesis:DescribeStream",
        "kinesis:ListStreams"
      ],
      "Resource": "*"
    }
  ]
}

The role must have the following trust relationship:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

To learn more, see Creating IAM roles in the IAM User Guide.