Contextual ad targeting with Elemental Inference

Elemental Inference analyzes video content and produces IAB Content Taxonomy classifications and GARM brand safety signals for each shot. By integrating MediaTailor with Elemental Inference, you can enrich ad requests with contextual metadata. Your ad decision server can then target ads based on what is happening in the content.

With this integration, you use the AWS_SERVICE_REQUEST function type to call the Elemental Inference GetMetadata API during ad breaks. For more information about the function type, see AWS service request.

The following describes the integration flow:

Prerequisites

Before you configure this integration, you need the following:

MediaTailor and Elemental Inference do not need to be in the same AWS Region. The supported combinations depend on whether each service is in an opt-in Region. For details, see Grant MediaTailor access to your Elemental Inference feed. For the Regions where Elemental Inference is available, see Elemental Inference endpoints and quotas.

Grant MediaTailor access to your Elemental Inference feed

MediaTailor calls GetMetadata using its own service credentials. The target Elemental Inference feed must have a resource policy that trusts the MediaTailor service principal.

MediaTailor automatically adds security headers that identify your account and playback configuration, ensuring your Elemental Inference feed can verify the request originates from your MediaTailor resources.

The following table shows which service principal to use based on your deployment scenario.

The following example resource policy grants MediaTailor permission to call GetMetadata on a feed.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "mediatailor.amazonaws.com"
            },
            "Action": "elemental-inference:GetMetadata",
            "Resource": "arn:aws:elemental-inference:us-west-2:123456789012:feed/my-contextual-feed",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "123456789012"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:mediatailor:us-west-2:123456789012:playbackConfiguration/*"
                }
            }
        }
    ]
}

Replace us-west-2, 123456789012, and my-contextual-feed with your Region, account ID, and feed ID.

The aws:SourceAccount condition ensures only MediaTailor operating on behalf of your account can access the feed. The aws:SourceArn condition further restricts access to specific playback configurations. Replace the wildcard with a specific playback configuration name for tighter scoping.

The following example shows a resource policy for an opt-in Region, where the service principal includes the Region name.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "mediatailor.af-south-1.amazonaws.com"
            },
            "Action": "elemental-inference:GetMetadata",
            "Resource": "arn:aws:elemental-inference:us-west-2:123456789012:feed/my-contextual-feed",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "123456789012"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:mediatailor:af-south-1:123456789012:playbackConfiguration/*"
                }
            }
        }
    ]
}

Configure using the MediaTailor console

The MediaTailor console provides a guided recipe for configuring Elemental Inference contextual metadata functions. The recipe automatically constructs the URL, templates the request body, and generates output expressions.

Before you begin

Complete the steps in Prerequisites. You need an Elemental Inference feed with a contextual metadata output, a resource policy granting MediaTailor access, and a MediaLive channel with contextual metadata enrichment enabled.

Create the function

Attach to a playback configuration

After creating the function, attach it to a playback configuration at the PRE_ADS_REQUEST lifecycle hook. You can attach it directly or as a step within a SEQUENTIAL_EXECUTOR function.

Configure using the API or CLI

To provide accurate timing information for Elemental Inference queries, enable MediaLive contextual metadata enrichment. The following sections describe the recommended production configuration using SCTE-35 markers from MediaLive, and a manual configuration option for testing purposes.

Using SCTE-35 markers from MediaLive (production)

When MediaLive is configured for contextual metadata enrichment, it embeds Elemental Inference query parameters directly in SCTE-35 ad break markers. MediaTailor parses these automatically and exposes them as inference.* variables. For the full list of available variables, see Elemental Inference contextual metadata variables (inference.*).

Use this approach for production deployments for the following reasons:

For instructions on enabling contextual metadata enrichment on your MediaLive channel, see Set up contextual metadata enrichment in the AWS Elemental MediaLive User Guide.

The following example shows a complete function configuration using SCTE-35 enrichment.

{
    "FunctionId": "eiContextualMetadata",
    "FunctionType": "AWS_SERVICE_REQUEST",
    "AwsServiceRequestConfiguration": {
        "Runtime": "JSONATA",
        "TargetService": "elemental-inference",
        "TargetRegion": "{%inference.region%}",
        "MethodType": "POST",
        "Url": "{%inference.dataEndpoint & '/v1/feed/' & inference.feedId & '/input/0/metadata'%}",
        "Headers": {
            "Content-Type": "application/json"
        },
        "Body": "{%'{\"outputName\": \"my-contextual-output\", \"timeSpecification\": {\"ptsBased\": {\"startPts\": ' & $string(($exists(inference.previousBreakEndPts) and inference.previousBreakEndPts > inference.pts - 30 * inference.timescale ? inference.previousBreakEndPts : inference.pts - 30 * inference.timescale)) & ', \"endPts\": ' & $string(inference.pts + 1) & ', \"timescale\": ' & $string(inference.timescale) & '}}, \"parameters\": {\"contextualMetadata\": {}}}' %}",
        "RequestTimeoutMilliseconds": 2000,
        "Output": {
            "player_params.iabCategories": "{%response.statusCode = 200 ? $join($distinct(response.body.items.metadata.contextualMetadata.iabTaxonomy.categories.uniqueId), ',') : ''%}",
            "player_params.garmExcluded": "{%response.statusCode = 200 ? $join($distinct(response.body.items.metadata.contextualMetadata.garm.suitability.categories[flagged = true].category), ',') : ''%}"
        }
    }
}

The following list explains the key fields in the preceding example:

{ "FunctionId": "eiContextualMetadata", "RunCondition": "{%inference.enriched = true%}" }

Manual configuration (testing only)

Use this configuration for testing only. Without SCTE-35 enrichment from MediaLive, you must determine PTS values manually and cannot benefit from dynamic timing alignment. This mode has limited functionality. We do not support it for production workflows.

{
    "FunctionId": "eiContextualMetadataManual",
    "FunctionType": "AWS_SERVICE_REQUEST",
    "AwsServiceRequestConfiguration": {
        "Runtime": "JSONATA",
        "TargetService": "elemental-inference",
        "TargetRegion": "us-west-2",
        "MethodType": "POST",
        "Url": "https://abc123.elemental-inference-data.us-west-2.amazonaws.com/v1/feed/my-feed-id/input/0/metadata",
        "Headers": {
            "Content-Type": "application/json"
        },
        "Body": "{%'{\"outputName\": \"my-contextual-output\", \"timeSpecification\": {\"ptsBased\": {\"startPts\": 6300001, \"endPts\": 9000001, \"timescale\": 90000}}, \"parameters\": {\"contextualMetadata\": {}}}' %}",
        "RequestTimeoutMilliseconds": 2000,
        "Output": {
            "player_params.iabCategories": "{%response.statusCode = 200 ? $join($distinct(response.body.items.metadata.contextualMetadata.iabTaxonomy.categories.uniqueId), ',') : ''%}"
        }
    }
}

Attach to a playback configuration

After creating the function, attach it to a playback configuration at the PRE_ADS_REQUEST lifecycle hook using the PutPlaybackConfiguration API. Include the function ID in the function mapping:

{
    "FunctionMappings": {
        "PRE_ADS_REQUEST": {
            "FunctionId": "eiContextualMetadata"
        }
    }
}

You can also wrap it in a SEQUENTIAL_EXECUTOR to chain with other functions. For more information, see Chain with other functions.

Understand the time window

The Body expression builds a PTS-based time window for the GetMetadata request. The following list describes the lookback logic:

Use the response in ad requests

The following table describes the available output destination prefixes and their behavior.

For more output expression recipes, see Example 3: Contextual metadata.

Chain with other functions

To combine the Elemental Inference call with other functions, wrap them in a SEQUENTIAL_EXECUTOR. The following example shows a pipeline where the second function only runs when Elemental Inference successfully returned categories.

{
    "FunctionId": "contextualAdPipeline",
    "FunctionType": "SEQUENTIAL_EXECUTOR",
    "SequentialExecutorConfiguration": {
        "Runtime": "JSONATA",
        "FunctionList": [
            { "FunctionId": "eiContextualMetadata" },
            {
                "FunctionId": "buildEnrichedAdsRequest",
                "RunCondition": "{%$exists(player_params.iabCategories) and player_params.iabCategories != ''%}"
            }
        ],
        "TimeoutMilliseconds": 3000
    }
}

The RunCondition ensures that buildEnrichedAdsRequest only runs when the Elemental Inference function produces data. You can then build the ad decision server request using the contextual data.

Troubleshooting

The following table describes common issues and their resolutions.