Skip to content

Bedrock Agentcore Control  >  Operations  >  create_online_evaluation_config

create_online_evaluation_config

Operation

create_online_evaluation_config async 

create_online_evaluation_config(input: CreateOnlineEvaluationConfigInput, plugins: list[Plugin] | None = None) -> CreateOnlineEvaluationConfigOutput

Creates an online evaluation configuration for continuous monitoring of agent performance. Online evaluation automatically samples live traffic from CloudWatch logs at specified rates and applies evaluators to assess agent quality in production.

Parameters:

Name Type Description Default
input CreateOnlineEvaluationConfigInput

An instance of CreateOnlineEvaluationConfigInput.

 required
plugins list[Plugin] | None

A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations.

 None

Returns:

Type Description
CreateOnlineEvaluationConfigOutput

An instance of CreateOnlineEvaluationConfigOutput.

Input

CreateOnlineEvaluationConfigInput dataclass

Dataclass for CreateOnlineEvaluationConfigInput structure.

Attributes

client_token class-attribute instance-attribute 
client_token: str | None = None

A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If you don't specify this field, a value is randomly generated for you. If this token matches a previous request, the service ignores the request, but doesn't return an error. For more information, see Ensuring idempotency.

clustering_config class-attribute instance-attribute 
clustering_config: ClusteringConfig | None = None

Configuration for periodic batch evaluation clustering of insight results.

data_source_config class-attribute instance-attribute 
data_source_config: DataSourceConfig | None = None

The data source configuration that specifies CloudWatch log groups and service names to monitor for agent traces.

description class-attribute instance-attribute 
description: str | None = field(repr=False, default=None)

The description of the online evaluation configuration that explains its monitoring purpose and scope.

enable_on_create class-attribute instance-attribute 
enable_on_create: bool | None = None

Whether to enable the online evaluation configuration immediately upon creation. If true, evaluation begins automatically.

evaluation_execution_role_arn class-attribute instance-attribute 
evaluation_execution_role_arn: str | None = None

The Amazon Resource Name (ARN) of the IAM role that grants permissions to read from CloudWatch logs, write evaluation results, and invoke Amazon Bedrock models for evaluation. If the configuration references evaluators encrypted with a customer managed KMS key, this role must also have kms:Decrypt permission on the KMS key. The service validates this permission at configuration creation time. For more information, see Encryption at rest for AgentCore Evaluations.

evaluators class-attribute instance-attribute 
evaluators: list[EvaluatorReference] | None = None

The list of evaluators to apply during online evaluation. Can include both built-in evaluators and custom evaluators created with CreateEvaluator.

insights class-attribute instance-attribute 
insights: list[Insight] | None = None

The list of insight types to run against agent sessions.

online_evaluation_config_name class-attribute instance-attribute 
online_evaluation_config_name: str | None = None

The name of the online evaluation configuration. Must be unique within your account.

rule class-attribute instance-attribute 
rule: Rule | None = None

The evaluation rule that defines sampling configuration, filters, and session detection settings for the online evaluation.

tags class-attribute instance-attribute 
tags: dict[str, str] | None = None

A map of tag keys and values to assign to an AgentCore Online Evaluation Config. Tags enable you to categorize your resources in different ways, for example, by purpose, owner, or environment.

Output

CreateOnlineEvaluationConfigOutput dataclass

Dataclass for CreateOnlineEvaluationConfigOutput structure.

Attributes

created_at instance-attribute 
created_at: datetime

The timestamp when the online evaluation configuration was created.

execution_status instance-attribute 
execution_status: OnlineEvaluationExecutionStatus

The execution status indicating whether the online evaluation is currently running.

failure_reason class-attribute instance-attribute 
failure_reason: str | None = None

The reason for failure if the online evaluation configuration creation or execution failed.

online_evaluation_config_arn instance-attribute 
online_evaluation_config_arn: str

The Amazon Resource Name (ARN) of the created online evaluation configuration.

online_evaluation_config_id instance-attribute 
online_evaluation_config_id: str

The unique identifier of the created online evaluation configuration.

output_config class-attribute instance-attribute 
output_config: OutputConfig | None = None

The configuration that specifies where evaluation results should be written for monitoring and analysis.

status instance-attribute 
status: OnlineEvaluationConfigStatus

The status of the online evaluation configuration.