start_batch_evaluation¶
Operation¶
async
¶
start_batch_evaluation(input: StartBatchEvaluationInput, plugins: list[Plugin] | None = None) -> StartBatchEvaluationOutput
Starts a batch evaluation job that evaluates agent performance across multiple sessions. Batch evaluations pull agent traces from CloudWatch Logs or an existing online evaluation configuration and run specified evaluators and insights against them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
StartBatchEvaluationInput
|
An instance of |
required |
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
Returns:
| Type | Description |
|---|---|
StartBatchEvaluationOutput
|
An instance of |
Input¶
dataclass
¶
Dataclass for StartBatchEvaluationInput structure.
Attributes¶
class-attribute
instance-attribute
¶
batch_evaluation_name: str | None = None
The name of the batch evaluation. Must be unique within your account.
class-attribute
instance-attribute
¶
client_token: str | None = None
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, the service ignores the request, but does not return an error.
class-attribute
instance-attribute
¶
data_source_config: DataSourceConfig | None = None
The data source configuration that specifies where to pull agent session traces from for evaluation.
class-attribute
instance-attribute
¶
description: str | None = None
The description of the batch evaluation.
class-attribute
instance-attribute
¶
evaluation_metadata: EvaluationMetadata | None = None
Optional metadata for the evaluation, including session-specific ground truth data and test scenario identifiers.
class-attribute
instance-attribute
¶
evaluators: list[Evaluator] | None = None
The list of evaluators to apply during the batch evaluation. Can include both built-in evaluators and custom evaluators. Maximum of 10 evaluators.
class-attribute
instance-attribute
¶
insights: list[Insight] | None = None
The list of insight analyses to run against sessions during the batch evaluation. Maximum of 10 insights.
class-attribute
instance-attribute
¶
kms_key_arn: str | None = None
The ARN of the KMS key used to encrypt evaluation data. If provided, customer data is encrypted at rest with the specified key.
class-attribute
instance-attribute
¶
tags: dict[str, str] | None = None
A map of tag keys and values to associate with the batch evaluation.
Output¶
dataclass
¶
Dataclass for StartBatchEvaluationOutput structure.
Attributes¶
instance-attribute
¶
batch_evaluation_arn: str
The Amazon Resource Name (ARN) of the created batch evaluation.
instance-attribute
¶
batch_evaluation_id: str
The unique identifier of the created batch evaluation.
instance-attribute
¶
batch_evaluation_name: str
The name of the batch evaluation.
instance-attribute
¶
created_at: datetime
The timestamp when the batch evaluation was created.
class-attribute
instance-attribute
¶
description: str | None = None
The description of the batch evaluation.
class-attribute
instance-attribute
¶
evaluators: list[Evaluator] | None = None
The list of evaluators applied during the batch evaluation.
class-attribute
instance-attribute
¶
insights: list[Insight] | None = None
The list of insight analyses applied during the batch evaluation.
class-attribute
instance-attribute
¶
kms_key_arn: str | None = None
The ARN of the KMS key used to encrypt evaluation data.
class-attribute
instance-attribute
¶
output_config: OutputConfig | None = None
The output configuration specifying where evaluation results are written.
class-attribute
instance-attribute
¶
tags: dict[str, str] | None = None
The tags associated with the batch evaluation.