create_evaluation_job¶
Operation¶
async
¶
create_evaluation_job(input: CreateEvaluationJobInput, plugins: list[Plugin] | None = None) -> CreateEvaluationJobOutput
Creates an evaluation job.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
CreateEvaluationJobInput
|
An instance of |
required |
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
Returns:
| Type | Description |
|---|---|
CreateEvaluationJobOutput
|
An instance of |
Input¶
dataclass
¶
Dataclass for CreateEvaluationJobInput structure.
Attributes¶
class-attribute
instance-attribute
¶
application_type: ApplicationType | None = None
Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation).
class-attribute
instance-attribute
¶
client_request_token: str | None = None
A unique, case-sensitive identifier to ensure that the API request completes no more than one time. If this token matches a previous request, Amazon Bedrock ignores the request, but does not return an error. For more information, see Ensuring idempotency.
class-attribute
instance-attribute
¶
customer_encryption_key_id: str | None = None
Specify your customer managed encryption key Amazon Resource Name (ARN) that will be used to encrypt your evaluation job.
class-attribute
instance-attribute
¶
evaluation_config: EvaluationConfig | None = None
Contains the configuration details of either an automated or human-based evaluation job.
class-attribute
instance-attribute
¶
inference_config: EvaluationInferenceConfig | None = None
Contains the configuration details of the inference model for the evaluation job.
For model evaluation jobs, automated jobs support a single model or inference profile, and jobs that use human workers support two models or inference profiles.
class-attribute
instance-attribute
¶
job_description: str | None = field(repr=False, default=None)
A description of the evaluation job.
class-attribute
instance-attribute
¶
job_name: str | None = None
A name for the evaluation job. Names must unique with your Amazon Web Services account, and your account's Amazon Web Services region.
class-attribute
instance-attribute
¶
job_tags: list[Tag] | None = None
Tags to attach to the model evaluation job.
class-attribute
instance-attribute
¶
output_data_config: EvaluationOutputDataConfig | None = None
Contains the configuration details of the Amazon S3 bucket for storing the results of the evaluation job.
class-attribute
instance-attribute
¶
role_arn: str | None = None
The Amazon Resource Name (ARN) of an IAM service role that Amazon Bedrock can assume to perform tasks on your behalf. To learn more about the required permissions, see Required permissions for model evaluations.