AutomatedEvaluationConfig¶
Structure Class¶
dataclass
¶
The configuration details of an automated evaluation job. The
EvaluationDatasetMetricConfig object is used to specify the prompt
datasets, task type, and metric names.
Attributes¶
class-attribute
instance-attribute
¶
custom_metric_config: AutomatedEvaluationCustomMetricConfig | None = None
Defines the configuration of custom metrics to be used in an evaluation job.
instance-attribute
¶
dataset_metric_configs: list[EvaluationDatasetMetricConfig]
Configuration details of the prompt datasets and metrics you want to use for your evaluation job.
class-attribute
instance-attribute
¶
evaluator_model_config: EvaluatorModelConfig | None = None
Contains the evaluator model configuration details.
EvaluatorModelConfig is required for evaluation jobs that use a
knowledge base or in model evaluation job that use a model as judge.
This model computes all evaluation related metrics.