LlmAsAJudgeEvaluatorConfig¶
Structure Class¶
dataclass
¶
The configuration for LLM-as-a-Judge evaluation that uses a language model to assess agent performance based on custom instructions and rating scales.
Attributes¶
class-attribute
instance-attribute
¶
instructions: str = field(repr=False)
The evaluation instructions that guide the language model in assessing agent performance, including criteria and evaluation guidelines.
instance-attribute
¶
model_config: EvaluatorModelConfig
The model configuration that specifies which foundation model to use and how to configure it for evaluation.
class-attribute
instance-attribute
¶
rating_scale: RatingScale = field(repr=False)
The rating scale that defines how the evaluator should score agent performance, either numerical or categorical.