EvaluationReferenceInput¶
Structure Class¶
dataclass
¶
A reference input containing ground truth data for evaluation, scoped to a specific context level (session or trace) through its span context.
Attributes¶
class-attribute
instance-attribute
¶
assertions: list[EvaluationContent] | None = None
A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.
instance-attribute
¶
context: Context
The span context that identifies which session or trace this reference input applies to, used for correlating ground truth with agent output.
class-attribute
instance-attribute
¶
expected_response: EvaluationContent | None = None
The expected response for trace-level evaluation. Built-in evaluators
that support this field compare the agent's actual response against
this value for assessment. Custom evaluators can access it through the
{expected_response} placeholder in their instructions.
class-attribute
instance-attribute
¶
expected_trajectory: EvaluationExpectedTrajectory | None = None
The expected tool call sequence for session-level trajectory evaluation. Contains a list of tool names representing the tools the agent is expected to invoke.