Interface EvaluationReferenceInput.Builder

All Superinterfaces:
Buildable, CopyableBuilder<EvaluationReferenceInput.Builder,EvaluationReferenceInput>, SdkBuilder<EvaluationReferenceInput.Builder,EvaluationReferenceInput>, SdkPojo
Enclosing class:
EvaluationReferenceInput
@Mutable @NotThreadSafe public static interface EvaluationReferenceInput.Builder extends SdkPojo, CopyableBuilder<EvaluationReferenceInput.Builder,EvaluationReferenceInput>

  • Method Details

    • context

      EvaluationReferenceInput.Builder context(Context context)

      The span context that identifies which session or trace this reference input applies to, used for correlating ground truth with agent output.

      Parameters:
      context - The span context that identifies which session or trace this reference input applies to, used for correlating ground truth with agent output.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • context

      default EvaluationReferenceInput.Builder context(Consumer<Context.Builder> context)

      The span context that identifies which session or trace this reference input applies to, used for correlating ground truth with agent output.

      This is a convenience method that creates an instance of the Context.Builder avoiding the need to create one manually via Context.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to context(Context).

      Parameters:
      context - a consumer that will call methods on Context.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:

    • expectedResponse

      EvaluationReferenceInput.Builder expectedResponse(EvaluationContent expectedResponse)

      The expected response for trace-level evaluation. Built-in evaluators that support this field compare the agent's actual response against this value for assessment. Custom evaluators can access it through the {expected_response} placeholder in their instructions.

      Parameters:
      expectedResponse - The expected response for trace-level evaluation. Built-in evaluators that support this field compare the agent's actual response against this value for assessment. Custom evaluators can access it through the {expected_response} placeholder in their instructions.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • expectedResponse

      default EvaluationReferenceInput.Builder expectedResponse(Consumer<EvaluationContent.Builder> expectedResponse)

      The expected response for trace-level evaluation. Built-in evaluators that support this field compare the agent's actual response against this value for assessment. Custom evaluators can access it through the {expected_response} placeholder in their instructions.

      This is a convenience method that creates an instance of the EvaluationContent.Builder avoiding the need to create one manually via EvaluationContent.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to expectedResponse(EvaluationContent).

      Parameters:
      expectedResponse - a consumer that will call methods on EvaluationContent.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:

    • assertions

      EvaluationReferenceInput.Builder assertions(Collection<EvaluationContent> assertions)

      A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.

      Parameters:
      assertions - A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • assertions

      EvaluationReferenceInput.Builder assertions(EvaluationContent... assertions)

      A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.

      Parameters:
      assertions - A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • assertions

      EvaluationReferenceInput.Builder assertions(Consumer<EvaluationContent.Builder>... assertions)

      A list of assertion statements for session-level evaluation. Each assertion describes an expected behavior or outcome the agent should demonstrate during the session.

      This is a convenience method that creates an instance of the EvaluationContent.Builder avoiding the need to create one manually via EvaluationContent.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to assertions(List<EvaluationContent>).

      Parameters:
      assertions - a consumer that will call methods on EvaluationContent.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:

    • expectedTrajectory

      EvaluationReferenceInput.Builder expectedTrajectory(EvaluationExpectedTrajectory expectedTrajectory)

      The expected tool call sequence for session-level trajectory evaluation. Contains a list of tool names representing the tools the agent is expected to invoke.

      Parameters:
      expectedTrajectory - The expected tool call sequence for session-level trajectory evaluation. Contains a list of tool names representing the tools the agent is expected to invoke.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • expectedTrajectory

      default EvaluationReferenceInput.Builder expectedTrajectory(Consumer<EvaluationExpectedTrajectory.Builder> expectedTrajectory)

      The expected tool call sequence for session-level trajectory evaluation. Contains a list of tool names representing the tools the agent is expected to invoke.

      This is a convenience method that creates an instance of the EvaluationExpectedTrajectory.Builder avoiding the need to create one manually via EvaluationExpectedTrajectory.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to expectedTrajectory(EvaluationExpectedTrajectory).

      Parameters:
      expectedTrajectory - a consumer that will call methods on EvaluationExpectedTrajectory.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also: