InferenceConfiguration

The configuration parameters that control how the foundation model behaves during evaluation, including response generation settings.

Types

The maximum number of tokens to generate in the model response during evaluation.

The list of sequences that will cause the model to stop generating tokens when encountered.

The temperature value that controls randomness in the model's responses. Lower values produce more deterministic outputs.

The top-p sampling parameter that controls the diversity of the model's responses by limiting the cumulative probability of token choices.

open operator override fun equals(other: Any?): Boolean

open override fun hashCode(): Int

open override fun toString(): String