Bedrock Agent  >  Structures  >  PromptModelInferenceConfiguration

PromptModelInferenceConfiguration

Structure Class

PromptModelInferenceConfiguration dataclass

Contains inference configurations related to model inference for a prompt. For more information, see Inference parameters.

Attributes

max_tokens class-attribute instance-attribute

max_tokens: int | None = None

The maximum number of tokens to return in the response.

stop_sequences class-attribute instance-attribute

stop_sequences: list[str] | None = None

A list of strings that define sequences after which the model will stop generating.

temperature class-attribute instance-attribute

temperature: float | None = None

Controls the randomness of the response. Choose a lower value for more predictable outputs and a higher value for more surprising outputs.

top_p class-attribute instance-attribute

top_p: float | None = None

The percentage of most-likely candidates that the model considers for the next token.