Class: Aws::BedrockAgentRuntime::Types::TextInferenceConfig
- Inherits:
-
Struct
- Object
- Struct
- Aws::BedrockAgentRuntime::Types::TextInferenceConfig
- Defined in:
- gems/aws-sdk-bedrockagentruntime/lib/aws-sdk-bedrockagentruntime/types.rb
Overview
Configuration settings for text generation using a language model via the RetrieveAndGenerate operation. Includes parameters like temperature, top-p, maximum token count, and stop sequences.
maxTokens
depends on the accepted values for your
chosen model's inference parameters. To see the inference parameters
for your model, see Inference parameters for foundation models.
Constant Summary collapse
- SENSITIVE =
[]
Instance Attribute Summary collapse
-
#max_tokens ⇒ Integer
The maximum number of tokens to generate in the output text.
-
#stop_sequences ⇒ Array<String>
A list of sequences of characters that, if generated, will cause the model to stop generating further tokens.
-
#temperature ⇒ Float
Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options.
-
#top_p ⇒ Float
A probability distribution threshold which controls what the model considers for the set of possible next tokens.
Instance Attribute Details
#max_tokens ⇒ Integer
The maximum number of tokens to generate in the output text. Do not use the minimum of 0 or the maximum of 65536. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.
3639 3640 3641 3642 3643 3644 3645 3646 |
# File 'gems/aws-sdk-bedrockagentruntime/lib/aws-sdk-bedrockagentruntime/types.rb', line 3639 class TextInferenceConfig < Struct.new( :max_tokens, :stop_sequences, :temperature, :top_p) SENSITIVE = [] include Aws::Structure end |
#stop_sequences ⇒ Array<String>
A list of sequences of characters that, if generated, will cause the model to stop generating further tokens. Do not use a minimum length of 1 or a maximum length of 1000. The limit values described here are arbitary values, for actual values consult the limits defined by your specific model.
3639 3640 3641 3642 3643 3644 3645 3646 |
# File 'gems/aws-sdk-bedrockagentruntime/lib/aws-sdk-bedrockagentruntime/types.rb', line 3639 class TextInferenceConfig < Struct.new( :max_tokens, :stop_sequences, :temperature, :top_p) SENSITIVE = [] include Aws::Structure end |
#temperature ⇒ Float
Controls the random-ness of text generated by the language model, influencing how much the model sticks to the most predictable next words versus exploring more surprising options. A lower temperature value (e.g. 0.2 or 0.3) makes model outputs more deterministic or predictable, while a higher temperature (e.g. 0.8 or 0.9) makes the outputs more creative or unpredictable.
3639 3640 3641 3642 3643 3644 3645 3646 |
# File 'gems/aws-sdk-bedrockagentruntime/lib/aws-sdk-bedrockagentruntime/types.rb', line 3639 class TextInferenceConfig < Struct.new( :max_tokens, :stop_sequences, :temperature, :top_p) SENSITIVE = [] include Aws::Structure end |
#top_p ⇒ Float
A probability distribution threshold which controls what the model considers for the set of possible next tokens. The model will only consider the top p% of the probability distribution when generating the next token.
3639 3640 3641 3642 3643 3644 3645 3646 |
# File 'gems/aws-sdk-bedrockagentruntime/lib/aws-sdk-bedrockagentruntime/types.rb', line 3639 class TextInferenceConfig < Struct.new( :max_tokens, :stop_sequences, :temperature, :top_p) SENSITIVE = [] include Aws::Structure end |