Class: Aws::Bedrock::Types::InferenceConfiguration

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-bedrock/lib/aws-sdk-bedrock/types.rb

Overview

Base inference parameters to pass to a model. For more information, see Inference parameters for foundation models.

See Also:

Constant Summary collapse

SENSITIVE = 
[]

Instance Attribute Summary collapse

Instance Attribute Details

#max_tokens ⇒ Integer

The maximum number of tokens to allow in the generated response. The default value is the maximum allowed value for the model that you are using.

Returns:

  • (Integer) 


9771
9772
9773
9774
9775
9776
9777
9778
 
# File 'gems/aws-sdk-bedrock/lib/aws-sdk-bedrock/types.rb', line 9771

class InferenceConfiguration < Struct.new(
  :max_tokens,
  :temperature,
  :top_p,
  :stop_sequences)
  SENSITIVE = []
  include Aws::Structure
end

#stop_sequences ⇒ Array<String>

A list of stop sequences. A stop sequence is a sequence of characters that causes the model to stop generating the response.

Returns:

  • (Array<String>) 


9771
9772
9773
9774
9775
9776
9777
9778
 
# File 'gems/aws-sdk-bedrock/lib/aws-sdk-bedrock/types.rb', line 9771

class InferenceConfiguration < Struct.new(
  :max_tokens,
  :temperature,
  :top_p,
  :stop_sequences)
  SENSITIVE = []
  include Aws::Structure
end

#temperature ⇒ Float

The likelihood of the model selecting higher-probability options while generating a response. A lower value makes the model more likely to choose higher-probability options, while a higher value makes the model more likely to choose lower-probability options.

Returns:

  • (Float) 


9771
9772
9773
9774
9775
9776
9777
9778
 
# File 'gems/aws-sdk-bedrock/lib/aws-sdk-bedrock/types.rb', line 9771

class InferenceConfiguration < Struct.new(
  :max_tokens,
  :temperature,
  :top_p,
  :stop_sequences)
  SENSITIVE = []
  include Aws::Structure
end

#top_p ⇒ Float

The percentage of most-likely candidates that the model considers for the next token. For example, if you choose a value of 0.8 for topP, the model selects from the top 80% of the probability distribution of tokens that could be next in the sequence.

Returns:

  • (Float) 


9771
9772
9773
9774
9775
9776
9777
9778
 
# File 'gems/aws-sdk-bedrock/lib/aws-sdk-bedrock/types.rb', line 9771

class InferenceConfiguration < Struct.new(
  :max_tokens,
  :temperature,
  :top_p,
  :stop_sequences)
  SENSITIVE = []
  include Aws::Structure
end