InferenceProviderTargetConfiguration¶
Structure Class¶
dataclass
¶
The configuration for a provider-based inference target. This configuration explicitly defines the endpoint, model mapping, and operations used to route requests to a large language model (LLM) provider.
Attributes¶
instance-attribute
¶
endpoint: str
The HTTPS endpoint of the inference provider that the gateway forwards requests to.
class-attribute
instance-attribute
¶
model_mapping: ModelMapping | None = None
The configuration that translates client-facing model IDs to the model IDs expected by the provider.
class-attribute
instance-attribute
¶
operations: list[InferenceOperationConfiguration] | None = None
A list of per-operation configurations that map request paths to the models supported for each operation.