Defines the compute resources to allocate to run a model, plus any adapter models, that you assign to an inference component. These resources include CPU cores, accelerators, and memory.
Syntax
To declare this entity in your AWS CloudFormation template, use the following syntax:
JSON
{
"MaxMemoryRequiredInMb" : Integer,
"MinMemoryRequiredInMb" : Integer,
"NumberOfAcceleratorDevicesRequired" : Number,
"NumberOfCpuCoresRequired" : Number
}
YAML
MaxMemoryRequiredInMb: Integer
MinMemoryRequiredInMb: Integer
NumberOfAcceleratorDevicesRequired:
Number
NumberOfCpuCoresRequired:
Number
Properties
MaxMemoryRequiredInMb-
The maximum MB of memory to allocate to run a model that you assign to an inference component.
Required: No
Type: Integer
Minimum:
128Update requires: No interruption
MinMemoryRequiredInMb-
The minimum MB of memory to allocate to run a model that you assign to an inference component.
Required: No
Type: Integer
Minimum:
128Update requires: No interruption
NumberOfAcceleratorDevicesRequired-
The number of accelerators to allocate to run a model that you assign to an inference component. Accelerators include GPUs and AWS Inferentia.
Required: No
Type: Number
Minimum:
1Update requires: No interruption
NumberOfCpuCoresRequired-
The number of CPU cores to allocate to run a model that you assign to an inference component.
Required: No
Type: Number
Minimum:
0Update requires: No interruption
