Interface InferenceComponentComputeResourceRequirements.Builder

All Superinterfaces:
Buildable, CopyableBuilder<InferenceComponentComputeResourceRequirements.Builder,InferenceComponentComputeResourceRequirements>, SdkBuilder<InferenceComponentComputeResourceRequirements.Builder,InferenceComponentComputeResourceRequirements>, SdkPojo
Enclosing class:
InferenceComponentComputeResourceRequirements
@Mutable @NotThreadSafe public static interface InferenceComponentComputeResourceRequirements.Builder extends SdkPojo, CopyableBuilder<InferenceComponentComputeResourceRequirements.Builder,InferenceComponentComputeResourceRequirements>

  • Method Details

    • numberOfCpuCoresRequired

      InferenceComponentComputeResourceRequirements.Builder numberOfCpuCoresRequired(Float numberOfCpuCoresRequired)

      The number of CPU cores to allocate to run a model that you assign to an inference component.

      Parameters:
      numberOfCpuCoresRequired - The number of CPU cores to allocate to run a model that you assign to an inference component.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • numberOfAcceleratorDevicesRequired

      InferenceComponentComputeResourceRequirements.Builder numberOfAcceleratorDevicesRequired(Float numberOfAcceleratorDevicesRequired)

      The number of accelerators to allocate to run a model that you assign to an inference component. Accelerators include GPUs and Amazon Web Services Inferentia.

      Parameters:
      numberOfAcceleratorDevicesRequired - The number of accelerators to allocate to run a model that you assign to an inference component. Accelerators include GPUs and Amazon Web Services Inferentia.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • minMemoryRequiredInMb

      InferenceComponentComputeResourceRequirements.Builder minMemoryRequiredInMb(Integer minMemoryRequiredInMb)

      The minimum MB of memory to allocate to run a model that you assign to an inference component.

      Parameters:
      minMemoryRequiredInMb - The minimum MB of memory to allocate to run a model that you assign to an inference component.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • maxMemoryRequiredInMb

      InferenceComponentComputeResourceRequirements.Builder maxMemoryRequiredInMb(Integer maxMemoryRequiredInMb)

      The maximum MB of memory to allocate to run a model that you assign to an inference component.

      Parameters:
      maxMemoryRequiredInMb - The maximum MB of memory to allocate to run a model that you assign to an inference component.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • acceleratorPartitionType

      InferenceComponentComputeResourceRequirements.Builder acceleratorPartitionType(String acceleratorPartitionType)

      The accelerator partition profile this inference component occupies; on NVIDIA GPUs this is a Multi-Instance GPU (MIG) profile such as mig-3g.40gb. It must be one of the partition profiles declared in the target endpoint's accelerator partition layout, and when set, NumberOfAcceleratorDevicesRequired must be 1. This field is required when the endpoint's production variant has an accelerator partition layout. For endpoints without a layout, omit this field to place the inference component on a whole accelerator.

      Parameters:
      acceleratorPartitionType - The accelerator partition profile this inference component occupies; on NVIDIA GPUs this is a Multi-Instance GPU (MIG) profile such as mig-3g.40gb. It must be one of the partition profiles declared in the target endpoint's accelerator partition layout, and when set, NumberOfAcceleratorDevicesRequired must be 1. This field is required when the endpoint's production variant has an accelerator partition layout. For endpoints without a layout, omit this field to place the inference component on a whole accelerator.
      Returns:
      Returns a reference to this object so that method calls can be chained together.