SageMaker / Client / create_ai_recommendation_job

create_ai_recommendation_job¶

SageMaker.Client.create_ai_recommendation_job(**kwargs)¶

Creates a recommendation job that generates intelligent optimization recommendations for generative AI inference deployments. The job analyzes your model, workload configuration, and performance targets to recommend optimal instance types, model optimization techniques (such as quantization and speculative decoding), and deployment configurations.

Request Syntax

response = client.create_ai_recommendation_job(
    AIRecommendationJobName='string',
    ModelSource={
        'S3': {
            'S3Uri': 'string'
        }
    },
    OutputConfig={
        'S3OutputLocation': 'string',
        'ModelPackageGroupIdentifier': 'string',
        'MlflowConfig': {
            'MlflowResourceArn': 'string',
            'MlflowExperimentName': 'string',
            'MlflowRunName': 'string'
        }
    },
    AIWorkloadConfigIdentifier='string',
    PerformanceTarget={
        'Constraints': [
            {
                'Metric': 'ttft-ms'|'throughput'|'cost'
            },
        ]
    },
    RoleArn='string',
    InferenceSpecification={
        'Framework': 'LMI'|'VLLM'
    },
    OptimizeModel=True|False,
    ComputeSpec={
        'InstanceTypes': [
            'ml.g5.xlarge'|'ml.g5.2xlarge'|'ml.g5.4xlarge'|'ml.g5.8xlarge'|'ml.g5.12xlarge'|'ml.g5.16xlarge'|'ml.g5.24xlarge'|'ml.g5.48xlarge'|'ml.g6.xlarge'|'ml.g6.2xlarge'|'ml.g6.4xlarge'|'ml.g6.8xlarge'|'ml.g6.12xlarge'|'ml.g6.16xlarge'|'ml.g6.24xlarge'|'ml.g6.48xlarge'|'ml.g6e.xlarge'|'ml.g6e.2xlarge'|'ml.g6e.4xlarge'|'ml.g6e.8xlarge'|'ml.g6e.12xlarge'|'ml.g6e.16xlarge'|'ml.g6e.24xlarge'|'ml.g6e.48xlarge'|'ml.g7e.2xlarge'|'ml.g7e.4xlarge'|'ml.g7e.8xlarge'|'ml.g7e.12xlarge'|'ml.g7e.24xlarge'|'ml.g7e.48xlarge'|'ml.p3.2xlarge'|'ml.p3.8xlarge'|'ml.p3.16xlarge'|'ml.p4d.24xlarge'|'ml.p4de.24xlarge'|'ml.p5.4xlarge'|'ml.p5.48xlarge'|'ml.p5e.48xlarge'|'ml.p5en.48xlarge'|'ml.p6-b200.48xlarge',
        ],
        'CapacityReservationConfig': {
            'CapacityReservationPreference': 'capacity-reservations-only',
            'MlReservationArns': [
                'string',
            ]
        }
    },
    AdapterSource={
        'ModelPackageArns': [
            {
                'AdapterId': 'string',
                'ModelPackageArn': 'string'
            },
        ],
        'S3Uris': [
            {
                'AdapterId': 'string',
                'S3Uri': 'string'
            },
        ]
    },
    Tags=[
        {
            'Key': 'string',
            'Value': 'string'
        },
    ]
)

Parameters:

AIRecommendationJobName (string) –
[REQUIRED]

The name of the AI recommendation job. The name must be unique within your Amazon Web Services account in the current Amazon Web Services Region.
ModelSource (dict) –
[REQUIRED]

The source of the model to optimize. Specify the Amazon S3 location of the model artifacts.

Note
This is a Tagged Union structure. Only one of the following top level keys can be set: S3.
- S3 (dict) –
  
  The Amazon S3 location of the model artifacts.
  - S3Uri (string) –
    
    The Amazon S3 URI of the model artifacts.
OutputConfig (dict) –
[REQUIRED]

The output configuration for the recommendation job, including the Amazon S3 location for results and an optional model package group where the optimized model is registered.
- S3OutputLocation (string) –
  
  The Amazon S3 URI where recommendation results are stored.
- ModelPackageGroupIdentifier (string) –
  
  The name or Amazon Resource Name (ARN) of the model package group where the optimized model is registered as a new model package version.
- MlflowConfig (dict) –
  
  The MLflow tracking configuration for the job. If you don’t specify this parameter, MLflow tracking is disabled.
  - MlflowResourceArn (string) – [REQUIRED]
    
    The Amazon Resource Name (ARN) of the SageMaker managed MLflow resource.
  - MlflowExperimentName (string) –
    
    The MLflow experiment name used for tracking.
  - MlflowRunName (string) –
    
    The MLflow run name used for tracking.
AIWorkloadConfigIdentifier (string) –
[REQUIRED]

The name or Amazon Resource Name (ARN) of the AI workload configuration to use for this recommendation job.
PerformanceTarget (dict) –
[REQUIRED]

The performance targets for the recommendation job. Specify constraints on metrics such as time to first token ( ttft-ms), throughput, or cost.
- Constraints (list) – [REQUIRED]
  
  An array of performance constraints that define the optimization objectives.
  - (dict) –
    
    A performance constraint for an AI recommendation job.
    - Metric (string) – [REQUIRED]
      
      The performance metric. Valid values are ttft-ms (time to first token in milliseconds), throughput, and cost.
RoleArn (string) –
[REQUIRED]

The Amazon Resource Name (ARN) of an IAM role that enables Amazon SageMaker AI to perform tasks on your behalf.
InferenceSpecification (dict) –
The inference framework configuration. Specify the framework (such as LMI or vLLM) for the recommendation job.
- Framework (string) –
  
  The inference framework. Valid values are LMI and VLLM.
OptimizeModel (boolean) – Whether to allow model optimization techniques such as quantization, speculative decoding, and kernel tuning. The default is true.
ComputeSpec (dict) –
The compute resource specification for the recommendation job. You can specify up to 3 instance types to consider, and optionally provide capacity reservation configuration.
- InstanceTypes (list) –
  
  The list of instance types to consider for recommendations. You can specify up to 3 instance types.
  - (string) –
- CapacityReservationConfig (dict) –
  
  The capacity reservation configuration.
  - CapacityReservationPreference (string) –
    
    The capacity reservation preference. The only valid value is capacity-reservations-only.
  - MlReservationArns (list) –
    
    The list of ML reservation ARNs to use.
    - (string) –
AdapterSource (dict) –
The LoRA adapter source for the recommendation job. Specify either a list of model package ARNs or Amazon S3 URIs for your LoRA adapters. When this parameter is absent, the recommendation job runs without LoRA adapter support.

Note
This is a Tagged Union structure. Only one of the following top level keys can be set: ModelPackageArns, S3Uris.
- ModelPackageArns (list) –
  
  A list of LoRA adapters identified by their model package ARNs. Use this when your adapters were produced by a SageMaker AI fine-tuning workflow that registers model packages.
  - (dict) –
    
    A LoRA adapter entry identified by a model package ARN.
    - AdapterId (string) – [REQUIRED]
      
      A unique identifier for the adapter. This ID is used as the inference component name when the adapter is deployed. The ID must start and end with an alphanumeric character, can contain hyphens between alphanumeric characters, and can be up to 63 characters long.
    - ModelPackageArn (string) – [REQUIRED]
      
      The Amazon Resource Name (ARN) of the model package that contains the LoRA adapter artifacts.
- S3Uris (list) –
  
  A list of LoRA adapters identified by their Amazon S3 URIs. Use this when your adapters are stored as raw artifacts in Amazon S3.
  - (dict) –
    
    A LoRA adapter entry identified by an Amazon S3 URI.
    - AdapterId (string) – [REQUIRED]
      
      A unique identifier for the adapter. This ID is used as the inference component name when the adapter is deployed. The ID must start and end with an alphanumeric character, can contain hyphens between alphanumeric characters, and can be up to 63 characters long.
    - S3Uri (string) – [REQUIRED]
      
      The Amazon S3 URI of the directory that contains the LoRA adapter artifacts in PEFT format.
Tags (list) –
The metadata that you apply to Amazon Web Services resources to help you categorize and organize them.
- (dict) –
  
  A tag object that consists of a key and an optional value, used to manage metadata for SageMaker Amazon Web Services resources.
  
  You can add tags to notebook instances, training jobs, hyperparameter tuning jobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations, and endpoints. For more information on adding tags to SageMaker resources, see AddTags.
  
  For more information on adding metadata to your Amazon Web Services resources with tagging, see Tagging Amazon Web Services resources. For advice on best practices for managing Amazon Web Services resources with tagging, see Tagging Best Practices: Implement an Effective Amazon Web Services Resource Tagging Strategy.
  - Key (string) – [REQUIRED]
    
    The tag key. Tag keys must be unique per resource.
  - Value (string) – [REQUIRED]
    
    The tag value.

Return type:

dict

Returns:

Response Syntax

{
    'AIRecommendationJobArn': 'string'
}

Response Structure

(dict) –
- AIRecommendationJobArn (string) –
  
  The Amazon Resource Name (ARN) of the created recommendation job.

Exceptions

SageMaker.Client.exceptions.ResourceNotFound
SageMaker.Client.exceptions.ResourceInUse
SageMaker.Client.exceptions.ResourceLimitExceeded