实时推理

使用以下代码示例可获取候选项定义。

import sagemaker 
import boto3

session = sagemaker.session.Session()

sagemaker_client = boto3.client('sagemaker', region_name='us-west-2')
job_name = 'test-auto-ml-job'

describe_response = sm_client.describe_auto_ml_job(AutoMLJobName=job_name)
# extract the best candidate definition from DescribeAutoMLJob response
best_candidate = describe_response['BestCandidate']
# extract the InferenceContainers definition from the caandidate definition
inference_containers = best_candidate['InferenceContainers']

使用以下代码示例可创建模型。

# Create Model
model_name = 'test-model' 
sagemaker_role = 'arn:aws:iam:444455556666:role/sagemaker-execution-role'
create_model_response = sagemaker_client.create_model(
   ModelName = model_name,
   ExecutionRoleArn = sagemaker_role,
   Containers = inference_containers 
)       

使用以下代码示例可创建端点配置。

endpoint_config_name = 'test-endpoint-config'
                                                        
instance_type = 'ml.m5.2xlarge' 
# for all supported instance types, see 
# https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ProductionVariant.html#sagemaker-Type-ProductionVariant-InstanceType    # Create endpoint config

endpoint_config_response = sagemaker_client.create_endpoint_config(
   EndpointConfigName=endpoint_config_name, 
   ProductionVariants=[
       {
           "VariantName": "variant1",
           "ModelName": model_name, 
           "InstanceType": instance_type,
           "InitialInstanceCount": 1
       }
   ]
)

print(f"Created EndpointConfig: {endpoint_config_response['EndpointConfigArn']}")                

使用以下代码示例可创建端点并部署模型。

# create endpoint and deploy the model
endpoint_name = 'test-endpoint'
create_endpoint_response = sagemaker_client.create_endpoint(
                                            EndpointName=endpoint_name, 
                                            EndpointConfigName=endpoint_config_name)
print(create_endpoint_response)                 

使用以下代码示例可检查端点创建状态。

# describe endpoint creation status
status = sagemaker_client.describe_endpoint(EndpointName=endpoint_name)["EndpointStatus"]                    

使用以下命令结构可调用端点进行实时推理。

# once endpoint status is InService, you can invoke the endpoint for inferencing
if status == "InService":
  sm_runtime = boto3.Session().client('sagemaker-runtime')
  inference_result = sm_runtime.invoke_endpoint(EndpointName='test-endpoint', ContentType='text/csv', Body='1,2,3,4,class')    

使用以下代码示例可获取候选项定义。

aws sagemaker describe-auto-ml-job --auto-ml-job-name 'test-automl-job' --region us-west-2

使用以下代码示例可创建模型。

aws sagemaker create-model --model-name 'test-sagemaker-model'
--containers '[{
    "Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3", amzn-s3-demo-bucket1
    "ModelDataUrl": "s3://amzn-s3-demo-bucket/output/model.tar.gz",
    "Environment": {
        "AUTOML_SPARSE_ENCODE_RECORDIO_PROTOBUF": "1",
        "AUTOML_TRANSFORM_MODE": "feature-transform",
        "SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "application/x-recordio-protobuf",
        "SAGEMAKER_PROGRAM": "sagemaker_serve",
        "SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
    }
}, {
    "Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-xgboost:1.3-1-cpu-py3",
    "ModelDataUrl": "s3://amzn-s3-demo-bucket/output/model.tar.gz",
    "Environment": {
        "MAX_CONTENT_LENGTH": "20971520",
        "SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv",
        "SAGEMAKER_INFERENCE_OUTPUT": "predicted_label", 
        "SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,probabilities" 
    }
}, {
    "Image": "348316444620.dkr.ecr.us-west-2.amazonaws.com/sagemaker-sklearn-automl:2.5-1-cpu-py3", aws-region
    "ModelDataUrl": "s3://amzn-s3-demo-bucket/output/model.tar.gz", 
    "Environment": { 
        "AUTOML_TRANSFORM_MODE": "inverse-label-transform", 
        "SAGEMAKER_DEFAULT_INVOCATIONS_ACCEPT": "text/csv", 
        "SAGEMAKER_INFERENCE_INPUT": "predicted_label", 
        "SAGEMAKER_INFERENCE_OUTPUT": "predicted_label", 
        "SAGEMAKER_INFERENCE_SUPPORTED": "predicted_label,probability,labels,probabilities", 
        "SAGEMAKER_PROGRAM": "sagemaker_serve", 
        "SAGEMAKER_SUBMIT_DIRECTORY": "/opt/ml/model/code"
    } 
}]' \
--execution-role-arn 'arn:aws:iam::1234567890:role/sagemaker-execution-role' \ 
--region 'us-west-2'

有关详细信息，请参阅创建模型。

此 create model 命令会返回类似以下格式的响应：

{
    "ModelArn": "arn:aws:sagemaker:us-west-2:1234567890:model/test-sagemaker-model"
}

使用以下代码示例可创建端点配置。

aws sagemaker create-endpoint-config --endpoint-config-name 'test-endpoint-config' \
--production-variants '[{"VariantName": "variant1", 
                        "ModelName": "test-sagemaker-model",
                        "InitialInstanceCount": 1,
                        "InstanceType": "ml.m5.2xlarge"
                       }]' \
--region us-west-2 

create endpoint 配置命令会返回类似以下格式的响应：

{
    "EndpointConfigArn": "arn:aws:sagemaker:us-west-2:1234567890:endpoint-config/test-endpoint-config"
}

使用以下代码示例创建端点。

aws sagemaker create-endpoint --endpoint-name 'test-endpoint' \    
--endpoint-config-name 'test-endpoint-config' \                 
--region us-west-2

create endpoint 命令会返回类似以下格式的响应：

{
    "EndpointArn": "arn:aws:sagemaker:us-west-2:1234567890:endpoint/test-endpoint"
}

使用以下描述CLI端点代码示例检查端点部署的进度。

aws sagemaker describe-endpoint --endpoint-name 'test-endpoint' --region us-west-2

上面的进度检查将返回以下格式的响应。

{
    "EndpointName": "test-endpoint",
    "EndpointArn": "arn:aws:sagemaker:us-west-2:1234567890:endpoint/test-endpoint",
    "EndpointConfigName": "test-endpoint-config",
    "EndpointStatus": "Creating",
    "CreationTime": 1660251167.595,
    "LastModifiedTime": 1660251167.595
}

将 EndpointStatus 更改为 InService 后，端点即可用于实时推理。

使用以下命令结构可调用端点进行实时推理。

aws sagemaker-runtime invoke-endpoint --endpoint-name 'test-endpoint' \
--region 'us-west-2' \
--body '1,51,3.5,1.4,0.2' \
--content-type 'text/csv' \
'/tmp/inference_output'

有关更多选项，请参阅调用端点。