Amazon SageMaker 模型卡 - Amazon SageMaker

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

Amazon SageMaker 模型卡

重要

Amazon SageMaker 模型卡與 SageMaker 型號註冊表集成。如果您要在 Model Registry 中註冊模型,則可以使用整合來新增稽核資訊。如需詳細資訊,請參閱檢視和更新模型版本的詳細資訊

使用 Amazon SageMaker Model Card 在單一位置記錄機器學習 (ML) 模型的重要詳細資料,以簡化控管和報告。

目錄詳細資訊,例如模型的預定用途和風險評等、訓練詳細資訊和指標、評估結果和觀察,以及其他表示法 (例如考量事項、建議和自訂資訊)。透過建立模型卡,您可以執行下列操作:

  • 提供有關如何使用模型的指引。

  • 使用模型訓練和效能的詳細描述來支援稽核活動。

  • 溝通模型的目的是如何支援業務目標。

模型卡提供要記錄哪些資訊的方案指引,並包含自訂資訊的欄位。創建模型卡後,您可以將其導出到PDF或下載以與相關利益相關者共享。除了對模型卡進行的核准狀態更新,任何編輯都會產生額外的模型卡版本,以便擁有模型變更的不可變記錄。

必要條件

若要開始使用 Amazon SageMaker 模型卡,您必須擁有建立、編輯、檢視和匯出模型卡的權限。

模型的預期用途

指定模型的預定用途有助於確保模型開發人員和使用者負責地擁有訓練或部署模型所需的資訊。模型的預期用途應描述適合使用模型的情況,以及不建議使用模型的情況。

我們推薦包括:

  • 該模型的一般用途

  • 模型預定的使用案例

  • 模型並非預期使用的使用案例

  • 開發模型時所做的假設

模型的預定用途超出了技術細節,並描述了在生產環境中應如何使用模型、適合使用模型的情況,以及其他考量事項,例如與模型搭配使用的資料類型或在開發期間所做的任何假設。

風險評等

開發人員針對不同風險程度的使用案例建立機器學習 (ML) 模型。例如,核准貸款申請的模型可能比檢測到電子郵件類別的模型為風險更高的模型。鑑於模型的各種風險概況,模型卡為您提供了一個欄位,供您對模型的風險評等進行分類。

此風險評等可以是unknownlowmediumhigh。使用這些風險評等欄位來標示未知、低、中或高風險模型,並協助您的組織遵守有關將特定模型投入生產環境的任何現有規則。

模型卡JSON架構

型號卡的評估詳細資料必須以JSON格式提供。如果您擁有由SageMaker 澄清SageMaker 模型監控器產生的現有JSON格式評估報告,請將它們上傳到 Amazon S3,並提供 S3 URI 以自動剖析評估指標。如需詳細資訊和範例報告,請參閱 Amazon SageMaker 模型控管-模型卡範例筆記本中的範例指標資料夾。

使用 SageMaker Python 創建模型卡時SDK,模型內容必須在模型卡JSON架構中並以字符串形式提供。提供類似以下範例的模型內容。

{ "$schema": "http://json-schema.org/draft-07/schema#", "$id": "http://json-schema.org/draft-07/schema#", "title": "SageMakerModelCardSchema", "description": "Default model card schema", "version": "0.1.0", "type": "object", "additionalProperties": false, "properties": { "model_overview": { "description": "Overview about the model", "type": "object", "additionalProperties": false, "properties": { "model_description": { "description": "description of model", "type": "string", "maxLength": 1024 }, "model_owner": { "description": "Owner of model", "type": "string", "maxLength": 1024 }, "model_creator": { "description": "Creator of model", "type": "string", "maxLength": 1024 }, "problem_type": { "description": "Problem being solved with the model", "type": "string" }, "algorithm_type": { "description": "Algorithm used to solve the problem", "type": "string", "maxLength": 1024 }, "problem_type": { "description": "Problem being solved with the model", "type": "string" }, "model_owner": { "description": "Owner of model", "type": "string", "maxLength": 1024 } }, "model_id": { "description": "SageMaker Model Arn or Non SageMaker Model id", "type": "string", "maxLength": 1024 }, "model_artifact": { "description": "Location of the model artifact", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } }, "model_name": { "description": "Name of the model", "type": "string", "maxLength": 1024 }, "model_version": { "description": "Version of the model", "type": "number", "minimum": 1 }, "inference_environment": { "description": "Overview about the inference", "type": "object", "additionalProperties": false, "properties": { "container_image": { "description": "SageMaker inference image uri", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } } } } } }, "model_package_details": { "description": "Metadata information related to model package version", "type": "object", "additionalProperties": false, "properties": { "model_package_description": { "description": "A brief summary of the model package", "type": "string", "maxLength": 1024 }, "model_package_arn": { "description": "The Amazon Resource Name (ARN) of the model package", "type": "string", "minLength": 1, "maxLength": 2048 }, "created_by": { "description": "Information about the user who created model package.", "type": "object", "additionalProperties": false, "properties": { "user_profile_name": { "description": "The name of the user's profile in SageMaker Studio", "type": "string", "maxLength": 63 } } }, "model_package_status": { "description": "Current status of model package", "type": "string", "enum": [ "Pending", "InProgress", "Completed", "Failed", "Deleting" ] }, "model_approval_status": { "description": "Current approval status of model package", "type": "string", "enum": [ "Approved", "Rejected", "PendingManualApproval" ] }, "approval_description": { "description": "A description provided for the model approval", "type": "string", "maxLength": 1024 }, "model_package_group_name": { "description": "If the model is a versioned model, the name of the model group that the versioned model belongs to.", "type": "string", "minLength": 1, "maxLength": 63 }, "model_package_name": { "description": "Name of the model package", "type": "string", "minLength": 1, "maxLength": 63 }, "model_package_version": { "description": "Version of the model package", "type": "number", "minimum": 1 }, "domain": { "description": "The machine learning domain of the model package you specified. Common machine learning domains include computer vision and natural language processing.", "type": "string" }, "task": { "description": "The machine learning task you specified that your model package accomplishes. Common machine learning tasks include object detection and image classification.", "type": "string" }, "source_algorithms": { "description": "A list of algorithms that were used to create a model package.", "$ref": "#/definitions/source_algorithms" }, "inference_specification": { "description": "Details about inference jobs that can be run with models based on this model package.", "$ref": "#/definitions/inference_specification" } } }, "intended_uses": { "description": "Intended usage of model", "type": "object", "additionalProperties": false, "properties": { "purpose_of_model": { "description": "Why the model was developed?", "type": "string", "maxLength": 2048 }, "intended_uses": { "description": "intended use cases", "type": "string", "maxLength": 2048 }, "factors_affecting_model_efficiency": { "type": "string", "maxLength": 2048 }, "risk_rating": { "description": "Risk rating for model card", "$ref": "#/definitions/risk_rating" }, "explanations_for_risk_rating": { "type": "string", "maxLength": 2048 } } }, "business_details": { "description": "Business details of model", "type": "object", "additionalProperties": false, "properties": { "business_problem": { "description": "What business problem does the model solve?", "type": "string", "maxLength": 2048 }, "business_stakeholders": { "description": "Business stakeholders", "type": "string", "maxLength": 2048 }, "line_of_business": { "type": "string", "maxLength": 2048 } } }, "training_details": { "description": "Overview about the training", "type": "object", "additionalProperties": false, "properties": { "objective_function": { "description": "the objective function the model will optimize for", "function": { "$ref": "#/definitions/objective_function" }, "notes": { "type": "string", "maxLength": 1024 } }, "training_observations": { "type": "string", "maxLength": 1024 }, "training_job_details": { "type": "object", "additionalProperties": false, "properties": { "training_arn": { "description": "SageMaker Training job arn", "type": "string", "maxLength": 1024 }, "training_datasets": { "description": "Location of the model datasets", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } }, "training_environment": { "type": "object", "additionalProperties": false, "properties": { "container_image": { "description": "SageMaker training image uri", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } } } }, "training_metrics": { "type": "array", "items": { "maxItems": 50, "$ref": "#/definitions/training_metric" } }, "user_provided_training_metrics": { "type": "array", "items": { "maxItems": 50, "$ref": "#/definitions/training_metric" } }, "hyper_parameters": { "type": "array", "items": { "maxItems": 100, "$ref": "#/definitions/training_hyper_parameter" } }, "user_provided_hyper_parameters": { "type": "array", "items": { "maxItems": 100, "$ref": "#/definitions/training_hyper_parameter" } } } } } }, "evaluation_details": { "type": "array", "default": [], "items": { "type": "object", "required": [ "name" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,63}" }, "evaluation_observation": { "type": "string", "maxLength": 2096 }, "evaluation_job_arn": { "type": "string", "maxLength": 256 }, "datasets": { "type": "array", "items": { "type": "string", "maxLength": 1024 }, "maxItems": 10 }, "metadata": { "description": "additional attributes associated with the evaluation results", "type": "object", "additionalProperties": { "type": "string", "maxLength": 1024 } }, "metric_groups": { "type": "array", "default": [], "items": { "type": "object", "required": [ "name", "metric_data" ], "properties": { "name": { "type": "string", "pattern": ".{1,63}" }, "metric_data": { "type": "array", "items": { "anyOf": [ { "$ref": "#/definitions/simple_metric" }, { "$ref": "#/definitions/linear_graph_metric" }, { "$ref": "#/definitions/bar_chart_metric" }, { "$ref": "#/definitions/matrix_metric" } ] } } } } } } } }, "additional_information": { "additionalProperties": false, "type": "object", "properties": { "ethical_considerations": { "description": "Any ethical considerations that the author wants to provide", "type": "string", "maxLength": 2048 }, "caveats_and_recommendations": { "description": "Caveats and recommendations for people who might use this model in their applications.", "type": "string", "maxLength": 2048 }, "custom_details": { "type": "object", "additionalProperties": { "$ref": "#/definitions/custom_property" } } } } }, "definitions": { "source_algorithms": { "type": "array", "minContains": 1, "maxContains": 1, "items": { "type": "object", "additionalProperties": false, "required": [ "algorithm_name" ], "properties": { "algorithm_name": { "description": "The name of an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.", "type": "string", "maxLength": 170 }, "model_data_url": { "description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.", "type": "string", "maxLength": 1024 } } } }, "inference_specification": { "type": "object", "additionalProperties": false, "required": [ "containers" ], "properties": { "containers": { "description": "Contains inference related information which were used to create model package.", "type": "array", "minContains": 1, "maxContains": 15, "items": { "type": "object", "additionalProperties": false, "required": [ "image" ], "properties": { "model_data_url": { "description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.", "type": "string", "maxLength": 1024 }, "image": { "description": "Inference environment path. The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored.", "type": "string", "maxLength": 255 }, "nearest_model_name": { "description": "The name of a pre-trained machine learning benchmarked by Amazon SageMaker Inference Recommender model that matches your model.", "type": "string" } } } } } }, "risk_rating": { "description": "Risk rating of model", "type": "string", "enum": [ "High", "Medium", "Low", "Unknown" ] }, "custom_property": { "description": "Additional property in section", "type": "string", "maxLength": 1024 }, "objective_function": { "description": "objective function that training job is optimized for", "additionalProperties": false, "properties": { "function": { "type": "string", "enum": [ "Maximize", "Minimize" ] }, "facet": { "type": "string", "maxLength": 63 }, "condition": { "type": "string", "maxLength": 63 } } }, "training_metric": { "description": "training metric data", "type": "object", "required": [ "name", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "value": { "type": "number" } } }, "training_hyper_parameter": { "description": "training hyper parameter", "type": "object", "required": [ "name", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "value": { "type": "string", "pattern": ".{1,255}" } } }, "linear_graph_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "linear_graph" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2 }, "minItems": 1 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_string" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "bar_chart_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "bar_chart" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "number" }, "minItems": 1 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_array" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "matrix_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "matrix" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "array", "items": { "type": "number" }, "minItems": 1, "maxItems": 20 }, "minItems": 1, "maxItems": 20 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_array" }, "y_axis_name": { "$ref": "#/definitions/axis_name_array" } } }, "simple_metric": { "description": "metric data", "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "number", "string", "boolean" ] }, "value": { "anyOf": [ { "type": "number" }, { "type": "string", "maxLength": 63 }, { "type": "boolean" } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_string" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "axis_name_array": { "type": "array", "items": { "type": "string", "maxLength": 63 } }, "axis_name_string": { "type": "string", "maxLength": 63 } } }