Amazon SageMaker 模型卡 - Amazon SageMaker

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

Amazon SageMaker 模型卡

重要

Amazon SageMaker Model Card 已與 SageMaker Model Registry 整合。如果您要在模型登錄檔中註冊模型,您可以使用 整合來新增稽核資訊。如需詳細資訊,請參閱更新模型版本的詳細資訊

使用 Amazon SageMaker Model Card 將機器學習 (ML) 模型的重要詳細資訊記錄於單一位置,以簡化管理和報告。模型卡可協助您在整個模型生命週期中擷取模型的重要資訊,並實作負責任的 AI 實務。

目錄詳細資訊,例如模型的預定用途和風險評等、訓練詳細資訊和指標、評估結果和觀察,以及其他表示法 (例如考量事項、建議和自訂資訊)。透過建立模型卡,您可以執行下列操作:

  • 提供有關如何使用模型的指引。

  • 使用模型訓練和效能的詳細描述來支援稽核活動。

  • 溝通模型的目的是如何支援業務目標。

模型卡提供要記錄哪些資訊的方案指引,並包含自訂資訊的欄位。建立模型卡後,您可以將其匯出至 PDF或下載,以便與相關利益相關者共用。除了對模型卡進行的核准狀態更新,任何編輯都會產生額外的模型卡版本,以便擁有模型變更的不可變記錄。

必要條件

若要開始使用 Amazon SageMaker Model Card,您必須具有建立、編輯、檢視和匯出模型卡的許可。

模型的預期用途

指定模型的預定用途有助於確保模型開發人員和使用者負責地擁有訓練或部署模型所需的資訊。模型的預期用途應描述適合使用模型的情況,以及不建議使用模型的情況。

我們推薦包括:

  • 該模型的一般用途

  • 模型預定的使用案例

  • 模型並非預期使用的使用案例

  • 開發模型時所做的假設

模型的預定用途超出了技術細節,並描述了在生產環境中應如何使用模型、適合使用模型的情況,以及其他考量事項,例如與模型搭配使用的資料類型或在開發期間所做的任何假設。

風險評等

開發人員針對不同風險程度的使用案例建立機器學習 (ML) 模型。例如,核准貸款申請的模型可能比檢測到電子郵件類別的模型為風險更高的模型。鑑於模型的各種風險概況,模型卡為您提供了一個欄位,供您對模型的風險評等進行分類。

此風險評等可以是unknownlowmediumhigh。使用這些風險評等欄位來標示未知、低、中或高風險模型,並協助您的組織遵守有關將特定模型投入生產環境的任何現有規則。

模型卡JSON結構描述

模型卡的評估詳細資訊必須以 JSON 格式提供。如果您有 SageMaker Clarify SageMaker 或模型監視器 產生的現有JSON格式評估報告,請將其上傳至 Amazon S3,並提供 S3 URI以自動剖析評估指標。如需詳細資訊和範例報告,請參閱 Amazon SageMaker Model Governance - Model Card 範例筆記本中的範例指標資料夾。

使用 SageMaker Python 建立模型卡時SDK,模型內容必須位於模型卡JSON結構描述中,並以字串的形式提供。提供類似以下範例的模型內容。

{ "$schema": "http://json-schema.org/draft-07/schema#", "$id": "http://json-schema.org/draft-07/schema#", "title": "SageMakerModelCardSchema", "description": "Internal model card schema for SageMakerRepositoryService without model_package_details", "version": "0.1.0", "type": "object", "additionalProperties": false, "properties": { "model_overview": { "description": "Overview about the model", "type": "object", "additionalProperties": false, "properties": { "model_description": { "description": "description of model", "type": "string", "maxLength": 1024 }, "model_creator": { "description": "Creator of model", "type": "string", "maxLength": 1024 }, "model_artifact": { "description": "Location of the model artifact", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } }, "algorithm_type": { "description": "Algorithm used to solve the problem", "type": "string", "maxLength": 1024 }, "problem_type": { "description": "Problem being solved with the model", "type": "string" }, "model_owner": { "description": "Owner of model", "type": "string", "maxLength": 1024 } } }, "intended_uses": { "description": "Intended usage of model", "type": "object", "additionalProperties": false, "properties": { "purpose_of_model": { "description": "Why the model was developed?", "type": "string", "maxLength": 2048 }, "intended_uses": { "description": "intended use cases", "type": "string", "maxLength": 2048 }, "factors_affecting_model_efficiency": { "type": "string", "maxLength": 2048 }, "risk_rating": { "description": "Risk rating for model card", "$ref": "#/definitions/risk_rating" }, "explanations_for_risk_rating": { "type": "string", "maxLength": 2048 } } }, "business_details": { "description": "Business details of model", "type": "object", "additionalProperties": false, "properties": { "business_problem": { "description": "What business problem does the model solve?", "type": "string", "maxLength": 2048 }, "business_stakeholders": { "description": "Business stakeholders", "type": "string", "maxLength": 2048 }, "line_of_business": { "type": "string", "maxLength": 2048 } } }, "training_details": { "description": "Overview about the training", "type": "object", "additionalProperties": false, "properties": { "objective_function": { "description": "the objective function the model will optimize for", "function": { "$ref": "#/definitions/objective_function" }, "notes": { "type": "string", "maxLength": 1024 } }, "training_observations": { "type": "string", "maxLength": 1024 }, "training_job_details": { "type": "object", "additionalProperties": false, "properties": { "training_arn": { "description": "SageMaker Training job arn", "type": "string", "maxLength": 1024 }, "training_datasets": { "description": "Location of the model datasets", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } }, "training_environment": { "type": "object", "additionalProperties": false, "properties": { "container_image": { "description": "SageMaker training image uri", "type": "array", "maxContains": 15, "items": { "type": "string", "maxLength": 1024 } } } }, "training_metrics": { "type": "array", "items": { "maxItems": 50, "$ref": "#/definitions/training_metric" } }, "user_provided_training_metrics": { "type": "array", "items": { "maxItems": 50, "$ref": "#/definitions/training_metric" } }, "hyper_parameters": { "type": "array", "items": { "maxItems": 100, "$ref": "#/definitions/training_hyper_parameter" } }, "user_provided_hyper_parameters": { "type": "array", "items": { "maxItems": 100, "$ref": "#/definitions/training_hyper_parameter" } } } } } }, "evaluation_details": { "type": "array", "default": [], "items": { "type": "object", "required": [ "name" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,63}" }, "evaluation_observation": { "type": "string", "maxLength": 2096 }, "evaluation_job_arn": { "type": "string", "maxLength": 256 }, "datasets": { "type": "array", "items": { "type": "string", "maxLength": 1024 }, "maxItems": 10 }, "metadata": { "description": "additional attributes associated with the evaluation results", "type": "object", "additionalProperties": { "type": "string", "maxLength": 1024 } }, "metric_groups": { "type": "array", "default": [], "items": { "type": "object", "required": [ "name", "metric_data" ], "properties": { "name": { "type": "string", "pattern": ".{1,63}" }, "metric_data": { "type": "array", "items": { "anyOf": [ { "$ref": "#/definitions/simple_metric" }, { "$ref": "#/definitions/linear_graph_metric" }, { "$ref": "#/definitions/bar_chart_metric" }, { "$ref": "#/definitions/matrix_metric" } ] } } } } } } } }, "additional_information": { "additionalProperties": false, "type": "object", "properties": { "ethical_considerations": { "description": "Any ethical considerations that the author wants to provide", "type": "string", "maxLength": 2048 }, "caveats_and_recommendations": { "description": "Caveats and recommendations for people who might use this model in their applications.", "type": "string", "maxLength": 2048 }, "custom_details": { "type": "object", "additionalProperties": { "$ref": "#/definitions/custom_property" } } } } }, "definitions": { "source_algorithms": { "type": "array", "minContains": 1, "maxContains": 1, "items": { "type": "object", "additionalProperties": false, "required": [ "algorithm_name" ], "properties": { "algorithm_name": { "description": "The name of an algorithm that was used to create the model package. The algorithm must be either an algorithm resource in your SageMaker account or an algorithm in AWS Marketplace that you are subscribed to.", "type": "string", "maxLength": 170 }, "model_data_url": { "description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.", "type": "string", "maxLength": 1024 } } } }, "inference_specification": { "type": "object", "additionalProperties": false, "required": [ "containers" ], "properties": { "containers": { "description": "Contains inference related information which were used to create model package.", "type": "array", "minContains": 1, "maxContains": 15, "items": { "type": "object", "additionalProperties": false, "required": [ "image" ], "properties": { "model_data_url": { "description": "The Amazon S3 path where the model artifacts, which result from model training, are stored.", "type": "string", "maxLength": 1024 }, "image": { "description": "Inference environment path. The Amazon EC2 Container Registry (Amazon ECR) path where inference code is stored.", "type": "string", "maxLength": 255 }, "nearest_model_name": { "description": "The name of a pre-trained machine learning benchmarked by Amazon SageMaker Inference Recommender model that matches your model.", "type": "string" } } } } } }, "risk_rating": { "description": "Risk rating of model", "type": "string", "enum": [ "High", "Medium", "Low", "Unknown" ] }, "custom_property": { "description": "Additional property in section", "type": "string", "maxLength": 1024 }, "objective_function": { "description": "objective function that training job is optimized for", "additionalProperties": false, "properties": { "function": { "type": "string", "enum": [ "Maximize", "Minimize" ] }, "facet": { "type": "string", "maxLength": 63 }, "condition": { "type": "string", "maxLength": 63 } } }, "training_metric": { "description": "training metric data", "type": "object", "required": [ "name", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "value": { "type": "number" } } }, "training_hyper_parameter": { "description": "training hyper parameter", "type": "object", "required": [ "name" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "value": { "type": "string", "pattern": ".{0,255}" } } }, "linear_graph_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "linear_graph" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "array", "items": { "type": "number" }, "minItems": 2, "maxItems": 2 }, "minItems": 1 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_string" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "bar_chart_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "bar_chart" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "number" }, "minItems": 1 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_array" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "matrix_metric": { "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "matrix" ] }, "value": { "anyOf": [ { "type": "array", "items": { "type": "array", "items": { "type": "number" }, "minItems": 1, "maxItems": 20 }, "minItems": 1, "maxItems": 20 } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_array" }, "y_axis_name": { "$ref": "#/definitions/axis_name_array" } } }, "simple_metric": { "description": "metric data", "type": "object", "required": [ "name", "type", "value" ], "additionalProperties": false, "properties": { "name": { "type": "string", "pattern": ".{1,255}" }, "notes": { "type": "string", "maxLength": 1024 }, "type": { "type": "string", "enum": [ "number", "string", "boolean" ] }, "value": { "anyOf": [ { "type": "number" }, { "type": "string", "maxLength": 63 }, { "type": "boolean" } ] }, "x_axis_name": { "$ref": "#/definitions/axis_name_string" }, "y_axis_name": { "$ref": "#/definitions/axis_name_string" } } }, "axis_name_array": { "type": "array", "items": { "type": "string", "maxLength": 63 } }, "axis_name_string": { "type": "string", "maxLength": 63 } } }