本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。
先决条件
Multi-turn RL 使用 SageMaker AI CreateJob API,除了现有的模型自定义先决条件外,还需要其他权限。下表汇总了新增内容与已涵盖内容。
| 内容 | Status | 注意 |
|---|---|---|
| 信任政策:job.sagemaker.amazonaws.com | New | CreateJob API 是必需的 |
| 信任政策:bedrock-agentcore.amazonaws.com | New | 在代理运行时角色上是必需的(仅限AgentCore 路径) |
| PassRole: job.sagemaker.amazonaws.com | New | 现有的先决条件仅涵盖 sagemaker.amazonaws.com |
| Job 操作(CreateJob DescribeJob、等) | New | 不在 AmazonSageMakerFullAccess |
| bedrock-agentcore: ListAgentRuntimes | New | Studio 运行时选择器是必需的 |
| bedrock-agentcore: ListAgentRuntimeVersions | New | Studio 版本选择器必填项 |
| AmazonSageMakerJobFullAccess 托管策略 | New | 附加到任务执行角色 |
| AmazonSageMakerJobRuntimeAccess 托管策略 | New | 附加到代理运行时角色 |
| Lambda、Bedrock 部署、S3、KMS、mlFlow、 CloudWatch | 已经涵盖了 | 通过现有的先决条件或托管策略 |
来电者角色
{ "Version": "2012-10-17", "Statement": [ { "Sid": "PassRoleForCreateJob", "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws:iam::<account-id>:role/<ExecutionRoleName>", "Condition": { "StringEquals": { "iam:PassedToService": "job.sagemaker.amazonaws.com" } } }, { "Sid": "MTRLJobActions", "Effect": "Allow", "Action": [ "sagemaker:CreateJob", "sagemaker:DescribeJob", "sagemaker:StopJob", "sagemaker:DeleteJob" ], "Resource": "arn:aws:sagemaker:*:*:job/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "MTRLListJobsPermission", "Effect": "Allow", "Action": [ "sagemaker:ListJobs", "sagemaker:ListJobSchemaVersions", "sagemaker:DescribeJobSchemaVersion" ], "Resource": "*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } } ] }
PassRole — 让调用者将执行角色委托给job.sagemaker.amazonaws.com。
托管策略
Multi-turn RL 使用两个 AWS 托管策略。在创建任务之前,将每个人都分配到相应的角色。
AmazonSageMakerJobFullAccess— 附加到 A SageMaker I 任务执行角色(RoleArn您传递给的CreateJob)。
此策略向作业授予运行时所需的权限: read/write 访问 S3 中的训练数据和检查点、KMS S3-side 加密权限、访问 SageMaker AI 中心内容和模型包、mlFlow 实验跟踪、通过 Bedrock 或 L AgentCore ambda 调用代理、VPC 网络接口管理和日志写入。 CloudWatch 参考:AmazonSageMakerJobFullAccess。
AmazonSageMakerJobRuntimeAccess— 附加到代理运行时角色(您的代理所处的角色,无论是在 Bedrock AgentCore 上还是 Lambda 转发器后面)。此策略授予您的代理在推出期间调用的运行时 API:sagemaker:Sample、sagemaker:SampleWithResponseStream、sagemaker:CompleteRolloutsagemaker:UpdateReward、和sagemaker:CallWithBearerToken。参考:AmazonSageMakerJobRuntimeAccess。
信任政策更新
将以下job.sagemaker.amazonaws.com内容添加到您的执行角色的信任策略中:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "job.sagemaker.amazonaws.com" }, "Action": ["sts:AssumeRole", "sts:TagSession"] } ] }
如果使用 Bedrock AgentCore,则您的代理运行时角色还需要自己的信任策略:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "bedrock-agentcore.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
Studio 用户界面的其他内联政策
如果您通过 SageMaker AI Studio 提交作业,请将以下内联策略添加到您的 SageMaker AI 域执行角色中:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "MTRLJobActions", "Effect": "Allow", "Action": [ "sagemaker:CreateJob", "sagemaker:DescribeJob", "sagemaker:StopJob", "sagemaker:DeleteJob", "sagemaker:ListJobs", "sagemaker:ListJobSchemaVersions", "sagemaker:DescribeJobSchemaVersion" ], "Resource": "arn:aws:sagemaker:*:*:job/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockAgentCoreListPermissions", "Effect": "Allow", "Action": [ "bedrock-agentcore:ListAgentRuntimes", "bedrock-agentcore:ListAgentRuntimeVersions" ], "Resource": "*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "PassRoleForCreateJob", "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws:iam::*:role/*", "Condition": { "StringEquals": { "iam:PassedToService": "job.sagemaker.amazonaws.com", "aws:ResourceAccount": "${aws:PrincipalAccount}" } } } ] }
自定义策略选项
如果您更喜欢定义自己的策略而不是使用 AWS 托管策略,请使用以下内容作为起点。
SageMaker AI 任务执行角色策略(等同于AmazonSageMakerJobFullAccess):
{ "Version": "2012-10-17", "Statement": [ { "Sid": "S3Permissions", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:ListBucket" ], "Resource": "*", "Condition": { "StringEquals": { "s3:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "KMSPermissions", "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "arn:aws:kms:*:*:key/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" }, "StringLike": { "kms:ViaService": "s3.*.amazonaws.com" } } }, { "Sid": "KMSDescribeKey", "Effect": "Allow", "Action": "kms:DescribeKey", "Resource": "arn:aws:kms:*:*:key/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "SageMakerHubPermissions", "Effect": "Allow", "Action": [ "sagemaker:DescribeHubContent" ], "Resource": [ "arn:aws:sagemaker:*:*:hub/*", "arn:aws:sagemaker:*:*:hub-content/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "SageMakerModelPackagePermissions", "Effect": "Allow", "Action": [ "sagemaker:AccessModelPackage", "sagemaker:CreateModelPackage", "sagemaker:DescribeModelPackage", "sagemaker:DescribeModelPackageGroup" ], "Resource": [ "arn:aws:sagemaker:*:*:model-package/*", "arn:aws:sagemaker:*:*:model-package-group/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "MLflowPermissions", "Effect": "Allow", "Action": [ "sagemaker:DescribeMlflowApp", "sagemaker:CallMlflowAppApi", "sagemaker-mlflow:CreateExperiment", "sagemaker-mlflow:CreateRun", "sagemaker-mlflow:UpdateRun", "sagemaker-mlflow:LogBatch", "sagemaker-mlflow:GetExperimentByName", "sagemaker-mlflow:GetMetricHistory", "sagemaker-mlflow:GetRun", "sagemaker-mlflow:StartTrace", "sagemaker-mlflow:EndTrace", "sagemaker-mlflow:SearchTraces", "sagemaker-mlflow:ListArtifacts" ], "Resource": [ "arn:aws:sagemaker:*:*:mlflow-app/*" ], "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BedrockAgentCorePermissions", "Effect": "Allow", "Action": [ "bedrock-agentcore:InvokeAgentRuntime", "bedrock-agentcore:StopRuntimeSession", "bedrock-agentcore:GetAgentRuntime" ], "Resource": "arn:aws:bedrock-agentcore:*:*:runtime/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "EC2NetworkPermissions", "Effect": "Allow", "Action": [ "ec2:CreateNetworkInterface", "ec2:CreateNetworkInterfacePermission", "ec2:DescribeNetworkInterfaces", "ec2:DescribeVpcs", "ec2:DescribeSubnets", "ec2:DescribeSecurityGroups", "ec2:DescribeDhcpOptions" ], "Resource": "*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "EC2NetworkInterfaceTagPermissions", "Effect": "Allow", "Action": [ "ec2:CreateTags" ], "Resource": "arn:aws:ec2:*:*:network-interface/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}", "ec2:CreateAction": "CreateNetworkInterface" } } }, { "Sid": "EC2NetworkInterfaceDeletePermissions", "Effect": "Allow", "Action": [ "ec2:DeleteNetworkInterface", "ec2:DeleteNetworkInterfacePermission" ], "Resource": "*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "CloudWatchLogsPermissions", "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Resource": "arn:aws:logs:*:*:log-group:/aws/sagemaker/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "LambdaPermissions", "Effect": "Allow", "Action": [ "lambda:InvokeFunction" ], "Resource": "arn:aws:lambda:*:*:function:*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } } ] }
执行角色的信任关系:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "job.sagemaker.amazonaws.com" }, "Action": ["sts:AssumeRole", "sts:TagSession"] } ] }
AmazonSageMakerJobRuntimeAccess 等效 + BedrockAgentCoreFullAccess
{ "Version": "2012-10-17", "Statement": [ { "Sid": "SageMakerJobRuntimePermissions", "Effect": "Allow", "Action": [ "sagemaker:Sample", "sagemaker:SampleWithResponseStream", "sagemaker:CompleteRollout", "sagemaker:UpdateReward" ], "Resource": "arn:aws:sagemaker:*:*:job/*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } }, { "Sid": "BearerTokenPermissions", "Effect": "Allow", "Action": [ "sagemaker:CallWithBearerToken" ], "Resource": "*", "Condition": { "StringEquals": { "aws:ResourceAccount": "${aws:PrincipalAccount}" } } } ] }
代理运行时角色的信任关系:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "bedrock-agentcore.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
Lambda 转发器注意事项
现有的模型自定义 Lambda 权限涵盖名称中带有SageMaker或 sagemaker 的函数。如果您的 Lambda 转发器使用不同的命名约定,请明确添加其 ARN:
{ "Sid": "CustomAgentLambdaPermission", "Effect": "Allow", "Action": ["lambda:InvokeFunction"], "Resource": "arn:aws:lambda:*:*:function:your-agent-forwarder-function-name", "Condition": { "StringEquals": {"aws:ResourceAccount": "${aws:PrincipalAccount}"} } }
其他设置
-
如果您使用客户托管的 VPC,请参阅为多轮 RL 任务配置 VPC。
-
如果您使用 KMS 密钥加密任务输入和输出,则执行角色和调用者角色需要额外的权限。请参阅用于多回合强化学习的静态加密。