Before you can use an inference profile, check that you've fulfilled the following prerequisites:
-
Your role has access to the inference profile API actions. If your role has the AmazonBedrockFullAccess AWS-managed policy attached, you can skip this step. Otherwise, do the following:
-
Follow the steps at Creating IAM policies and create the following policy, which allows a role to do inference profile-related actions and run model inference using all foundation models and inference profiles.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:InvokeModel*", "bedrock:CreateInferenceProfile" ], "Resource": [ "arn:aws:bedrock:*::foundation-model/*", "arn:aws:bedrock:*:*:inference-profile/*", "arn:aws:bedrock:*:*:application-inference-profile/*" ] }, { "Effect": "Allow", "Action": [ "bedrock:GetInferenceProfile", "bedrock:ListInferenceProfiles", "bedrock:DeleteInferenceProfile", "bedrock:TagResource", "bedrock:UntagResource", "bedrock:ListTagsForResource" ], "Resource": [ "arn:aws:bedrock:*:*:inference-profile/*", "arn:aws:bedrock:*:*:application-inference-profile/*" ] } ] }(Optional) You can restrict the role's access in the following ways:
-
To restrict the API actions that the role can make, modify the list in the
Actionfield to contain only the API operations that you want to allow access to. -
To restrict the role's access to specific inference profiles, modify the
Resourcelist to contain only the inference profiles and foundation models that you want to allow access to. System-defined inference profiles begin withinference-profileand application inference profiles begin withapplication-inference-profile.Important
When you specify an inference profile in the
Resourcefield in the first statement, you must also specify the foundation model in each Region associated with it. -
To restrict user access such that they can invoke a foundation model only through an inference profile, add a
Conditionfield and use theaws:InferenceProfileArncondition key. Specify the inference profile that you want to filter access on. This condition can be included in a statement that scopes to thefoundation-modelresources. -
For example, you can attach the following policy to a role to allow it to invoke the Anthropic Claude 3 Haiku model only through the US Anthropic Claude 3 Haiku inference profile in the account
111122223333in us-west-2:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "bedrock:InvokeModel*" ], "Resource": [ "arn:aws:bedrock:us-west-2:111122223333:inference-profile/us.anthropic.claude-3-haiku-20240307-v1:0" ] }, { "Effect": "Allow", "Action": [ "bedrock:InvokeModel*" ], "Resource": [ "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0" "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-haiku-20240307-v1:0" ], "Condition": { "StringLike": { "bedrock:InferenceProfileArn": "arn:aws:bedrock:us-west-2:111122223333:inference-profile/us.anthropic.claude-3-haiku-20240307-v1:0" } } } ] }
-
-
Follow the steps at Adding and removing IAM identity permissions to attach the policy to a role to grant the role permissions to view and use all the inference profiles.
-
-
You've requested access to the model defined in the inference profile that you want to use, in the Region from which you want to call the inference profile.
