Prerequisites for inference profiles - Amazon Bedrock

Prerequisites for inference profiles

Before you can use an inference profile, check that you've fulfilled the following prerequisites:

  1. Your role has access to the inference profile API actions. If your role has the AmazonBedrockFullAccess AWS-managed policy attached, you can skip this step. Otherwise, do the following:

    1. Follow the steps at Creating IAM policies and create the following policy, which allows a role to do inference profile-related actions and run model inference using all foundation models and inference profiles.

      {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel*",
                "bedrock:CreateInferenceProfile"
            ],
            "Resource": [
                "arn:aws:bedrock:*::foundation-model/*",
                "arn:aws:bedrock:*:*:inference-profile/*",
                "arn:aws:bedrock:*:*:application-inference-profile/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:GetInferenceProfile",
                "bedrock:ListInferenceProfiles",
                "bedrock:DeleteInferenceProfile",
                "bedrock:TagResource",
                "bedrock:UntagResource",
                "bedrock:ListTagsForResource"
            ],
            "Resource": [
                "arn:aws:bedrock:*:*:inference-profile/*",
                "arn:aws:bedrock:*:*:application-inference-profile/*"
            ]
        }
    ]
}

      (Optional) You can restrict the role's access in the following ways:

      • To restrict the API actions that the role can make, modify the list in the Action field to contain only the API operations that you want to allow access to.

      • To restrict the role's access to specific inference profiles, modify the Resource list to contain only the inference profiles and foundation models that you want to allow access to. System-defined inference profiles begin with inference-profile and application inference profiles begin with application-inference-profile.

        Important

        When you specify an inference profile in the Resource field in the first statement, you must also specify the foundation model in each Region associated with it.

      • To restrict user access such that they can invoke a foundation model only through an inference profile, add a Condition field and use the aws:InferenceProfileArn condition key. Specify the inference profile that you want to filter access on.

      • For example, you can attach the following policy to a role to allow it to invoke the Anthropic Claude 3 Haiku model only through the US Anthropic Claude 3 Haiku inference profile in the account 123456789012 in us-west-2:

        {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel*"
            ],
            "Resource": [
                "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
                "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
                "arn:aws:bedrock:us-west-2:123456789012:inference-profile/us.anthropic.claude-3-haiku-20240307-v1:0"
            ],
            "Condition": {
                "StringLike": {
                    "bedrock:InferenceProfileArn": "arn:aws:bedrock:us-west-2:123456789012:inference-profile/us.anthropic.claude-3-haiku-20240307-v1:0"
                }
            }
        }
    ]
}

    2. Follow the steps at Adding and removing IAM identity permissions to attach the policy to a role to grant the role permissions to view and use all the inference profiles.

  2. You've requested access to the models and the regions defined in the inference profiles that you want to use. For example, to gain access to make calls to the US Anthropic Claude 3 Haiku inference profile from the US West (Oregon) Region, do the following:

    1. Sign into the AWS Management Console in the US East (N. Virginia) Region and request model access to Anthropic Claude 3 Haiku by following the steps at Access Amazon Bedrock foundation models.

    2. Change to the US West (Oregon) Region and request model access to Anthropic Claude 3 Haiku.