Run an Amazon SageMaker Ground Truth Labeling Job in an Amazon Virtual Private Cloud

Ground Truth supports the following functionalities in Amazon VPC.

You may want to use these options in any of the following ways.

We recommend that you review Prerequisites for running a Ground Truth labeling job in a VPC before you create a labeling job in an Amazon VPC.

Prerequisites for running a Ground Truth labeling job in a VPC

Review the following prerequisites before you create a Ground Truth labeling job in an Amazon VPC.

Allow Ground Truth to Access VPC Restricted Amazon S3 Buckets

The following sections provide details about the permissions Ground Truth requires to launch labeling jobs using Amazon S3 buckets that have access restricted to your VPC and VPC endpoints. To learn how to restrict access to an Amazon S3 bucket to a VPC, see Controlling access from VPC endpoints with bucket policies in the Amazon Simple Storage Service User Guide guide. To learn how to add a policy to an S3 bucket, see Adding a bucket policy using the Amazon S3 console.

You can restrict Amazon S3 bucket access to users in your VPC using an AWS PrivateLink endpoint. For example, the following S3 bucket policy allows access to a specific bucket, <bucket-name>, from <vpc> and the endpoint <vpc-endpoint> only. When you modify this policy, you must replace all red-italized text with your resources and specifications.

{
    "Version": "2012-10-17",
    "Id": "Policy1415115909152",
    "Statement": [
        {
            "Sid": "Access-to-specific-VPCE-only",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Effect": "Deny",
            "Resource": [
                "arn:aws:s3:::<bucket-name>",
                "arn:aws:s3:::<bucket-name>/*"
            ],
            "Condition": {
                "StringNotEquals": {
                    "aws:sourceVpce": [
                        "<vpc-endpoint>",
                        "<vpc>"
                    ]
                }
            }
        }
    ]
}

Ground Truth must be able to perform the following Amazon S3 actions on the S3 buckets you use to configure the labeling job.

"s3:AbortMultipartUpload",
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket",
"s3:GetBucketLocation"

You can do this by adding a Ground Truth endpoint to the bucket policy like the one previously mentioned. The following table includes Ground Truth service endpoints for each AWS Region. Add an endpoint in the same AWS Region you use to run your labeling job to your bucket policy.

For example, the following policy restricts GetObject and PutObject actions on:

{
    "Version": "2012-10-17",
    "Id": "1",
    "Statement": [
        {
            "Sid": "DenyAccessFromNonGTandCustomerVPC",
            "Effect": "Deny",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket-name>",
                "arn:aws:s3:::<bucket-name>/*"
            ],
            "Condition": {
                "ForAllValues:StringNotEquals": {
                    "aws:sourceVpce": [
                        "<vpc-endpoint>",
                        "<ground-truth-endpoint>"
                    ],
                    "aws:SourceVpc": "<vpc>"
                }
            }
        }
    ]
}

If you want a user to have permission to launch a labeling job using the Ground Truth console, you must also add the user's ARN to the bucket policy using the aws:PrincipalArn condition. This user must also have permission to perform the following Amazon S3 actions on the bucket you use to launch the labeling job.

"s3:GetObject",
"s3:PutObject",
"s3:ListBucket",
"s3:GetBucketCors",
"s3:PutBucketCors",
"s3:ListAllMyBuckets",

The following code is an example of a bucket policy that restricts permission to perform the actions listed in Action on the S3 bucket <bucket-name> to the following.

{
    "Version": "2012-10-17",
    "Id": "1",
    "Statement": [
        {
            "Sid": "DenyAccessFromNonGTandCustomerVPC",
            "Effect": "Deny",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::<bucket-name>/*",
                "arn:aws:s3:::<bucket-name>"
            ],
            "Condition": {
                "ForAllValues:StringNotEquals": {
                    "aws:sourceVpce": [
                        "<vpc-endpoint>",
                        "<ground-truth-endpoint>"
                    ],
                    "aws:PrincipalArn": "arn:aws:iam::<aws-account-id>:role/<role-name>",
                    "aws:SourceVpc": "<vpc>"
                }
            }
        }
    ]
}

After you have granted Ground Truth permission to access your Amazon S3 buckets, you can use one of the topics in Create a Labeling Job to launch a labeling job. Specify the VPC-restricted Amazon S3 buckets for your input and output data buckets.

Create an Automated Data Labeling Job in a VPC

To create an automated data labeling job using an Amazon VPC, you provide a VPC configuration using the Ground Truth console or CreateLabelingJob API operation. SageMaker AI uses the subnets and security groups you provide to launch the training and inferences jobs used for automated labeling.

Use the following procedures to learn how to add a VPC configuration to your labeling job request.

Add a VPC configuration to an automated data labeling job (API):

To configure a labeling job using the Ground Truth API operation, CreateLabelingJob, follow the instructions in Create an Automated Data Labeling Job (API) to configure your request. In addition to the parameters described in this documentation, you must include a VpcConfig parameter in LabelingJobResourceConfig to specify one or more subnets and security groups using the following schema.

"LabelingJobAlgorithmsConfig": { 
      "InitialActiveLearningModelArn": "string",
      "LabelingJobAlgorithmSpecificationArn": "string",
      "LabelingJobResourceConfig": { 
         "VolumeKmsKeyId": "string",
         "VpcConfig": { 
            "SecurityGroupIds": [ "string" ],
            "Subnets": [ "string" ]
         }
      }
}

The following is an example of an AWS Python SDK (Boto3) request to create an automated data labeling job in the US East (N. Virginia) Region using a private workforce. Replace all red-italicized text with your labeling job resources and specifications. To learn more about the CreateLabelingJob operation, see the Create a Labeling Job (API) tutorial and CreateLabelingJob API documentation.

import boto3
client = boto3.client(service_name='sagemaker')

response = client.create_labeling_job(
    LabelingJobName="example-labeling-job",
    LabelAttributeName="label",
    InputConfig={
        'DataSource': {
            'S3DataSource': {
                'ManifestS3Uri': "s3://bucket/path/manifest-with-input-data.json"
            }
        }
    },
    "LabelingJobAlgorithmsConfig": {
      "LabelingJobAlgorithmSpecificationArn": "arn:aws:sagemaker:us-east-1:027400017018:labeling-job-algorithm-specification/tasktype",
      "LabelingJobResourceConfig": { 
         "VpcConfig": { 
            "SecurityGroupIds": [ "sg-01233456789", "sg-987654321" ],
            "Subnets": [ "subnet-e0123456", "subnet-e7891011" ]
         }
      }
    },
    OutputConfig={
        'S3OutputPath': "s3://bucket/path/file-to-store-output-data",
        'KmsKeyId': "string"
    },
    RoleArn="arn:aws:iam::*:role/*,
    LabelCategoryConfigS3Uri="s3://bucket/path/label-categories.json",
    StoppingConditions={
        'MaxHumanLabeledObjectCount': 123,
        'MaxPercentageOfInputDatasetLabeled': 123
    },
    HumanTaskConfig={
        'WorkteamArn': "arn:aws:sagemaker:region:*:workteam/private-crowd/*",
        'UiConfig': {
            'UiTemplateS3Uri': "s3://bucket/path/custom-worker-task-template.html"
        },
        'PreHumanTaskLambdaArn': "arn:aws:lambda:us-east-1:432418664414:function:PRE-tasktype",
        'TaskKeywords': [
            "Images",
            "Classification",
            "Multi-label"
        ],
        'TaskTitle': "Add task title here",
        'TaskDescription': "Add description of task here for workers",
        'NumberOfHumanWorkersPerDataObject': 1,
        'TaskTimeLimitInSeconds': 3600,
        'TaskAvailabilityLifetimeInSeconds': 21600,
        'MaxConcurrentTaskCount': 1000,
        'AnnotationConsolidationConfig': {
            'AnnotationConsolidationLambdaArn': "arn:aws:lambda:us-east-1:432418664414:function:ACS-tasktype"
        },
    Tags=[
        {
            'Key': "string",
            'Value': "string"
        },
    ]
)