Detecting labels in an image

To detect labels in an image

1. If you haven't already:

   1. Create or update a user with AmazonRekognitionFullAccess and AmazonS3ReadOnlyAccess permissions. For more information, see Step 1: Set up an AWS account and create a User.

   2. Install and configure the AWS CLI and the AWS SDKs. For more information, see Step 2: Set up the AWS CLI and AWS SDKs.

2. Upload an image that contains one or more objects—such as trees, houses, and boat—to your S3 bucket. The image must be in .jpg or .png format.

   For instructions, see Uploading Objects into Amazon S3 in the Amazon Simple Storage Service User Guide.

3. Use the following examples to call the DetectLabels operation.

   Java

   This example displays a list of labels that were detected in the input image. Replace the values of bucket and photo with the names of the Amazon S3 bucket and image that you used in step 2.

   
   package com.amazonaws.samples;
   import java.util.List;
   
   import com.amazonaws.services.rekognition.model.BoundingBox;
   import com.amazonaws.services.rekognition.model.DetectLabelsRequest;
   import com.amazonaws.services.rekognition.model.DetectLabelsResult;
   import com.amazonaws.services.rekognition.model.Image;
   import com.amazonaws.services.rekognition.model.Instance;
   import com.amazonaws.services.rekognition.model.Label;
   import com.amazonaws.services.rekognition.model.Parent;
   import com.amazonaws.services.rekognition.model.S3Object;
   import com.amazonaws.services.rekognition.AmazonRekognition;
   import com.amazonaws.services.rekognition.AmazonRekognitionClientBuilder;
   import com.amazonaws.services.rekognition.model.AmazonRekognitionException;
   
   public class DetectLabels {
   
       public static void main(String[] args) throws Exception {
   
           String photo = "photo";
           String bucket = "bucket";
   
           AmazonRekognition rekognitionClient = AmazonRekognitionClientBuilder.defaultClient();
   
           DetectLabelsRequest request = new DetectLabelsRequest()
                   .withImage(new Image().withS3Object(new S3Object().withName(photo).withBucket(bucket)))
                   .withMaxLabels(10).withMinConfidence(75F);
   
           try {
               DetectLabelsResult result = rekognitionClient.detectLabels(request);
               List<Label> labels = result.getLabels();
   
               System.out.println("Detected labels for " + photo + "\n");
               for (Label label : labels) {
                   System.out.println("Label: " + label.getName());
                   System.out.println("Confidence: " + label.getConfidence().toString() + "\n");
   
                   List<Instance> instances = label.getInstances();
                   System.out.println("Instances of " + label.getName());
                   if (instances.isEmpty()) {
                       System.out.println("  " + "None");
                   } else {
                       for (Instance instance : instances) {
                           System.out.println("  Confidence: " + instance.getConfidence().toString());
                           System.out.println("  Bounding box: " + instance.getBoundingBox().toString());
                       }
                   }
                   System.out.println("Parent labels for " + label.getName() + ":");
                   List<Parent> parents = label.getParents();
                   if (parents.isEmpty()) {
                       System.out.println("  None");
                   } else {
                       for (Parent parent : parents) {
                           System.out.println("  " + parent.getName());
                       }
                   }
                   System.out.println("--------------------");
                   System.out.println();
                  
               }
           } catch (AmazonRekognitionException e) {
               e.printStackTrace();
           }
       }
   }

   AWS CLI

   This example displays the JSON output from the detect-labels CLI operation. Replace the values of bucket and photo with the names of the Amazon S3 bucket and image that you used in Step 2. Replace the value of profile-name with the name of your developer profile.

   aws rekognition detect-labels --image '{ "S3Object": { "Bucket": "bucket-name", "Name": "file-name" } }' \
   --features GENERAL_LABELS IMAGE_PROPERTIES \
   --settings '{"ImageProperties": {"MaxDominantColors":1}, {"GeneralLabels":{"LabelInclusionFilters":["Cat"]}}}' \
   --profile profile-name \
   --region us-east-1

   If you are accessing the CLI on a Windows device, use double quotes instead of single quotes and escape the inner double quotes by backslash (i.e. \) to address any parser errors you may encounter. For an example, see the following:

   
   aws rekognition detect-labels --image "{\"S3Object\":{\"Bucket\":\"bucket-name\",\"Name\":\"file-name\"}}" --features GENERAL_LABELS IMAGE_PROPERTIES \
   --settings "{\"GeneralLabels\":{\"LabelInclusionFilters\":[\"Car\"]}}" --profile profile-name --region us-east-1
                               

   Python

   This example displays the labels that were detected in the input image. In the function main, replace the values of bucket and photo with the names of the Amazon S3 bucket and image that you used in Step 2. Replace the value of profile_name in the line that creates the Rekognition session with the name of your developer profile.

   #Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
   #PDX-License-Identifier: MIT-0 (For details, see https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/LICENSE-SAMPLECODE.)
   
   import boto3
   
   def detect_labels(photo, bucket):
   
        session = boto3.Session(profile_name='profile-name')
        client = session.client('rekognition')
   
        response = client.detect_labels(Image={'S3Object':{'Bucket':bucket,'Name':photo}},
        MaxLabels=10,
        # Uncomment to use image properties and filtration settings
        #Features=["GENERAL_LABELS", "IMAGE_PROPERTIES"],
        #Settings={"GeneralLabels": {"LabelInclusionFilters":["Cat"]},
        # "ImageProperties": {"MaxDominantColors":10}}
        )
   
        print('Detected labels for ' + photo)
        print()
        for label in response['Labels']:
            print("Label: " + label['Name'])
            print("Confidence: " + str(label['Confidence']))
            print("Instances:")
   
            for instance in label['Instances']:
                print(" Bounding box")
                print(" Top: " + str(instance['BoundingBox']['Top']))
                print(" Left: " + str(instance['BoundingBox']['Left']))
                print(" Width: " + str(instance['BoundingBox']['Width']))
                print(" Height: " + str(instance['BoundingBox']['Height']))
                print(" Confidence: " + str(instance['Confidence']))
                print()
   
            print("Parents:")
            for parent in label['Parents']:
               print(" " + parent['Name'])
   
            print("Aliases:")
            for alias in label['Aliases']:
                print(" " + alias['Name'])
   
                print("Categories:")
            for category in label['Categories']:
                print(" " + category['Name'])
                print("----------")
                print()
   
        if "ImageProperties" in str(response):
            print("Background:")
            print(response["ImageProperties"]["Background"])
            print()
            print("Foreground:")
            print(response["ImageProperties"]["Foreground"])
            print()
            print("Quality:")
            print(response["ImageProperties"]["Quality"])
            print()
   
        return len(response['Labels'])
   
   def main():
       photo = 'photo-name'
       bucket = 'bucket-name'
       label_count = detect_labels(photo, bucket)
       print("Labels detected: " + str(label_count))
   
   if __name__ == "__main__":
       main()
   

   .NET

   This example displays a list of labels that were detected in the input image. Replace the values of bucket and photo with the names of the Amazon S3 bucket and image that you used in Step 2.

   //Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
   //PDX-License-Identifier: MIT-0 (For details, see https://github.com/awsdocs/amazon-rekognition-developer-guide/blob/master/LICENSE-SAMPLECODE.)
   
   using System;
   using Amazon.Rekognition;
   using Amazon.Rekognition.Model;
   
   public class DetectLabels
   {
       public static void Example()
       {
           String photo = "input.jpg";
           String bucket = "bucket";
   
           AmazonRekognitionClient rekognitionClient = new AmazonRekognitionClient();
   
           DetectLabelsRequest detectlabelsRequest = new DetectLabelsRequest()
           {
               Image = new Image()
               {
                   S3Object = new S3Object()
                   {
                       Name = photo,
                       Bucket = bucket
                   },
               },
               MaxLabels = 10,
               MinConfidence = 75F
           };
   
           try
           {
               DetectLabelsResponse detectLabelsResponse = rekognitionClient.DetectLabels(detectlabelsRequest);
               Console.WriteLine("Detected labels for " + photo);
               foreach (Label label in detectLabelsResponse.Labels)
                   Console.WriteLine("{0}: {1}", label.Name, label.Confidence);
           }
           catch (Exception e)
           {
               Console.WriteLine(e.Message);
           }
       }
   }

   Ruby

   This example displays a list of labels that were detected in the input image. Replace the values of bucket and photo with the names of the Amazon S3 bucket and image that you used in Step 2.

   
   
      # Add to your Gemfile
      # gem 'aws-sdk-rekognition'
      require 'aws-sdk-rekognition'
      credentials = Aws::Credentials.new(
         ENV['AWS_ACCESS_KEY_ID'],
         ENV['AWS_SECRET_ACCESS_KEY']
      )
      bucket = 'bucket' # the bucket name without s3://
      photo  = 'photo' # the name of file
      client   = Aws::Rekognition::Client.new credentials: credentials
      attrs = {
        image: {
          s3_object: {
            bucket: bucket,
            name: photo
          },
        },
        max_labels: 10
      }
     response = client.detect_labels attrs
     puts "Detected labels for: #{photo}"
     response.labels.each do |label|
       puts "Label:      #{label.name}"
       puts "Confidence: #{label.confidence}"
       puts "Instances:"
       label['instances'].each do |instance|
         box = instance['bounding_box']
         puts "  Bounding box:"
         puts "    Top:        #{box.top}"
         puts "    Left:       #{box.left}"
         puts "    Width:      #{box.width}"
         puts "    Height:     #{box.height}"
         puts "  Confidence: #{instance.confidence}"
       end
       puts "Parents:"
       label.parents.each do |parent|
         puts "  #{parent.name}"
       end
       puts "------------"
       puts ""
     end

   Node.js

   This example displays a list of labels that were detected in the input image. Replace the values of bucket and photo with the names of the Amazon S3 bucket and image that you used in Step 2. Replace the value of profile_name in the line that creates the Rekognition session with the name of your developer profile.

   If you are using TypeScript definitions, you may need to use import AWS from 'aws-sdk' instead of const AWS = require('aws-sdk'), in order to run the program with Node.js. You can consult the AWS SDK for Javascript for more details. Depending on how you have your configurations set up, you also may need to specify your region with AWS.config.update({region:region});.

   
                                   
   // Load the SDK
   var AWS = require('aws-sdk');
   const bucket = 'bucket-name' // the bucketname without s3://
   const photo  = 'image-name' // the name of file
   
   var credentials = new AWS.SharedIniFileCredentials({profile: 'profile-name'});
   AWS.config.credentials = credentials;
   AWS.config.update({region:'region-name'});
   
   const client = new AWS.Rekognition();
   const params = {
     Image: {
       S3Object: {
         Bucket: bucket,
         Name: photo
       },
     },
     MaxLabels: 10
   }
   client.detectLabels(params, function(err, response) {
     if (err) {
       console.log(err, err.stack); // if an error occurred
     } else {
       console.log(`Detected labels for: ${photo}`)
       response.Labels.forEach(label => {
         console.log(`Label:      ${label.Name}`)
         console.log(`Confidence: ${label.Confidence}`)
         console.log("Instances:")
         label.Instances.forEach(instance => {
           let box = instance.BoundingBox
           console.log("  Bounding box:")
           console.log(`    Top:        ${box.Top}`)
           console.log(`    Left:       ${box.Left}`)
           console.log(`    Width:      ${box.Width}`)
           console.log(`    Height:     ${box.Height}`)
           console.log(`  Confidence: ${instance.Confidence}`)
         })
         console.log("Parents:")
         label.Parents.forEach(parent => {
           console.log(`  ${parent.Name}`)
         })
         console.log("------------")
         console.log("")
       }) // for response.labels
     } // if
   });
                               
                               

   Java V2

   This code is taken from the AWS Documentation SDK examples GitHub repository. See the full example here.

   //snippet-start:[rekognition.java2.detect_labels.import]
   import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
   import software.amazon.awssdk.regions.Region;
   import software.amazon.awssdk.services.rekognition.RekognitionClient;
   import software.amazon.awssdk.services.rekognition.model.Image;
   import software.amazon.awssdk.services.rekognition.model.DetectLabelsRequest;
   import software.amazon.awssdk.services.rekognition.model.DetectLabelsResponse;
   import software.amazon.awssdk.services.rekognition.model.Label;
   import software.amazon.awssdk.services.rekognition.model.RekognitionException;
   import software.amazon.awssdk.services.rekognition.model.S3Object;
   import java.util.List;
   
   /**
   * Before running this Java V2 code example, set up your development environment, including your credentials.
   *
   * For more information, see the following documentation topic:
   *
   * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
   */
   public class DetectLabels {
   
       public static void main(String[] args) {
   
           final String usage = "\n" +
               "Usage: " +
               "   <bucket> <image>\n\n" +
               "Where:\n" +
               "   bucket - The name of the Amazon S3 bucket that contains the image (for example, ,ImageBucket)." +
               "   image - The name of the image located in the Amazon S3 bucket (for example, Lake.png). \n\n";
   
           if (args.length != 2) {
               System.out.println(usage);
               System.exit(1);
           }
   
           String bucket = args[0];
           String image = args[1];
           Region region = Region.US_WEST_2;
           RekognitionClient rekClient = RekognitionClient.builder()
               .region(region)
               .credentialsProvider(ProfileCredentialsProvider.create("profile-name"))
               .build();
   
           getLabelsfromImage(rekClient, bucket, image);
           rekClient.close();
       }
   
       // snippet-start:[rekognition.java2.detect_labels_s3.main]
       public static void getLabelsfromImage(RekognitionClient rekClient, String bucket, String image) {
   
           try {
               S3Object s3Object = S3Object.builder()
                   .bucket(bucket)
                   .name(image)
                   .build() ;
   
               Image myImage = Image.builder()
                   .s3Object(s3Object)
                   .build();
   
               DetectLabelsRequest detectLabelsRequest = DetectLabelsRequest.builder()
                   .image(myImage)
                   .maxLabels(10)
                   .build();
   
               DetectLabelsResponse labelsResponse = rekClient.detectLabels(detectLabelsRequest);
               List<Label> labels = labelsResponse.labels();
               System.out.println("Detected labels for the given photo");
               for (Label label: labels) {
                   System.out.println(label.name() + ": " + label.confidence().toString());
               }
   
           } catch (RekognitionException e) {
               System.out.println(e.getMessage());
               System.exit(1);
           }
       }
    // snippet-end:[rekognition.java2.detect_labels.main]
   }