DetectLabels

Detects instances of real-world entities within an image (JPEG or PNG) provided as input. This includes objects like flower, tree, and table; events like wedding, graduation, and birthday party; and concepts like landscape, evening, and nature.

For an example, see Analyzing images stored in an Amazon S3 bucket.

You pass the input image as base64-encoded image bytes or as a reference to an image in an Amazon S3 bucket. If you use the AWS CLI to call Amazon Rekognition operations, passing image bytes is not supported. The image must be either a PNG or JPEG formatted file.

Optional Parameters

You can specify one or both of the GENERAL_LABELS and IMAGE_PROPERTIES feature types when calling the DetectLabels API. Including GENERAL_LABELS will ensure the response includes the labels detected in the input image, while including IMAGE_PROPERTIES will ensure the response includes information about the image quality and color.

When using GENERAL_LABELS and/or IMAGE_PROPERTIES you can provide filtering criteria to the Settings parameter. You can filter with sets of individual labels or with label categories. You can specify inclusive filters, exclusive filters, or a combination of inclusive and exclusive filters. For more information on filtering see Detecting Labels in an Image.

When getting labels, you can specify MinConfidence to control the confidence threshold for the labels returned. The default is 55%. You can also add the MaxLabels parameter to limit the number of labels returned. The default and upper limit is 1000 labels. These arguments are only valid when supplying GENERAL_LABELS as a feature type.

Response Elements

For each object, scene, and concept the API returns one or more labels. The API returns the following types of information about labels:

The API returns the following information regarding the image, as part of the ImageProperties structure:

The list of returned labels will include at least one label for every detected object, along with information about that label. In the following example, suppose the input image has a lighthouse, the sea, and a rock. The response includes all three labels, one for each object, as well as the confidence in the label:

{Name: lighthouse, Confidence: 98.4629}

{Name: rock,Confidence: 79.2097}

{Name: sea,Confidence: 75.061}

The list of labels can include multiple labels for the same object. For example, if the input image shows a flower (for example, a tulip), the operation might return the following three labels.

{Name: flower,Confidence: 99.0562}

{Name: plant,Confidence: 99.0562}

{Name: tulip,Confidence: 99.0562}

In this example, the detection algorithm more precisely identifies the flower as a tulip.

This is a stateless API operation that doesn't return any data.

This operation requires permissions to perform the rekognition:DetectLabels action.

Request Syntax

{
   "Features": [ "string" ],
   "Image": { 
      "Bytes": blob,
      "S3Object": { 
         "Bucket": "string",
         "Name": "string",
         "Version": "string"
      }
   },
   "MaxLabels": number,
   "MinConfidence": number,
   "Settings": { 
      "GeneralLabels": { 
         "LabelCategoryExclusionFilters": [ "string" ],
         "LabelCategoryInclusionFilters": [ "string" ],
         "LabelExclusionFilters": [ "string" ],
         "LabelInclusionFilters": [ "string" ]
      },
      "ImageProperties": { 
         "MaxDominantColors": number
      }
   }
}

Request Parameters

For information about the parameters that are common to all actions, see Common Parameters.

The request accepts the following data in JSON format.

Response Syntax

{
   "ImageProperties": { 
      "Background": { 
         "DominantColors": [ 
            { 
               "Blue": number,
               "CSSColor": "string",
               "Green": number,
               "HexCode": "string",
               "PixelPercent": number,
               "Red": number,
               "SimplifiedColor": "string"
            }
         ],
         "Quality": { 
            "Brightness": number,
            "Contrast": number,
            "Sharpness": number
         }
      },
      "DominantColors": [ 
         { 
            "Blue": number,
            "CSSColor": "string",
            "Green": number,
            "HexCode": "string",
            "PixelPercent": number,
            "Red": number,
            "SimplifiedColor": "string"
         }
      ],
      "Foreground": { 
         "DominantColors": [ 
            { 
               "Blue": number,
               "CSSColor": "string",
               "Green": number,
               "HexCode": "string",
               "PixelPercent": number,
               "Red": number,
               "SimplifiedColor": "string"
            }
         ],
         "Quality": { 
            "Brightness": number,
            "Contrast": number,
            "Sharpness": number
         }
      },
      "Quality": { 
         "Brightness": number,
         "Contrast": number,
         "Sharpness": number
      }
   },
   "LabelModelVersion": "string",
   "Labels": [ 
      { 
         "Aliases": [ 
            { 
               "Name": "string"
            }
         ],
         "Categories": [ 
            { 
               "Name": "string"
            }
         ],
         "Confidence": number,
         "Instances": [ 
            { 
               "BoundingBox": { 
                  "Height": number,
                  "Left": number,
                  "Top": number,
                  "Width": number
               },
               "Confidence": number,
               "DominantColors": [ 
                  { 
                     "Blue": number,
                     "CSSColor": "string",
                     "Green": number,
                     "HexCode": "string",
                     "PixelPercent": number,
                     "Red": number,
                     "SimplifiedColor": "string"
                  }
               ]
            }
         ],
         "Name": "string",
         "Parents": [ 
            { 
               "Name": "string"
            }
         ]
      }
   ],
   "OrientationCorrection": "string"
}

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

Errors

For information about the errors that are common to all actions, see Common Errors.

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following: