Understanding Amazon Rekognition Custom Labels

This section gives you an overview of the workflow to train and use an Amazon Rekognition Custom Labels model with the console and the AWS SDK.

Decide your model type

You first decide which type of model you want to train, which depends on your business goals. For example, you could train a model to find your logo in social media posts, identify your products on store shelves, or classify machine parts in an assembly line.

Amazon Rekognition Custom Labels can train the following types of model:

To help you decide which type of model to train, Amazon Rekognition Custom Labels provides example projects that you can use. For more information, see Getting started with Amazon Rekognition Custom Labels.

Find objects, scenes, and concepts

The model predicts classifications for the objects, scenes, and concepts associated with an entire image. For example, you can train a model that determines if an image contains a tourist attraction, or not. For an example project, see Image classification. The following image of a lake is an example of the kind of image you can recognize objects, scenes, and concepts in.

Alternatively, you can train a model that categorizes images into multiple categories. For example, the previous image might have categories such as sky color, reflection, or lake. For an example project, see Multi-label image classification.

Find object locations

The model predicts the location of an object on an image. The prediction includes bounding box information for the object location and a label that identifies the object within the bounding box. For example, the following image shows bounding boxes around various parts of a circuit board, such as a comparator or pot resistor.

The Object localization example project shows how Amazon Rekognition Custom Labels uses labeled bounding boxes to train a model that finds object locations.

Find the location of brands

Amazon Rekognition Custom Labels can train a model that finds the location of brands, such as logos, on an image. The prediction includes bounding box information for the brand location and a label that identifies the object within the bounding box. For an example project, see Brand detection. The following image is an example of some of the brands that the model can detect.

Create a model

The steps to create a model are creating a project, creating training and test datasets, and training the model.

Create a project

An Amazon Rekognition Custom Labels project is a group of resources needed to create and manage a model. A project manages the following:

We recommend that you use a project for a single use case, such as finding circuit board parts on a circuit board.

You can create a project with the Amazon Rekognition Custom Labels console and with the CreateProject API. For more information, see Creating a project.

Create training and test datasets

A dataset is a set of images and labels that describe those images. Within your project, you create a training dataset and a test dataset that Amazon Rekognition Custom Labels uses to train and test your model.

A label identifies an object, scene, concept, or bounding box around an object in an image. Labels are either assigned to an entire image (image-level) or they are assigned to a bounding box that surrounds an object on an image.

Images must be in PNG and JPEG format, and you should follow the input images recommendations. For more information, see Preparing images.

Create training and test datasets (Console)

You can start a project with a single dataset, or with separate training and test datasets. If you start with a single dataset, Amazon Rekognition Custom Labels splits your dataset during training to create a training dataset (80%) and a test dataset (20%) for your project. Start with a single dataset if you want Amazon Rekognition Custom Labels to decide which images are used for training and testing. For complete control over training, testing, and performance tuning, we recommend that you start your project with separate training and test datasets.

To create the datasets for a project, you import the images in one of the following ways:

For more information, see Creating training and test datasets with images.

Depending on where you import your images from, your images might be unlabeled. For example, images imported from a local computer aren't labeled. Images imported from an Amazon SageMaker Ground Truth manifest file are labeled. You can use the Amazon Rekognition Custom Labels console to add, change, and assign labels. For more information, see Labeling images.

To create your training and test datasets with the console, see Creating training and test datasets with images. For a tutorial that includes creating training and test datasets, see Classifying images.

Create training and test datasets (SDK)

To create your training and test datasets, you use the CreateDataset API. You can create a dataset by using an Amazon Sagemaker format manifest file or by copying an existing Amazon Rekognition Custom Labels dataset. For more information, see Create training and test datasets (SDK) If necessary, you can create your own manifest file. For more information, see Creating a manifest file.

Train your model

Train your model with the training dataset. A new version of a model is created each time it is trained. During training, Amazon Rekognition Custom Labels test the performance of your trained model. You can use the results to evaluate and improve your model. Training takes a while to complete. You are only charged for a successful model training. For more information, see Training an Amazon Rekognition Custom Labels model. If model training fails, Amazon Rekognition Custom Labels provides debugging information that you can use. For more information, see Debugging a failed model training.

Train your model (Console)

To train your model with the console, see Training a model (Console).

Training a model (SDK)

You train an Amazon Rekognition Custom Labels model by calling CreateProjectVersion. For more information, see Training a model (SDK).

Improve your model

During testing, Amazon Rekognition Custom Labels creates evaluation metrics that you can use to improve your trained model.

Evaluate your model

Evaluate the performance of your model by using the performance metrics created during testing. Performance metrics, such as F1, precision, and recall, allow you to understand the performance of your trained model, and decide if you're ready to use it in production. For more information, see Metrics for evaluating your model.

Evaluate a model (console)

To view performance metrics, see Accessing evaluation metrics (Console).

Evaluate a model (SDK)

To get performance metrics, you call DescribeProjectVersions to get the testing results. For more information, see Accessing Amazon Rekognition Custom Labels evaluation metrics (SDK). The testing results include metrics not available in the console, such as a confusion matrix for classification results. The testing results are returned in the following formats:

Improve your model

If improvements are needed, you can add more training images or improve dataset labeling. For more information, see Improving an Amazon Rekognition Custom Labels model. You can also give feedback on the predictions your model makes and use it to make improvements to your model. For more information, see Improving a model with Model feedback.

Improve your model (console)

To add images to a dataset, see Adding more images to a dataset. To add or change labels, see Labeling images.

To retrain your model, see Training a model (Console).

Improve your model (SDK)

To add images to a dataset or change the labeling for an image, use the UpdateDatasetEntries API. UpdateDatasetEntries updates or adds JSON lines to a manifest file. Each JSON line contains information for a single image, such as assigned labels or bounding box information. For more information, see Adding more images (SDK). To view the entries in a dataset, use the ListDatasetEntries API.

To retrain your model, see Training a model (SDK).

Start your model

Before you can use your model, you start the model by using the Amazon Rekognition Custom Labels console or the StartProjectVersion API. You are charged for the amount of time that your model runs. For more information, see Running a trained Amazon Rekognition Custom Labels model.

Start your model (console)

To start your model using the console, see Starting an Amazon Rekognition Custom Labels model (Console).

Start your model

You start your model calling StartProjectVersion. For more information, see Starting an Amazon Rekognition Custom Labels model (SDK).

Analyze an image

To analyze an image with your model, you use the DetectCustomLabels API. You can specify a local image, or an image stored in an S3 bucket. The operation also requires the Amazon Resource Name (ARN) of the model that you want to use.

If your model finds objects, scenes, and concepts, the response includes a list of image-level labels found in the image. For example, the following image shows the image-level labels found using Rooms example project.

If the model finds object locations, the response includes list of labeled bounding boxes found in the image. A bounding box represents the location of an object on an image. You can use the bounding box information to draw a bounding box around an object. For example, the following image shows bounding boxes around circuit board parts found using the Circuit boards example project.

For more information, see Analyzing an image with a trained model.

Stop your model

You are charged for the time that your model is running. If you are no longer using your model, stop the model by using the Amazon Rekognition Custom Labels console, or by using the StopProjectVersion API. For more information, see Stopping an Amazon Rekognition Custom Labels model.

Stop your model (Console)

To stop a running model with the console, see Stopping an Amazon Rekognition Custom Labels model (Console).

Stop your model (SDK)

To stop a running model, call StopProjectVersion. For more information, see Stopping an Amazon Rekognition Custom Labels model (SDK).