Tutorial: Perform sample image classification inference using TensorFlow Lite - AWS IoT Greengrass
PrerequisitesStep 1: Subscribe to the default notifications topicStep 2: Deploy the TensorFlow Lite image classification componentStep 3: View inference resultsNext steps

Tutorial: Perform sample image classification inference using TensorFlow Lite

This tutorial shows you how to use the TensorFlow Lite image classification inference component to perform sample image classification inference on a Greengrass core device. This component includes the following component dependencies:

  • TensorFlow Lite image classification model store component

  • TensorFlow Lite runtime component

When you deploy this component, it downloads a pre-trained MobileNet v1 model and installs the TensorFlow Lite runtime and its dependencies. This component publishes inference results on the ml/tflite/image-classification topic. To view these inference results, use the AWS IoT MQTT client in the AWS IoT console to subscribe to this topic.

In this tutorial you deploy the sample inference component to perform image classification on the sample image that is provided by AWS IoT Greengrass. After you complete this tutorial, you can complete Tutorial: Perform sample image classification inference on images from a camera using TensorFlow Lite, which shows you how to modify the sample inference component to perform image classification on images from a camera locally on a Greengrass core device.

For more information about machine learning on Greengrass devices, see Perform machine learning inference.

Prerequisites

To complete this tutorial, you need the following:

  • A Linux Greengrass core device. If you don't have one, see Tutorial: Getting started with AWS IoT Greengrass V2. The core device must meet the following requirements:

    • On Greengrass core devices running Amazon Linux 2 or Ubuntu 18.04, GNU C Library (glibc) version 2.27 or later installed on the device.

    • On Armv7l devices, such as Raspberry Pi, dependencies for OpenCV-Python installed on the device. Run the following command to install the dependencies.

      sudo apt-get install libopenjp2-7 libilmbase23 libopenexr-dev libavcodec-dev libavformat-dev libswscale-dev libv4l-dev libgtk-3-0 libwebp-dev

    • Raspberry Pi devices that run Raspberry Pi OS Bullseye must meet the following requirements:

      • NumPy 1.22.4 or later installed on the device. Raspberry Pi OS Bullseye includes an earlier version of NumPy, so you can run the following command to upgrade NumPy on the device.

        pip3 install --upgrade numpy

      • The legacy camera stack enabled on the device. Raspberry Pi OS Bullseye includes a new camera stack that is enabled by default and isn't compatible, so you must enable the legacy camera stack.

        To enable the legacy camera stack

        1. Run the following command to open the Raspberry Pi configuration tool.

          sudo raspi-config

        2. Select Interface Options.

        3. Select Legacy camera to enable the legacy camera stack.

        4. Reboot the Raspberry Pi.

Step 1: Subscribe to the default notifications topic

In this step, you configure the AWS IoT MQTT client in the AWS IoT console to watch MQTT messages published by the TensorFlow Lite image classification component. By default, the component publishes inference results on the ml/tflite/image-classification topic. Subscribe to this topic before you deploy the component to your Greengrass core device to see the inference results when the component runs for the first time.

To subscribe to the default notifications topic

  1. In the AWS IoT console navigation menu, choose Test, MQTT test client.

  2. Under Subscribe to a topic, in the Topic name box, enter ml/tflite/image-classification.

  3. Choose Subscribe.

Step 2: Deploy the TensorFlow Lite image classification component

In this step, you deploy the TensorFlow Lite image classification component to your core device:

  1. In the AWS IoT Greengrass console navigation menu, choose Components.

  2. On the Components page, on the Public components tab, choose aws.greengrass.TensorFlowLiteImageClassification.

  3. On the aws.greengrass.TensorFlowLiteImageClassification page, choose Deploy.

  4. From Add to deployment, choose one of the following:

    1. To merge this component to an existing deployment on your target device, choose Add to existing deployment, and then select the deployment that you want to revise.

    2. To create a new deployment on your target device, choose Create new deployment. If you have an existing deployment on your device, choosing this step replaces the existing deployment.

  5. On the Specify target page, do the following:

    1. Under Deployment information, enter or modify the friendly name for your deployment.

    2. Under Deployment targets, select a target for your deployment, and choose Next. You cannot change the deployment target if you are revising an existing deployment.

  6. On the Select components page, under Public components, verify that the aws.greengrass.TensorFlowLiteImageClassification component is selected, and choose Next.

  7. On the Configure components page, keep the default configuration settings, and choose Next.

  8. On the Configure advanced settings page, keep the default configuration settings, and choose Next.

  9. On the Review page, choose Deploy

  1. Create a deployment.json file to define the deployment configuration for the TensorFlow Lite image classification component. This file should look like the following:

    {
  "targetArn":"targetArn",
  "components": {
    "aws.greengrass.TensorFlowLiteImageClassification": {
      "componentVersion": 2.1.0,
      "configurationUpdate": {
      }
    }
  }
}

    • In the targetArn field, replace targetArn with the Amazon Resource Name (ARN) of the thing or thing group to target for the deployment, in the following format:

      • Thing: arn:aws:iot:region:account-id:thing/thingName

      • Thing group: arn:aws:iot:region:account-id:thinggroup/thingGroupName

    • This tutorial uses component version 2.1.0. In the aws.greengrass.TensorFlowLiteObjectDetection component object, replace 2.1.0 to use a different version of the TensorFlow Lite object detection component.

  2. Run the following command to deploy the TensorFlow Lite image classification component on the device:

    aws greengrassv2 create-deployment \
    --cli-input-json file://path/to/deployment.json

The deployment can take several minutes to complete. In the next step, check the component log to verify that the deployment completed successfully and to view the inference results.

Step 3: View inference results

After you deploy the component, you can view the inference results in the component log on your Greengrass core device and in the AWS IoT MQTT client in the AWS IoT console. To subscribe to the topic on which the component publishes inference results, see Step 1: Subscribe to the default notifications topic.

  • AWS IoT MQTT client—To view the results that the inference component publishes on the default notifications topic, complete the following steps:

    1. In the AWS IoT console navigation menu, choose Test, MQTT test client.

    2. Under Subscriptions, choose ml/tflite/image-classification.

      You should see messages similar to the following example.

      {
  "timestamp": "2021-01-01 00:00:00.000000",
  "inference-type": "image-classification",
  "inference-description": "Top 5 predictions with score 0.3 or above ",
  "inference-results": [
    {
      "Label": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor",
      "Score": "0.5882352941176471"
    },
    {
      "Label": "Persian cat",
      "Score": "0.5882352941176471"
    },
    {
      "Label": "tiger cat",
      "Score": "0.5882352941176471"
    },
    {
      "Label": "dalmatian, coach dog, carriage dog",
      "Score": "0.5607843137254902"
    },
    {
      "Label": "malamute, malemute, Alaskan malamute",
      "Score": "0.5450980392156862"
    }
  ]
}

  • Component log—To view the inference results in the component log, run the following command on your Greengrass core device. 

    sudo tail -f /greengrass/v2/logs/aws.greengrass.TensorFlowLiteImageClassification.log

    You should see results similar to the following example.

    2021-01-01 00:00:00.000000 [INFO] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stdout. Publishing results to the IoT core.... {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.Run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}
        
2021-01-01 00:00:00.000000 [INFO] (Copier) aws.greengrass.TensorFlowLiteImageClassification: stdout. {"timestamp": "2021-01-01 00:00:00.000000", "inference-type": "image-classification", "inference-description": "Top 5 predictions with score 0.3 or above ", "inference-results": [{"Label": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor", "Score": "0.5882352941176471"}, {"Label": "Persian cat", "Score": "0.5882352941176471"}, {"Label": "tiger cat", "Score": "0.5882352941176471"}, {"Label": "dalmatian, coach dog, carriage dog", "Score": "0.5607843137254902"}, {"Label": "malamute, malemute, Alaskan malamute", "Score": "0.5450980392156862"}]}. {scriptName=services.aws.greengrass.TensorFlowLiteImageClassification.lifecycle.Run.script, serviceName=aws.greengrass.TensorFlowLiteImageClassification, currentState=RUNNING}

If you can't see inference results in the component log or in the MQTT client, the deployment failed or didn't reach the core device. This can occur if your core device isn't connected to the internet or doesn't have the right permissions to run the component. Run the following command on your core device to view the AWS IoT Greengrass Core software log file. This file includes logs from the Greengrass core device's deployment service.

sudo tail -f /greengrass/v2/logs/greengrass.log

For more information, see Troubleshooting machine learning inference.

Next steps

If you have a Greengrass core device with a supported camera interface, you can complete Tutorial: Perform sample image classification inference on images from a camera using TensorFlow Lite, which shows you how to modify the sample inference component to perform image classification on images from a camera.

To further explore the configuration of the sample TensorFlow Lite image classification inference component, try the following:

  • Modify the InferenceInterval configuration parameter to change how often the inference code runs.

  • Modify the ImageName and ImageDirectory configuration parameters in the inference component configuration to specify a custom image to use for inference.

For information about customizing the configuration of public components or creating custom machine learning components, see Customize your machine learning components.