ML Image Classification connector

The ML Image Classification connectors provide a machine learning (ML) inference service that runs on the AWS IoT Greengrass core. This local inference service performs image classification using a model trained by the SageMaker AI image classification algorithm.

User-defined Lambda functions use the AWS IoT Greengrass Machine Learning SDK to submit inference requests to the local inference service. The service runs inference locally and returns probabilities that the input image belongs to specific categories.

AWS IoT Greengrass provides the following versions of this connector, which is available for multiple platforms.

For information about version changes, see the Changelog.

Requirements

These connectors have the following requirements:

Connector Parameters

These connectors provide the following parameters.

Create Connector Example (AWS CLI)

The following CLI commands create a ConnectorDefinition with an initial version that contains an ML Image Classification connector.

In the AWS IoT Greengrass console, you can add a connector from the group's Connectors page. For more information, see Getting started with Greengrass connectors (console).

Input data

These connectors accept an image file as input. Input image files must be in jpeg or png format. For more information, see Usage Example.

These connectors don't accept MQTT messages as input data.

Output data

These connectors return a formatted prediction for the object identified in the input image:

[0.3,0.1,0.04,...]

The prediction contains a list of values that correspond with the categories used in the training dataset during model training. Each value represents the probability that the image falls under the corresponding category. The category with the highest probability is the dominant prediction.

These connectors don't publish MQTT messages as output data.

Usage Example

The following example Lambda function uses the AWS IoT Greengrass Machine Learning SDK to interact with an ML Image Classification connector.

The example initializes an SDK client and synchronously calls the SDK's invoke_inference_service function to invoke the local inference service. It passes in the algorithm type, service name, image type, and image content. Then, the example parses the service response to get the probability results (predictions).

import logging
from threading import Timer

import numpy as np

import greengrass_machine_learning_sdk as ml

# We assume the inference input image is provided as a local file
# to this inference client Lambda function.
with open('/test_img/test.jpg', 'rb') as f:
    content = bytearray(f.read())

client = ml.client('inference')

def infer():
    logging.info('invoking Greengrass ML Inference service')

    try:
        resp = client.invoke_inference_service(
            AlgoType='image-classification',
            ServiceName='imageClassification',
            ContentType='image/jpeg',
            Body=content
        )
    except ml.GreengrassInferenceException as e:
        logging.info('inference exception {}("{}")'.format(e.__class__.__name__, e))
        return
    except ml.GreengrassDependencyException as e:
        logging.info('dependency exception {}("{}")'.format(e.__class__.__name__, e))
        return

    logging.info('resp: {}'.format(resp))
    predictions = resp['Body'].read().decode("utf-8")
    logging.info('predictions: {}'.format(predictions))
    
    # The connector output is in the format: [0.3,0.1,0.04,...]
    # Remove the '[' and ']' at the beginning and end.
    predictions = predictions[1:-1]
    count = len(predictions.split(','))
    predictions_arr = np.fromstring(predictions, count=count, sep=',')

    # Perform business logic that relies on the predictions_arr, which is an array
    # of probabilities.
    
    # Schedule the infer() function to run again in one second.
    Timer(1, infer).start()
    return

infer()

def function_handler(event, context):
    return

The invoke_inference_service function in the AWS IoT Greengrass Machine Learning SDK accepts the following arguments.

Installing MXNet dependencies on the AWS IoT Greengrass core

To use an ML Image Classification connector, you must install the dependencies for the Apache MXNet framework on the core device. The connectors use the framework to serve the ML model.

AWS IoT Greengrass provides scripts to install the dependencies for the following common platforms and devices (or to use as a reference for installing them). If you're using a different platform or device, see the MXNet documentation for your configuration.

Before installing the MXNet dependencies, make sure that the required system libraries (with the specified minimum versions) are present on the device.

Logging and troubleshooting

Depending on your group settings, event and error logs are written to CloudWatch Logs, the local file system, or both. Logs from this connector use the prefix LocalInferenceServiceName. If the connector behaves unexpectedly, check the connector's logs. These usually contain useful debugging information, such as a missing ML library dependency or the cause of a connector startup failure.

If the AWS IoT Greengrass group is configured to write local logs, the connector writes log files to greengrass-root/ggc/var/log/user/region/aws/. For more information about Greengrass logging, see Monitoring with AWS IoT Greengrass logs.

Use the following information to help troubleshoot issues with the ML Image Classification connectors.

Required system libraries

The following tabs list the system libraries required for each ML Image Classification connector.

Issues

sudo ln /dev/null /dev/raw1394

Licenses

The ML Image Classification connectors includes the following third-party software/licensing:

Intel OpenMP Runtime Library licensing. The Intel® OpenMP* runtime is dual-licensed, with a commercial (COM) license as part of the Intel® Parallel Studio XE Suite products, and a BSD open source (OSS) license.

This connector is released under the Greengrass Core Software License Agreement.

Changelog

The following table describes the changes in each version of the connector.

A Greengrass group can contain only one version of the connector at a time. For information about upgrading a connector version, see Upgrading connector versions.

See also