Use Amazon SageMaker Edge Manager on Greengrass core devices - AWS IoT Greengrass
How it worksRequirementsGet started with SageMaker Edge Manager

Use Amazon SageMaker Edge Manager on Greengrass core devices

Important

SageMaker Edge Manager was discontinued on April 26th, 2024. For more information about continuing to deploy your models to edge devices, see SageMaker Edge Manager end of life.

Amazon SageMaker Edge Manager is a software agent that runs on edge devices. SageMaker Edge Manager provides model management for edge devices so that you can package and use Amazon SageMaker Neo-compiled models directly on Greengrass core devices. By using SageMaker Edge Manager, you can also sample model input and output data from your core devices, and send that data to the AWS Cloud for monitoring and analysis. Because SageMaker Edge Manager uses SageMaker Neo to optimize your models for your target hardware, you don't need to install the DLR runtime directly on your device. On Greengrass devices, SageMaker Edge Manager doesn't load local AWS IoT certificates or call the AWS IoT credential provider endpoint directly. Instead, SageMaker Edge Manager uses the token exchange service to fetch temporary credential from a TES endpoint.

This section describes how SageMaker Edge Manager works on Greengrass core devices.

How SageMaker Edge Manager works on Greengrass devices

To deploy the SageMaker Edge Manager agent to your core devices, create a deployment that includes the aws.greengrass.SageMakerEdgeManager component. AWS IoT Greengrass manages the installation and lifecycle of the Edge Manager agent on your devices. When a new version of the agent binary is available, deploy the updated version of the aws.greengrass.SageMakerEdgeManager component to upgrade the version of the agent that is installed on your device.

When you use SageMaker Edge Manager with AWS IoT Greengrass, your workflow includes the following high-level steps:

  1. Compile models with SageMaker Neo.

  2. Package your SageMaker Neo-compiled models using SageMaker edge packaging jobs. When you run an edge packaging job for your model, you can choose to create a model component with the packaged model as an artifact that can be deployed to your Greengrass core device.

  3. Create a custom inference component. You use this inference component to interact with the Edge Manager agent to perform inference on the core device. These operations include loading models, invoke prediction requests to run inference, and unloading models when the component shuts down.

  4. Deploy the SageMaker Edge Manager component, the packaged model component, and the inference component to run your model on the SageMaker inference engine (Edge Manager agent) on your device.

For more information about creating edge packaging jobs and inference components that work with SageMaker Edge Manager, see Deploy Model Package and Edge Manager Agent with AWS IoT Greengrass in the Amazon SageMaker Developer Guide.

The Tutorial: Get started with SageMaker Edge Manager tutorial shows you how to set up and use the SageMaker Edge Manager agent on an existing Greengrass core device, using AWS-provided example code that you can use to create sample inference and model components.

When you use SageMaker Edge Manager on Greengrass core devices, you can also use the capture data feature to upload sample data to the AWS Cloud. Capture data is a SageMaker feature that you use to upload inference input, inference results, and additional inference data to an S3 bucket or a local directory for future analysis. For more information about using capture data with SageMaker Edge Manager, see Manage Model in the Amazon SageMaker Developer Guide.

Requirements

You must meet the following requirements to use the SageMaker Edge Manager agent on Greengrass core devices.

  • A Greengrass core device running on Amazon Linux 2, a Debian-based Linux platform (x86_64 or Armv8), or Windows (x86_64). If you don't have one, see Tutorial: Getting started with AWS IoT Greengrass V2.

  • Python 3.6 or later, including pip for your version of Python, installed on your core device.

  • The Greengrass device role configured with the following:

    • A trust relationship that allows credentials.iot.amazonaws.com and sagemaker.amazonaws.com to assume the role, as shown in the following IAM policy example.

      { 
  "Version": "2012-10-17",
  "Statement": [ 
    { 
      "Effect": "Allow", 
      "Principal": {
        "Service": "credentials.iot.amazonaws.com"
       }, 
      "Action": "sts:AssumeRole" 
    },
    { 
      "Effect": "Allow", 
      "Principal": {
        "Service": "sagemaker.amazonaws.com"
      }, 
      "Action": "sts:AssumeRole" 
    } 
  ] 
}

    • The AmazonSageMakerEdgeDeviceFleetPolicy IAM managed policy.

    • The s3:PutObject action, as shown in the following IAM policy example.

      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:PutObject"
      ],
      "Resource": [
        "*"
      ],
      "Effect": "Allow"
    }
  ]
}

  • An Amazon S3 bucket created in the same AWS account and AWS Region as your Greengrass core device. SageMaker Edge Manager requires an S3 bucket to create an edge device fleet, and to store sample data from running inference on your device. For information about creating S3 buckets, see Getting started with Amazon S3.

  • A SageMaker edge device fleet that uses the same AWS IoT role alias as your Greengrass core device. For more information, see Create an edge device fleet.

  • Your Greengrass core device registered as an edge device in your SageMaker Edge device fleet. The edge device name must match the AWS IoT thing name for your core device. For more information, see Register your Greengrass core device.

Get started with SageMaker Edge Manager

You can complete a tutorial to get started using SageMaker Edge Manager. The tutorial shows you how to get started using SageMaker Edge Manager with AWS-provided sample components on an existing core device. These sample components use the SageMaker Edge Manager component as a dependency to deploy the Edge Manager agent, and perform inference using pre-trained models that were compiled using SageMaker Neo. For more information, see Tutorial: Get started with SageMaker Edge Manager.