Using Amazon Deep Learning Containers with Amazon Elastic Inference on Amazon EC2 - Amazon Elastic Inference
Prerequisites

Using Amazon Deep Learning Containers with Amazon Elastic Inference on Amazon EC2

Amazon Deep Learning Containers with Amazon Elastic Inference (Elastic Inference) are a set of Docker images for serving models in TensorFlow, Apache MXNet (MXNet), and PyTorch on Amazon Elastic Compute Cloud (Amazon EC2). Deep Learning Containers provide optimized environments with TensorFlow, MXNet, and PyTorch. They are available in the Amazon Elastic Container Registry (Amazon ECR).

These tutorials describe how to use Deep Learning Containers with Elastic Inference on Amazon Elastic Compute Cloud (Amazon EC2).

Topics

Prerequisites

Before you start this tutorial, set up the following resources in the AWS Management Console.

  1. Create an AWS Identity and Access Management (IAM) user and attach the following policies:

  2. Follow the instructions for Setting Up EI with the following modification:

    Create a security group (use the default VPC, or create a VPC with an internet gateway) and open the ports necessary for your desired inference server:

    • All frameworks require: 22 for SSH and 443 for HTTPS

    • TensorFlow inference: 8500 and 8501 open to TCP traffic

    • MXNet and PyTorch inference: 80 and 8081 open to TCP traffic

  3. Launch an Amazon EC2 instance with the Elastic Inference role using the AWS Deep Learning Base Amazon Machine Image (AMI). Because you need only the AWS Command Line Interface (AWS CLI) and Docker, this is the best AMI.

  4. SSH into the Amazon EC2 instance.

  5. On the instance, run the following commands using the keys associated with the user created in Step1. Confirm that Elastic Inference is available in your region.

    aws configure set aws_access_key_id <access_key_id>

aws configure set aws_secret_access_key <secret_access_key>

aws configure set region <region>

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com