Setting Up to Launch Amazon EC2 with Elastic Inference
The most convenient way to set up Amazon EC2 with Elastic Inference uses the Elastic Inference setup script
described in https://aws.amazon.com/blogs/machine-learning/launch-ei-accelerators-in-minutes-with-the-amazon-elastic-inference-setup-tool-for-ec2/
Topics
Configuring Your Security Groups for Elastic Inference
You need two security groups. One for inbound and outbound traffic for the new Elastic Inference VPC endpoint. A second one for outbound traffic for the associated Amazon EC2 instances that you launch.
Configure Your Security Groups for Elastic Inference
To configure a security group for an Elastic Inference accelerator (console)
Open the Amazon VPC console at https://console.aws.amazon.com/vpc/
. -
In the left navigation pane, choose Security, Security Groups.
-
Choose Create Security Group
-
Under Create Security Group, specify a name and description for the security group and choose the ID of the VPC. Choose Create and then choose Close.
-
Select the check box next to your security group and choose Actions, Edit inbound rules. Add a rule to allow HTTPS traffic on port 443 as follows:
-
Choose Add Rule.
-
For Type, select HTTPS.
-
For Source, specify a CIDR block (for example, 0.0.0.0/0) or the security group for your instance.
-
To allow traffic for port 22 to the EC2 instance, repeat the procedure. For Type, select SSH.
-
Choose Save rules and then choose Close.
-
-
Choose Edit outbound rules. Choose Add rule. To allow traffic for all ports, for Type, select All Traffic.
-
Choose Save rules.
To configure a security group for an Elastic Inference accelerator (AWS CLI)
-
Create a security group using the create-security-group command:
aws ec2 create-security-group --description
insert a description for the security group
--group-nameassign a name for the security group
[--vpc-identer the VPC ID
] -
Create inbound rules using the authorize-security-group-ingress command:
aws ec2 authorize-security-group-ingress --group-id
insert the security group ID
--protocol tcp --port 443 --cidr0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id
insert the security group ID
--protocol tcp --port 22 --cidr0.0.0.0/0
-
The default setting for outbound rules allows all traffic from all ports for this instance.
Configuring AWS PrivateLink Endpoint Services
Elastic Inference uses VPC endpoints to privately connect the instance in your VPC with their associated Elastic Inference accelerator. Create a VPC endpoint for Elastic Inference before you launch instances with accelerators. This needs to be done just one time per VPC. For more information, see Interface VPC Endpoints (AWS PrivateLink).
To configure an AWS PrivateLink endpoint service (console)
Open the Amazon VPC console at https://console.aws.amazon.com/vpc/
. -
In the left navigation pane, choose Endpoints, Create Endpoint.
-
For Service category, choose Find service by name.
-
For Service Name, select com.amazonaws.
<your-region>
.elastic-inference.runtime.For example, for the us-west-2 Region, select com.amazonaws.us-west-2.elastic-inference.runtime.
-
For Subnets, select one or more Availability Zones where the endpoint should be created. Where you plan to launch instances with accelerators, you must select subnets for the Availability Zone.
-
Enable the private DNS name and enter the security group for your endpoint. Choose Create endpoint. Note the VPC endpoint ID for later.
-
The security group that we configured for the endpoint in previous steps must allow inbound traffic to port 443.
To configure an AWS PrivateLink endpoint service (AWS CLI)
-
Use the https://docs.aws.amazon.com/cli/latest/reference/ec2/create-vpc-endpoint.html command and specify the following: VPC ID, type of VPC endpoint (interface), service name, subnets to use the endpoint, and security groups to associate with the endpoint network interfaces. For information about how to set up a security group for your VPC endpoint, see Configuring Your Security Groups for Elastic Inference.
aws ec2 create-vpc-endpoint --vpc-id vpc-
insert VPC ID
--vpc-endpoint-type Interface --service-name com.amazonaws.us-west-2
.elastic-inference.runtime --subnet-id subnet-insert subnet
--security-group-id sg-insert security group ID
Configuring an Instance Role with an Elastic Inference Policy
To launch an instance with an Elastic Inference accelerator, you must provide an IAM role that allows actions on Elastic Inference accelerators.
To configure an instance role with an Elastic Inference policy (console)
Open the IAM console at https://console.aws.amazon.com/iam/
. -
In the left navigation pane, choose Policies, Create Policy.
-
Choose JSON and paste the following policy:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elastic-inference:Connect", "iam:List*", "iam:Get*", "ec2:Describe*", "ec2:Get*" ], "Resource": "*" } ] }
Note
You may get a warning message about the Elastic Inference service not being recognizable. This is a known issue and does not block creation of the policy.
-
Choose Review policy and enter a name for the policy, such as
ec2-role-trust-policy.json
, and a description. -
Choose Create policy.
-
In the left navigation pane, choose Roles, Create role.
-
Choose AWS service, EC2, Next: Permissions.
-
Select the name of the policy that you just created (
ec2-role-trust-policy.json
). Choose Next: Tags. -
Provide a role name and choose Create Role.
When you create your instance, select the role under Configure Instance Details in the launch wizard.
To configure an instance role with an Elastic Inference policy (AWS CLI)
-
To configure an instance role with an Elastic Inference policy, follow the steps in Creating an IAM Role. Add the following policy to your instance:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "elastic-inference:Connect", "iam:List*", "iam:Get*", "ec2:Describe*", "ec2:Get*" ], "Resource": "*" } ] }
Note
You may get a warning message about the Elastic Inference service not being recognizable. This is a known issue and does not block creation of the policy.
Launching an Instance with Elastic Inference
You can now configure Amazon EC2 instances with accelerators to launch within your subnet. Choose any supported Amazon EC2 instance type and Elastic Inference accelerator size. Elastic Inference accelerators are available to all current generation instance types. There are two accelerator types.
EIA2 is the second generation of Elastic Inference accelerators. It offers improved performance and increased memory. With up to 8 GB of GPU memory, EIA2 is a cost-effective resource for deploying machine learning (ML) models. Use it for applications such as image classification, object detection, automated speech recognition, and language translation. Your accelerator memory choices depend on the size of your input and models. You can choose from the following Elastic Inference accelerators:
-
eia2.medium
with 2 GB of accelerator memory -
eia2.large
with 4 GB of accelerator memory -
eia2.xlarge
with 8 GB of accelerator memory
Note: We continue to support EIA1 in three sizes: eia1.medium, eia1.large, and eia1.xlarge
You can launch an instance with Elastic Inference automatically by using the Amazon Elastic Inference setup tool for EC2
To launch an instance with Elastic Inference (console)
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/
. -
Choose Launch Instance.
-
Under Choose an Amazon Machine Image, select an Amazon Linux or Ubuntu AMI. We recommend one of the Deep Learning AMIs.
Note
Attaching multiple Elastic Inference accelerators to a single Amazon EC2 instance requires that the instance has AWS Deep Learning AMIs (DLAMI) Version 25 or later.
-
Under Choose an Instance Type, select the hardware configuration of your instance.
-
Choose Next: Configure Instance Details.
-
Under Configure Instance Details, check the configuration settings. Ensure that you are using the VPC with the security groups for the instance and the Elastic Inference accelerator that you set up earlier. For more information, see Configuring Your Security Groups for Elastic Inference.
-
For IAM role, select the role that you created in the Configuring an Instance Role with an Elastic Inference Policy procedure.
-
Select Add an Elastic Inference accelerator.
-
Select the size and amount of Elastic Inference accelerators. Your options are:
eia2.medium
,eia2.large
, andeia2.xlarge
. -
To add another Elastic Inference accelerator, choose Add. Then select the size and amount of accelerators to add.
-
(Optional) You can choose to add storage and tags by choosing Next at the bottom of the page. Or, you can let the instance wizard complete the remaining configuration steps for you.
-
In the Add Security Group step, choose the security group created previously.
-
Review the configuration of your instance and choose Launch.
-
You are prompted to choose an existing key pair for your instance or to create a new key pair. For more information, see Amazon EC2 Key Pairs..
Warning
Don’t select the Proceed without a key pair option. If you launch your instance without a key pair, then you can’t connect to it.
-
After making your key pair selection, choose Launch Instances.
-
A confirmation page lets you know that your instance is launching. To close the confirmation page and return to the console, choose View Instances.
-
Under Instances, you can view the status of the launch. It takes a short time for an instance to launch. When you launch an instance, its initial state is
pending
. After the instance starts, its state changes torunning
. -
It can take a few minutes for the instance to be ready so that you can connect to it. Check that your instance has passed its status checks. You can view this information in the Status Checks column.
To launch an instance with Elastic Inference (AWS CLI)
To launch an instance with Elastic Inference at the command line, you need your key pair name, subnet ID, security group ID, AMI ID, and the name of the instance profile that you created in the section Configuring an Instance Role with an Elastic Inference Policy. For the security group ID, use the one you created for your instance that contains the AWS PrivateLink endpoint. For more information, see Configuring Your Security Groups for Elastic Inference). For more information about the AMI ID, see Finding a Linux AMI.
-
Use the run-instances command to launch your instance and accelerator:
aws ec2 run-instances --image-id ami-
image ID
--instance-typem5.large
--subnet-id subnet-subnet ID
--elastic-inference-accelerator Type=eia2.large
--key-namekey pair name
--security-group-ids sg-security group ID
--iam-instance-profile Name="accelerator profile name
"To launch an instance with multiple accelerators, you can add multiple
Type
parameters to--elastic-inference-accelerator
.aws ec2 run-instances --image-id ami-
image ID
--instance-typem5.large
--subnet-id subnet-subnet ID
--elastic-inference-accelerator Type=eia2.large
,Count=2 Type=eia2.xlarge
--key-namekey pair name
--regionregion name
--security-group-ids sg-security group ID
-
When the
run-instances
operation succeeds, your output is similar to the following. TheElasticInferenceAcceleratorArn
identifies the Elastic Inference accelerator."ElasticInferenceAcceleratorAssociations": [ { "ElasticInferenceAcceleratorArn": "arn:aws:elastic-inference:us-west-2:204044812891:elastic-inference-accelerator/eia-3e1de7c2f64a4de8b970c205e838af6b", "ElasticInferenceAcceleratorAssociationId": "eia-assoc-031f6f53ddcd5f260", "ElasticInferenceAcceleratorAssociationState": "associating", "ElasticInferenceAcceleratorAssociationTime": "2018-10-05T17:22:20.000Z" } ],
You are now ready to run your models using TensorFlow, MXNet, or PyTorch on the provided AMI.
Once your Elastic Inference accelerator is running, you can use the
describe-accelerators
AWS CLI. This command returns information about
the accelerator, such as the region it is in and the name of the accelerator. For
more information about the usage of this command, see the Elastic Inference AWS CLI Command Reference.