Getting started with AWS App Mesh and Amazon EC2
Important
End of support notice: On September 30, 2026, AWS will discontinue support for AWS App Mesh. After September 30, 2026, you will no longer be able to access the AWS App Mesh console or AWS App Mesh resources. For more information, visit this blog post Migrating from AWS App Mesh to Amazon ECS Service Connect
This topic helps you use AWS App Mesh with an actual service that is running on Amazon EC2. This tutorial covers basic features of several App Mesh resource types.
Scenario
To illustrate how to use App Mesh, assume that you have an application with the following characteristics:
-
Consists of two services named
serviceA
andserviceB
. -
Both services are registered to a namespace named
apps.local
. -
ServiceA
communicates withserviceB
over HTTP/2, port 80. -
You have already deployed version 2 of
serviceB
and registered it with the nameserviceBv2
in theapps.local
namespace.
You have the following requirements:
-
You want to send 75 percent of the traffic from
serviceA
toserviceB
and 25 percent of the traffic toserviceBv2
first. By only sending 25 percent toserviceBv2
, you can validate that it's bug free before you send 100 percent of the traffic fromserviceA
. -
You want to be able to easily adjust the traffic weighting so that 100 percent of the traffic goes to
serviceBv2
once it is proven to be reliable. Once all traffic is being sent toserviceBv2
, you want to discontinueserviceB
. -
You do not want to have to change any existing application code or service discovery registration for your actual services to meet the previous requirements.
To meet your requirements, you decide to create an App Mesh service mesh with virtual services, virtual nodes, a virtual router, and a route. After implementing your mesh, you update your services to use the Envoy proxy. Once updated, your services communicate with each other through the Envoy proxy rather than directly with each other.
Prerequisites
App Mesh supports Linux services that are registered with DNS, AWS Cloud Map, or both. To use this getting started guide, we recommend that you have three existing services that are registered with DNS. You can create a service mesh and its resources even if the services don't exist, but you cannot use the mesh until you have deployed actual services.
If you don't already have services running, you can launch Amazon EC2 instances and
deploy applications to them. For more information, see Tutorial: Getting started
with Amazon EC2 Linux instances in the Amazon EC2 User Guide. The remaining steps
assume that the actual services are named serviceA
, serviceB
,
and serviceBv2
and that all services are discoverable through a namespace
named apps.local
.
Step 1: Create a mesh and virtual service
A service mesh is a logical boundary for network traffic between the services that reside within it. For more information, see Service Meshes. A virtual service is an abstraction of an actual service. For more information, see Virtual services.
Create the following resources:
-
A mesh named
apps
, since all of the services in the scenario are registered to theapps.local
namespace. -
A virtual service named
serviceb.apps.local
, since the virtual service represents a service that is discoverable with that name, and you don't want to change your code to reference another name. A virtual service namedservicea.apps.local
is added in a later step.
You can use the AWS Management Console or the AWS CLI version 1.18.116 or higher or
2.0.38 or higher to complete the following steps. If using the AWS CLI,
use the aws --version
command to check your installed AWS CLI version. If you
don't have version 1.18.116 or higher or 2.0.38 or
higher installed, then you must install
or update the AWS CLI. Select the tab for the tool that you want to
use.
Step 2: Create a virtual node
A virtual node acts as a logical pointer to an actual service. For more information, see Virtual nodes.
Create a virtual node named serviceB
, since one of the virtual nodes
represents the actual service named serviceB
. The actual service that the
virtual node represents is discoverable through DNS
with a hostname of
serviceb.apps.local
. Alternately, you can discover actual services using
AWS Cloud Map. The virtual node listens for traffic using the HTTP/2 protocol on port 80. Other
protocols are also supported, as are health checks. You create virtual nodes for
serviceA
and serviceBv2
in a later step.
Step 3: Create a virtual router and route
Virtual routers route traffic for one or more virtual services within your mesh. For more information, see Virtual routers and Routes.
Create the following resources:
-
A virtual router named
serviceB
, since theserviceB.apps.local
virtual service does not initiate outbound communication with any other service. Remember that the virtual service that you created previously is an abstraction of your actualserviceb.apps.local
service. The virtual service sends traffic to the virtual router. The virtual router listens for traffic using the HTTP/2 protocol on port 80. Other protocols are also supported. -
A route named
serviceB
. It routes 100 percent of its traffic to theserviceB
virtual node. The weight is in a later step once you add theserviceBv2
virtual node. Though not covered in this guide, you can add additional filter criteria for the route and add a retry policy to cause the Envoy proxy to make multiple attempts to send traffic to a virtual node when it experiences a communication problem.
Step 4: Review and create
Review the settings against the previous instructions.
Step 5: Create additional resources
To complete the scenario, you need to:
-
Create one virtual node named
serviceBv2
and another namedserviceA
. Both virtual nodes listen for requests over HTTP/2 port 80. For theserviceA
virtual node, configure a backend ofserviceb.apps.local
. All outbound traffic from theserviceA
virtual node is sent to the virtual service namedserviceb.apps.local
. Though not covered in this guide, you can also specify a file path to write access logs to for a virtual node. -
Create one additional virtual service named
servicea.apps.local
, which sends all traffic directly to theserviceA
virtual node. -
Update the
serviceB
route that you created in a previous step to send 75 percent of its traffic to theserviceB
virtual node and 25 percent of its traffic to theserviceBv2
virtual node. Over time, you can continue to modify the weights untilserviceBv2
receives 100 percent of the traffic. Once all traffic is sent toserviceBv2
, you can shut down and discontinue theserviceB
virtual node and actual service. As you change weights, your code does not require any modification, because theserviceb.apps.local
virtual and actual service names don't change. Recall that theserviceb.apps.local
virtual service sends traffic to the virtual router, which routes the traffic to the virtual nodes. The service discovery names for the virtual nodes can be changed at any time.
Mesh summary
Before you created the service mesh, you had three actual services named
servicea.apps.local
, serviceb.apps.local
, and
servicebv2.apps.local
. In addition to the actual services, you now
have a service mesh that contains the following resources that represent the actual
services:
-
Two virtual services. The proxy sends all traffic from the
servicea.apps.local
virtual service to theserviceb.apps.local
virtual service through a virtual router. -
Three virtual nodes named
serviceA
,serviceB
, andserviceBv2
. The Envoy proxy uses the service discovery information configured for the virtual nodes to look up the IP addresses of the actual services. -
One virtual router with one route that instructs the Envoy proxy to route 75 percent of inbound traffic to the
serviceB
virtual node and 25 percent of the traffic to theserviceBv2
virtual node.
Step 6: Update services
After creating your mesh, you need to complete the following tasks:
-
Authorize the Envoy proxy that you deploy with each service to read the configuration of one or more virtual nodes. For more information about how to authorize the proxy, see Envoy Proxy authorization.
-
To update your existing service, complete the steps that follow.
To configure an Amazon EC2 instance as a virtual node member
-
Create an IAM role.
-
Create a file named
ec2-trust-relationship.json
with the following contents.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
-
Create an IAM role with the following command.
aws iam create-role --role-name
mesh-virtual-node-service-b
--assume-role-policy-document file://ec2-trust-relationship.json
-
-
Attach IAM policies to the role that allow it to read from Amazon ECR and only the configuration of a specific App Mesh virtual node.
-
Create a file named
virtual-node-policy.json
with the following contents.apps
is the name of the mesh you created in Step 1: Create a mesh and virtual service andserviceB
is the name of the virtual node that you created in Step 2: Create a virtual node. Replace111122223333
with your account ID andus-west-2
with the Region that you created your mesh in.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "appmesh:StreamAggregatedResources", "Resource": [ "arn:aws:appmesh:
us-west-2
:111122223333
:mesh/apps
/virtualNode/serviceB
" ] } ] } -
Create the policy with the following command.
aws iam create-policy --policy-name
virtual-node-policy
--policy-document file://virtual-node-policy.json -
Attach the policy that you created in the previous step to the role so the role can read the configuration for only the
serviceB
virtual node from App Mesh.aws iam attach-role-policy --policy-arn arn:aws:iam::
111122223333
:policy/virtual-node-policy --role-namemesh-virtual-node-service-b
-
Attach the
AmazonEC2ContainerRegistryReadOnly
managed policy to the role so that it can pull the Envoy container image from Amazon ECR.aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly --role-name
mesh-virtual-node-service-b
-
-
Launch an Amazon EC2 instance with the IAM role that you created.
-
Connect to your instance via SSH.
-
Install Docker and the AWS CLI on your instance according to your operating system documentation.
-
Authenticate to the Envoy Amazon ECR repository in the Region that you want your Docker client to pull the image from.
-
All Regions except
me-south-1
,ap-east-1
,ap-southeast-3
,eu-south-1
,il-central-1
, andaf-south-1
. You can replaceus-west-2
with any supported Region exceptme-south-1
,ap-east-1
,ap-southeast-3
,eu-south-1
,il-central-1
, andaf-south-1
.$
aws ecr get-login-password \ --region
us-west-2
\ | docker login \ --username AWS \ --password-stdin 840364872350.dkr.ecr.us-west-2
.amazonaws.com -
me-south-1
Region$
aws ecr get-login-password \ --region me-south-1 \ | docker login \ --username AWS \ --password-stdin 772975370895.dkr.ecr.me-south-1.amazonaws.com
-
ap-east-1
Region$
aws ecr get-login-password \ --region ap-east-1 \ | docker login \ --username AWS \ --password-stdin 856666278305.dkr.ecr.ap-east-1.amazonaws.com
-
-
Run one of the following commands to start the App Mesh Envoy container on your instance, depending on which Region you want to pull the image from. The
apps
andserviceB
values are the mesh and virtual node names defined in the scenario. This information tells the proxy which virtual node configuration to read from App Mesh. To complete the scenario, you also need to complete these steps for the Amazon EC2 instances that host the services represented by theserviceBv2
andserviceA
virtual nodes. For your own application, replace these values with your own.-
All Regions except
me-south-1
,ap-east-1
,ap-southeast-3
,eu-south-1
,il-central-1
, andaf-south-1
. You can replaceRegion-code
with any supported Region except theme-south-1
,ap-east-1
,ap-southeast-3
,eu-south-1
,il-central-1
, andaf-south-1
Regions. You can replace
with any value between1337
0
and2147483647
.sudo docker run --detach --env APPMESH_RESOURCE_ARN=
mesh/
\ -uapps
/virtualNode/serviceB
1337
--network host 840364872350.dkr.ecr.region-code
.amazonaws.com/aws-appmesh-envoy:v1.29.9.0-prod -
me-south-1
Region. You can replace
with any value between1337
0
and2147483647
.sudo docker run --detach --env APPMESH_RESOURCE_ARN=
mesh/
\ -uapps
/virtualNode/serviceB
1337
--network host 772975370895.dkr.ecr.me-south-1.amazonaws.com/aws-appmesh-envoy:v1.29.9.0-prod -
ap-east-1
Region. You can replace
with any value between1337
0
and2147483647
.sudo docker run --detach --env APPMESH_RESOURCE_ARN=
mesh/
\ -uapps
/virtualNode/serviceB
1337
--network host 856666278305.dkr.ecr.ap-east-1.amazonaws.com/aws-appmesh-envoy:v1.29.9.0-prod
Note
The
APPMESH_RESOURCE_ARN
property requires version1.15.0
or later of the Envoy image. For more information, see Envoy image.Important
Only version v1.9.0.0-prod or later is supported for use with App Mesh.
-
Select
Show more
below. Create a file namedenvoy-networking.sh
on your instance with the following contents. Replace8000
with the port that your application code uses for incoming traffic. You can change the value forAPPMESH_IGNORE_UID
, but the value must be the same as the value that you specified in the previous step; for example1337
. You can add additional addresses toAPPMESH_EGRESS_IGNORED_IP
if necessary. Do not modify any other lines.#!/bin/bash -e # # Start of configurable options # #APPMESH_START_ENABLED="0" APPMESH_IGNORE_UID="
1337
" APPMESH_APP_PORTS="8000
" APPMESH_ENVOY_EGRESS_PORT="15001" APPMESH_ENVOY_INGRESS_PORT="15000" APPMESH_EGRESS_IGNORED_IP="169.254.169.254,169.254.170.2" # Enable routing on the application start. [ -z "$APPMESH_START_ENABLED" ] && APPMESH_START_ENABLED="0" # Enable IPv6. [ -z "$APPMESH_ENABLE_IPV6" ] && APPMESH_ENABLE_IPV6="0" # Egress traffic from the processess owned by the following UID/GID will be ignored. if [ -z "$APPMESH_IGNORE_UID" ] && [ -z "$APPMESH_IGNORE_GID" ]; then echo "Variables APPMESH_IGNORE_UID and/or APPMESH_IGNORE_GID must be set." echo "Envoy must run under those IDs to be able to properly route it's egress traffic." exit 1 fi # Port numbers Application and Envoy are listening on. if [ -z "$APPMESH_ENVOY_EGRESS_PORT" ]; then echo "APPMESH_ENVOY_EGRESS_PORT must be defined to forward traffic from the application to the proxy." exit 1 fi # If an app port was specified, then we also need to enforce the proxies ingress port so we know where to forward traffic. if [ ! -z "$APPMESH_APP_PORTS" ] && [ -z "$APPMESH_ENVOY_INGRESS_PORT" ]; then echo "APPMESH_ENVOY_INGRESS_PORT must be defined to forward traffic from the APPMESH_APP_PORTS to the proxy." exit 1 fi # Comma separated list of ports for which egress traffic will be ignored, we always refuse to route SSH traffic. if [ -z "$APPMESH_EGRESS_IGNORED_PORTS" ]; then APPMESH_EGRESS_IGNORED_PORTS="22" else APPMESH_EGRESS_IGNORED_PORTS="$APPMESH_EGRESS_IGNORED_PORTS,22" fi # # End of configurable options # function initialize() { echo "=== Initializing ===" if [ ! -z "$APPMESH_APP_PORTS" ]; then iptables -t nat -N APPMESH_INGRESS if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then ip6tables -t nat -N APPMESH_INGRESS fi fi iptables -t nat -N APPMESH_EGRESS if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then ip6tables -t nat -N APPMESH_EGRESS fi } function enable_egress_routing() { # Stuff to ignore [ ! -z "$APPMESH_IGNORE_UID" ] && \ iptables -t nat -A APPMESH_EGRESS \ -m owner --uid-owner $APPMESH_IGNORE_UID \ -j RETURN [ ! -z "$APPMESH_IGNORE_GID" ] && \ iptables -t nat -A APPMESH_EGRESS \ -m owner --gid-owner $APPMESH_IGNORE_GID \ -j RETURN [ ! -z "$APPMESH_EGRESS_IGNORED_PORTS" ] && \ for IGNORED_PORT in $(echo "$APPMESH_EGRESS_IGNORED_PORTS" | tr "," "\n"); do iptables -t nat -A APPMESH_EGRESS \ -p tcp \ -m multiport --dports "$IGNORED_PORT" \ -j RETURN done if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then # Stuff to ignore ipv6 [ ! -z "$APPMESH_IGNORE_UID" ] && \ ip6tables -t nat -A APPMESH_EGRESS \ -m owner --uid-owner $APPMESH_IGNORE_UID \ -j RETURN [ ! -z "$APPMESH_IGNORE_GID" ] && \ ip6tables -t nat -A APPMESH_EGRESS \ -m owner --gid-owner $APPMESH_IGNORE_GID \ -j RETURN [ ! -z "$APPMESH_EGRESS_IGNORED_PORTS" ] && \ for IGNORED_PORT in $(echo "$APPMESH_EGRESS_IGNORED_PORTS" | tr "," "\n"); do ip6tables -t nat -A APPMESH_EGRESS \ -p tcp \ -m multiport --dports "$IGNORED_PORT" \ -j RETURN done fi # The list can contain both IPv4 and IPv6 addresses. We will loop over this list # to add every IPv4 address into `iptables` and every IPv6 address into `ip6tables`. [ ! -z "$APPMESH_EGRESS_IGNORED_IP" ] && \ for IP_ADDR in $(echo "$APPMESH_EGRESS_IGNORED_IP" | tr "," "\n"); do if [[ $IP_ADDR =~ .*:.* ]] then [ "$APPMESH_ENABLE_IPV6" == "1" ] && \ ip6tables -t nat -A APPMESH_EGRESS \ -p tcp \ -d "$IP_ADDR" \ -j RETURN else iptables -t nat -A APPMESH_EGRESS \ -p tcp \ -d "$IP_ADDR" \ -j RETURN fi done # Redirect everything that is not ignored iptables -t nat -A APPMESH_EGRESS \ -p tcp \ -j REDIRECT --to $APPMESH_ENVOY_EGRESS_PORT # Apply APPMESH_EGRESS chain to non local traffic iptables -t nat -A OUTPUT \ -p tcp \ -m addrtype ! --dst-type LOCAL \ -j APPMESH_EGRESS if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then # Redirect everything that is not ignored ipv6 ip6tables -t nat -A APPMESH_EGRESS \ -p tcp \ -j REDIRECT --to $APPMESH_ENVOY_EGRESS_PORT # Apply APPMESH_EGRESS chain to non local traffic ipv6 ip6tables -t nat -A OUTPUT \ -p tcp \ -m addrtype ! --dst-type LOCAL \ -j APPMESH_EGRESS fi } function enable_ingress_redirect_routing() { # Route everything arriving at the application port to Envoy iptables -t nat -A APPMESH_INGRESS \ -p tcp \ -m multiport --dports "$APPMESH_APP_PORTS" \ -j REDIRECT --to-port "$APPMESH_ENVOY_INGRESS_PORT" # Apply AppMesh ingress chain to everything non-local iptables -t nat -A PREROUTING \ -p tcp \ -m addrtype ! --src-type LOCAL \ -j APPMESH_INGRESS if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then # Route everything arriving at the application port to Envoy ipv6 ip6tables -t nat -A APPMESH_INGRESS \ -p tcp \ -m multiport --dports "$APPMESH_APP_PORTS" \ -j REDIRECT --to-port "$APPMESH_ENVOY_INGRESS_PORT" # Apply AppMesh ingress chain to everything non-local ipv6 ip6tables -t nat -A PREROUTING \ -p tcp \ -m addrtype ! --src-type LOCAL \ -j APPMESH_INGRESS fi } function enable_routing() { echo "=== Enabling routing ===" enable_egress_routing if [ ! -z "$APPMESH_APP_PORTS" ]; then enable_ingress_redirect_routing fi } function disable_routing() { echo "=== Disabling routing ===" iptables -t nat -F APPMESH_INGRESS iptables -t nat -F APPMESH_EGRESS if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then ip6tables -t nat -F APPMESH_INGRESS ip6tables -t nat -F APPMESH_EGRESS fi } function dump_status() { echo "=== iptables FORWARD table ===" iptables -L -v -n echo "=== iptables NAT table ===" iptables -t nat -L -v -n if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then echo "=== ip6tables FORWARD table ===" ip6tables -L -v -n echo "=== ip6tables NAT table ===" ip6tables -t nat -L -v -n fi } function clean_up() { disable_routing ruleNum=$(iptables -L PREROUTING -t nat --line-numbers | grep APPMESH_INGRESS | cut -d " " -f 1) iptables -t nat -D PREROUTING $ruleNum ruleNum=$(iptables -L OUTPUT -t nat --line-numbers | grep APPMESH_EGRESS | cut -d " " -f 1) iptables -t nat -D OUTPUT $ruleNum iptables -t nat -X APPMESH_INGRESS iptables -t nat -X APPMESH_EGRESS if [ "$APPMESH_ENABLE_IPV6" == "1" ]; then ruleNum=$(ip6tables -L PREROUTING -t nat --line-numbers | grep APPMESH_INGRESS | cut -d " " -f 1) ip6tables -t nat -D PREROUTING $ruleNum ruleNum=$(ip6tables -L OUTPUT -t nat --line-numbers | grep APPMESH_EGRESS | cut -d " " -f 1) ip6tables -t nat -D OUTPUT $ruleNum ip6tables -t nat -X APPMESH_INGRESS ip6tables -t nat -X APPMESH_EGRESS fi } function main_loop() { echo "=== Entering main loop ===" while read -p '> ' cmd; do case "$cmd" in "quit") clean_up break ;; "status") dump_status ;; "enable") enable_routing ;; "disable") disable_routing ;; *) echo "Available commands: quit, status, enable, disable" ;; esac done } function print_config() { echo "=== Input configuration ===" env | grep APPMESH_ || true } print_config initialize if [ "$APPMESH_START_ENABLED" == "1" ]; then enable_routing fi main_loop-
To configure
iptables
rules to route application traffic to the Envoy proxy, run the script that you created in the previous step.sudo ./envoy-networking.sh
-
Start your virtual node application code.
Note
For more examples and walkthroughs for App Mesh, see the App Mesh examples
repository