# Architect your solution for Amazon ECS
You must architect your applications so that they can run on *containers*. A container is a standardized unit of software development that holds everything that your software application requires to run. This includes relevant code, runtime, system tools, and system libraries. Containers are created from a read-only template that's called an *image*. A container image, is a static artifact containing your application and its dependencies. Images are typically built from a Dockerfile. A Dockerfile is a plaintext file that contains the instructions for building a container. After they're built, these images are stored in a *registry* such as Amazon ECR where they can be downloaded from.
After you create and store your image, you create an Amazon ECS task definition. A *task definition* is a blueprint for your application. It is a text file in JSON format that describes the parameters and one or more containers that form your application. For example, you can use it to specify the image and parameters for the operating system, which containers to use, which ports to open for your application, and what data volumes to use with the containers in the task. The specific parameters available for your task definition depend on the needs of your specific application.
After you define your task definition, you deploy it as either a service or a task on your cluster. A *cluster* is a logical grouping of tasks or services that runs on the capacity infrastructure that is registered to a cluster.
A *task* is the instantiation of a task definition within a cluster. You can run a standalone task, or you can run a task as part of a service. You can use an Amazon ECS *service* to run and maintain your desired number of tasks simultaneously in an Amazon ECS cluster. How it works is that, if any of your tasks fail or stop for any reason, the Amazon ECS service scheduler launches another instance based on your task definition. It does this to replace it and thereby maintain your desired number of tasks in the service.
The *container agent* runs on each container instance within an Amazon ECS cluster. The agent sends information about the current running tasks and resource utilization of your containers to Amazon ECS. It starts and stops tasks whenever it receives a request from Amazon ECS.
After you deploy the task or service, you can use any of the following tools to monitor your deployment and application:
+ CloudWatch
+ Runtime Monitoring
## Capacity
The capacity is the infrastructure where your containers run. Amazon ECS offers three infrastructure types for your clusters:
+ Amazon ECS Managed Instances - AWS fully manages the underlying Amazon EC2 instances running in your account, including provisioning, patching, and scaling. This option provides the optimal balance of performance, cost-effectiveness, and operational simplicity.
+ Serverless (Fargate) - Pay only for the resources your tasks use without managing any infrastructure. Ideal for variable workloads and getting started quickly.
+ Amazon EC2 instances - You manage the underlying Amazon EC2 instances directly, including instance selection, configuration, and maintenance.
Use Amazon ECS Managed Instances when:
+ You want the simplicity of Fargate with more control over the underlying infrastructure.
+ You need predictable performance and cost optimization.
+ You want AWS to handle infrastructure management while maintaining flexibility.
Use Fargate when:
+ You want to focus on your application without managing infrastructure.
+ You have variable or unpredictable workloads.
+ You're getting started with containers and want the simplest deployment option.
Use Amazon EC2 instances when:
+ You need specialized hardware requirements (GPU acceleration, high-performance computing).
+ You require capacity reservations or specific instance types.
+ You need privileged capabilities or custom AMIs.
You specify the infrastructure when you create a cluster. You also specify the infrastructure type when you register a task definition. The task definition refers to the infrastructure as the "launch type". You can also use capacity providers.
## Service endpoints
The service endpoint is the URL of the entry point for Amazon ECS that you use to connect to the service programmatically using either Internet Protocol version 4 (IPv4) or Internet Protocol version 6 (IPv6). By default, requests that you make to connect to Amazon ECS programmatically use service endpoints that support only IPv4 requests. To connect programmatically with Amazon ECS using either IPv4 or IPv6 requests, you can use a dual-stack endpoint. For information about using a dual-stack endpoint, see [Using Amazon ECS dual-stack endpoints](dual-stack-endpoint.md).
## Networking
AWS resources are created in subnets. When you use EC2 instances, Amazon ECS launches the instances in the subnet that you specify when you create a cluster. Your tasks run in the instance subnet. For Fargate or on-premises virtual machines, you specify the subnet when you run a task or create a service.
Depending on your application, the subnet can be a private or public subnet and the subnet can be in any of the following AWS resources:
+ Availability Zones
+ Local Zones
+ Wavelength Zones
+ AWS Regions
+ AWS Outposts
For more information, see [Amazon ECS applications in shared subnets, Local Zones, and Wavelength Zones](cluster-regions-zones.md) or [Amazon Elastic Container Service on AWS Outposts](using-outposts.md).
You can have your application connect to the internet by using one of the following methods:
+ A public subnet with an internet gateway
Use public subnets when you have public applications that require large amounts of bandwidth or minimal latency. Applicable scenarios include video streaming and gaming services.
+ A private subnet with a NAT gateway
Use private subnets when you want to protect your containers from direct external access. Applicable scenarios include payment processing systems or containers storing user data and passwords.
+ AWS PrivateLink
Use AWS PrivateLink to have private connectivity between VPCs, AWS services, and your on-premises networks without exposing your traffic to the public internet.
## Feature access
You can use your Amazon ECS account settings to access the following features:
+ Container Insights
CloudWatch Container Insights collects, aggregates, and summarizes metrics and logs from your containerized applications and microservices. The metrics include utilization for resources such as CPU, memory, disk, and network.
+ `awsvpc` trunking
For certain EC2 instances types, you can have additional network interfaces (ENIs) available on newly launched container instances.
+ Tagging authorization
Users must have permissions for actions that create a resource, such as `ecsCreateCluster`. If tags are specified in the resource-creating action, AWS performs additional authorization on the `ecs:TagResource` action to verify if users or roles have permissions to create tags.
+ Fargate FIPS-140 compliance
Fargate supports the Federal Information Processing Standard (FIPS-140) which specifies the security requirements for cryptographic modules that protect sensitive information. It is the current United States and Canadian government standard, and is applicable to systems that are required to be compliant with Federal Information Security Management Act (FISMA) or Federal Risk and Authorization Management Program (FedRAMP).
+ Fargate task retirement time changes
You can configure the wait period before Fargate tasks are retired for patching.
+ Dual stack VPC
Allow tasks to communicate over IPv4, IPv6, or both.
+ Amazon Resource Name (ARN) format
Certain features, such as tagging authorization, require a new Amazon Resource Name (ARN) format.
For more information, see [Access Amazon ECS features with account settings](ecs-account-settings.md).
## IAM roles
An IAM role is an IAM identity that you can create in your account that has specific permissions. In Amazon ECS, you can create roles to grant permissions to Amazon ECS resource such as containers or services.
Some Amazon ECS features require roles. For more information, see [IAM roles for Amazon ECS](ecs-iam-role-overview.md).
## Logging
Logging and monitoring are important aspects of maintaining the reliability, availability, and performance of Amazon ECS workloads. The following options are available:
+ Amazon CloudWatch logs - route logs to Amazon CloudWatch
+ FireLens for Amazon ECS - route logs to an AWS service or AWS Partner Network destination for log storage and analysis. The AWS Partner Network is a global community of partners that leverages programs, expertise, and resources to build, market, and sell customer offerings.
## Container image best practices
The following are key principles for Amazon ECS container images:
+ Include all dependencies in the image
+ Run one process per container
+ Handle `SIGTERM` for graceful shutdowns
+ Write logs to `stdout` and `stderr`
+ Use unique tags, avoid `latest` in production
# Amazon Amazon ECS launch types and capacity providers
Amazon ECS provides two methods for configuring capacity for workloads. You can use launch types or capacity providers. Launch types include EC2, Fargate, and External. Capacity providers offer enhanced flexibility and advanced features for capacity management. You can run workloads on serverless compute with Fargate and Fargate Spot capacity providers, on self-managed EC2 instances through Auto Scaling group capacity providers, or on fully-managed compute using Amazon ECS Managed Instances capacity providers that combine the simplicity of Fargate with the flexibility of EC2 compute. Capacity providers offer better control over resource allocation and can help optimize both performance and costs. Capacity providers are the recommended way to configure capacity for workloads compared to traditional launch types. Use the following to understand the differences between capacity providers and launch types.
## Best practices
The following are the best practices:
Use launch types to define infrastructure compatibility
Launch types define the infrastructure on which tasks and services run. When you define tasks, specify `RequiresCompatibilities` to include one or more launch types that are compatible with tasks. You can use following launch types: EC2, Fargate, External, and Amazon ECS Managed Instances. While you can also use launch type to run your tasks or services, we recommend using the launch type only for defining compatibilities in your task definitions, and use capacity providers for launching tasks or services. Note that you can choose one or more launch types to define compatibilities for tasks.
Use capacity providers to configure compute capacity
When you launch tasks or services, configure a capacity provider strategy. Amazon ECS supports following capacity providers: Fargate and FARGATE\$1SPOT, Auto Scaling groups for self-managed EC2 instances, and Amazon ECS Managed Instances. Note that Spot Fleet is only available as a capacity provider and not as a launch type. You can create one or more Amazon ECS Managed Instances or Auto Scaling groups capacity providers in a cluster. Fargate and Fargate Spot capacity providers are created and managed by Amazon ECS on every cluster and you do not need to create them. A cluster can have a mix of all capacity provider types, however, a capacity provider strategy can't have a mix of different capacity provider types.
Update the capacity for services
You can simply update a capacity provider strategy for a service to move it from one compute type to the other.
## Service mutability
Amazon ECS supports updating services between different capacity providers. This allows for:
+ Seamless update from launch types to capacity providers
+ Transitions between different capacity provider types
+ Testing different compute options without service recreation
The following is a high-level overview of the process:
1. Update the task definition – Ensure `requiresCompatibilities` includes the target capacity provider, for example` MANAGED_INSTANCES`.
**Note**
Task definitions must pass compatibility validation for the target capacity provider. If the `requiresCompatibilities` check fails for the task definition version, the `UpdateService` call fails.
1. Create a capacity provider – If you use custom Amazon EC2 Auto Scaling groups, create the capacity provider.
1. Update the service – Modify the service to use a capacity provider strategy instead of the launch type.
1. Validate the deployment – Confirm that tasks deploy successfully.
1. Monitor and optimize – Adjust capacity provider settings as needed.
### Capacity provider to capacity provider
All capacity provider to capacity provider updates are supported:
+ Amazon EC2 Auto Scaling group capacity provider to Amazon ECS Managed Instances
+ Fargate capacity provider to Amazon ECS Managed Instances
+ Amazon EC2 Auto Scaling group capacity provider to Fargate capacity provider
+ Amazon ECS Managed Instances to Fargate capacity provider
+ Fargate capacity provider to Amazon EC2 Auto Scaling group capacity provider
+ Amazon ECS Managed Instances to Amazon EC2 Auto Scaling group capacity provider
### Launch type to capacity provider
All launch type to capacity provider updates are supported:
+ EC2 launch type to Amazon ECS Managed Instances
+ Fargate launch type to Amazon ECS Managed Instances
+ EC2 launch type to Fargate capacity provider
+ EC2 launch type to EC2 Auto Scaling group capacity provider
+ Fargate launch type to Amazon EC2 Auto Scaling group capacity provider
+ Fargate launch type to Fargate capacity provider
+ External launch type to Amazon ECS Managed Instances
+ External launch type to Fargate capacity provider
+ External launch type to Amazon EC2 Auto Scaling group capacity provider
### Launch type to launch type
Launch type to launch type updates are not supported:
+ EC2 launch type to Fargate launch type (use Fargate capacity provider instead)
+ Fargate launch type to EC2 launch type (use Amazon EC2 Auto Scaling group capacity provider instead)
Instead of migrating between launch types, migrate to the equivalent capacity provider for enhanced functionality and future compatibility.
### Capacity provider to launch type
Capacity provider to launch type updates are not supported. If you originally created a service with a launch type and then updated the service to use capacity providers, you can revert the service to its original launch type. To revert, pass an empty list as the `capacityProviderStrategy` when you call `UpdateService`.
JSON:
```
"capacityProviderStrategy": []
```
AWS CLI:
```
--capacity-provider-strategy '[]'
```
**Note**
This only reverts the service to the launch type specified when the service was originally created. You cannot use this method to switch a service to a different launch type.
**Note**
Task definitions must pass compatibility validation for the target capacity provider. If the `requiresCompatibilities` check fails for the task definition version, the `UpdateService` call fails.
# Using Amazon ECS dual-stack endpoints
Amazon ECS dual-stack endpoints support requests to Amazon ECS over Internet Protocol version 4 (IPv4) and Internet Protocol version 6 (IPv6). For a list of all Amazon ECS endpoints, see [Amazon ECS endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/ecs-service.html) in the AWS General Reference.
When using the REST API, you directly access an Amazon ECS endpoint by using the endpoint name (URI). Amazon ECS supports only regional dual-stack endpoint names, which means that you must specify the region as part of the name.
Use the following naming convention for the dual-stack endpoint names: `ecs.region.api.aws`.
When using the AWS Command Line Interface (AWS CLI) and AWS SDKs, you can use a parameter or a flag to change to a dual-stack endpoint. You can also specify the dual-stack endpoint directly as an override of the Amazon ECS endpoint in the config file.
The following sections describe how to use dual-stack endpoints from the AWS CLI, the AWS SDKs, and the REST API.
**Topics**
+ [
## Using dual-stack endpoints from the AWS CLI
](#dual-stack-endpoints-cli)
+ [
## Using dual-stack endpoints from the AWS SDKs
](#dual-stack-endpoints-sdks)
+ [
## Using dual-stack endpoints from the REST API
](#dual-stack-endpoints-examples-rest-api)
## Using dual-stack endpoints from the AWS CLI
This section provides examples of AWS CLI commands used to make requests to a dual-stack endpoint. For more information about installing the AWS CLI or updating to the latest version, see [Installing or updating to the latest version of the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide for Version 2*.
To use a dual-stack endpoint, you can set the configuration value `use_dualstack_endpoint` to `true` in the `config` file for the AWS CLI to direct all Amazon ECS requests made by the `ecs` AWS CLI command to the dual-stack endpoint for the specified region. You can specify the region in the `config` file or in a command by using the `--region` option. For more information about configuration files for the AWS CLI, see [Configuration and credential file settings in the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) in the *AWS Command Line Interface User Guide for Version 2*.
If you want to use a dual-stack endpoint for specific AWS CLI commands, you can use either of the following methods:
+ You can use the dual-stack endpoint per command by setting the `--endpoint-url` parameter to `https://ecs.aws-region.api.aws` or `http://ecs.aws-region.api.aws` for any `ecs` command.
The following example command lists all available clusters and uses the dual-stack endpoint for the request.
```
$ aws ecs list-clusters --endpoint-url https://ecs.aws-region.api.aws
```
+ You can set up separate profiles in your AWS Config file. For example, create one profile that sets `use_dualstack_endpoint` to `true` and a profile that does not set `use_dualstack_endpoint`. When you run a command, specify which profile you want to use, depending upon whether or not you want to use the dual-stack endpoint.
## Using dual-stack endpoints from the AWS SDKs
This section provides examples of how to access a dual-stack endpoint by using the AWS SDKs.
------
#### [ AWS SDK for Java 2.x ]
The following example shows how to specify a dual-stack endpoint for the `us-east-1` Region using the AWS SDK for Java 2.x.
```
Region region = Region.US_EAST_1
EcsClient client = EcsClient.builder().region(region).dualstackEnabled(true).build();
```
------
#### [ AWS SDK for Go ]
The following example shows how to specify a dual-stack endpoint for the `us-east-1` Region using the AWS SDK for Go.
```
sess := session.Must(session.NewSession())
svc := ecs.New(sess, &aws.Config{
Region: aws.String(endpoints.UsEast1RegionID),
Endpoint: aws.String("https://ecs.us-east-1.api.aws")
})
```
------
For more information, see [Dual-stack and FIPS endpoints](https://docs.aws.amazon.com/sdkref/latest/guide/feature-endpoints.html) in the *AWS SDKs and Tools Reference Guide*.
## Using dual-stack endpoints from the REST API
When using the REST API, you can directly access a dual-stack endpoint by specifying it in your request. The following example uses the dual-stack endpoint to list all Amazon ECS clusters in the `us-east-1` Region.
```
POST / HTTP/1.1
Host: ecs.us-east-1.api.aws
Accept-Encoding: identity
Content-Length: 2
X-Amz-Target: AmazonEC2ContainerServiceV20141113.ListClusters
X-Amz-Date: 20150429T170621Z
Content-Type: application/x-amz-json-1.1
Authorization: AUTHPARAMS
{}
```
# Amazon ECS applications in shared subnets, Local Zones, and Wavelength Zones
Amazon ECS supports workloads that use Local Zones, Wavelength Zones, and AWS Outposts for when low latency or local data processing is a requirement.
+ You can use Local Zones as an extension of an AWS Region to place resources in multiple locations closer to your end users.
+ You can use Wavelength Zones to build applications that deliver ultra-low latencies to 5G devices and end users. Wavelength deploys standard AWS compute and storage services to the edge of telecommunication carriers' 5G networks.
+ AWS Outposts brings native AWS services, infrastructure, and operating models to virtually any data center, co-location space, or on-premises facility.
**Important**
Amazon ECS on AWS Fargate workloads aren't supported in Local Zones, Wavelength Zones, or on AWS Outposts at this time.
For information about the differences between Local Zones, Wavelength Zones, and AWS Outposts , see [How should I think about when to use AWS Wavelength, AWS Local Zones, or AWS Outposts for applications requiring low latency or local data processing](https://aws.amazon.com/wavelength/faqs/) in the AWS Wavelength FAQs.
## Shared subnets
You can use *VPC sharing* to share subnets with other AWS accounts within the same AWS Organizations.
You can use shared VPCs for EC2 with the following considerations:
+ The owner of the VPC subnet must share a subnet with a participant account before that account can use it for Amazon ECS resources.
+ You can't use the VPC default security group for your container instances because it belongs to the owner. Additionally, participants can't launch instances using security groups that are owned by other participants or the owner.
+ In a shared subnet, the participant and the owner separately controls the security groups within each respective account. The subnet owner can see security groups that are created by the participants but cannot perform any actions on them. If the subnet owner wants to remove or modify these security groups, the participant that created the security group must take the action.
+ The shared VPC owner cannot view, update or delete a cluster that a participant creates in the shared subnet. This is in addition to the VPC resources that each account has different access to. For more information, see [Responsibilities and permissions for owners and participants](https://docs.aws.amazon.com//vpc/latest/userguide/vpc-sharing.html#vpc-share-limitations) in the *Amazon VPC User Guide*.
You can use shared VPCs for Fargate with the following considerations::
+ The owner of the VPC subnet must share a subnet with a participant account before that account can use it for Amazon ECS resources.
+ You can't create a service or run a task using the default security group for the VPC because it belongs to the owner. Additionally, participants can't create a service or run a task using security groups that are owned by other participants or the owner.
+ In a shared subnet, the participant and the owner separately controls the security groups within each respective account. The subnet owner can see security groups that are created by the participants but cannot perform any actions on them. If the subnet owner wants to remove or modify these security groups, the participant that created the security group must take the action.
+ The shared VPC owner cannot view, update or delete a cluster that a participant creates in the shared subnet. This is in addition to the VPC resources that each account has different access to. For more information, see [Responsibilities and permissions for owners and participants](https://docs.aws.amazon.com//vpc/latest/userguide/vpc-sharing.html#vpc-share-limitations) in the *Amazon VPC User Guide*.
For more information about VPC subnet sharing, see [Share your VPC with other accounts](https://docs.aws.amazon.com//vpc/latest/userguide/vpc-sharing.html#vpc-share-limitations) in the *Amazon VPC User Guide*.
## Local Zones
A *Local Zone* is an extension of an AWS Region in close geographic proximity to your users. Local Zones have their own connections to the internet and support Direct Connect. Resources that are created in a Local Zone can serve local users with low-latency communications. For more information, see [AWS Local Zones](https://aws.amazon.com/about-aws/global-infrastructure/localzones/).
A Local Zone is represented by a Region code followed by an identifier that indicates the location (for example, `us-west-2-lax-1a`).
To use a Local Zone, you must opt in to the zone. After you opt in, you must create an Amazon VPC and subnet in the Local Zone.
You can launch Amazon EC2 instances, Amazon FSx file servers, and Application Load Balancers to use for your Amazon ECS clusters and tasks.
For more information, see [What is AWS Local Zones?](https://docs.aws.amazon.com/local-zones/latest/ug/what-is-aws-local-zones.html) in the *AWS Local Zones User Guide*.
## Wavelength Zones
You can use *AWS Wavelength* to build applications that deliver ultra-low latency to mobile devices and end users. Wavelength deploys standard AWS compute and storage services to the edge of telecommunication carriers' 5G networks. You can extend an Amazon Virtual Private Cloud to one or more Wavelength Zones. Then, you can use AWS resources such as Amazon EC2 instances to run applications that require ultra-low latency and a connection to AWS services in the Region.
A Wavelength Zone is an isolated Zone in the carrier location where the Wavelength infrastructure is deployed. Wavelength Zones are tied to an AWS Region. A Wavelength Zone is a logical extension of a Region, and is managed by the control plane in the Region.
A Wavelength Zone is represented by a Region code followed by an identifier that indicates the Wavelength Zone (for example, `us-east-1-wl1-bos-wlz-1`).
To use a Wavelength Zone, you must opt in to the Zone. After you opt in, you must create an Amazon VPC and subnet in the Wavelength Zone. Then, you can launch your Amazon EC2 instances in the Zone to use for your Amazon ECS clusters and tasks.
For more information, see [Get started with AWS Wavelength](https://docs.aws.amazon.com/wavelength/latest/developerguide/get-started-wavelength.html) in the *AWS Wavelength Developer Guide*.
Wavelength Zones aren't available in all AWS Regions. For information about the Regions that support Wavelength Zones, see [Available Wavelength Zones](https://docs.aws.amazon.com/wavelength/latest/developerguide/available-wavelength-zones.html) in the *AWS Wavelength Developer Guide*.
# Amazon Elastic Container Service on AWS Outposts
AWS Outposts enables native AWS services, infrastructure, and operating models in on-premises facilities. In AWS Outposts environments, you can use the same AWS APIs, tools, and infrastructure that you use in the AWS Cloud.
Amazon ECS on AWS Outposts is ideal for low-latency workloads that need to be run in close proximity to on-premises data and applications.
For more information about AWS Outposts, see the [https://docs.aws.amazon.com/outposts/latest/userguide/what-is-outposts.html](https://docs.aws.amazon.com/outposts/latest/userguide/what-is-outposts.html).
## Considerations
The following are considerations of using Amazon ECS on AWS Outposts:
+ Amazon Elastic Container Registry, AWS Identity and Access Management, and Network Load Balancer run in the AWS Region, not on AWS Outposts. This will increase latencies between these services and the containers.
+ AWS Fargate is not available on AWS Outposts.
The following are network connectivity considerations for AWS Outposts:
+ If network connectivity between your AWS Outposts and its AWS Region is lost, your clusters will continue to run. However, you cannot create new clusters or take new actions on existing clusters until connectivity is restored. In case of instance failures, the instance will not be automatically replaced. The CloudWatch Logs agent will be unable to update logs and event data.
+ We recommend that you provide reliable, highly available, and low latency connectivity between your AWS Outposts and its AWS Region.
## Prerequisites
The following are prerequisites for using Amazon ECS on AWS Outposts:
+ You must have installed and configured an Outpost in your on-premises data center.
+ You must have a reliable network connection between your Outpost and its AWS Region.
## Overview of cluster creation on AWS Outposts
The following is an overview of the configuration:
1. Create a role and policy with rights on AWS Outposts.
1. Create an IAM instance profile with rights on AWS Outposts.
1. Create a VPC, or use an existing one that is in the same Region as your AWS Outposts.
1. Create a subnet or use an existing one that is associated with the AWS Outposts.
This is the subnet where the container instances run.
1. Create a security group for the container instances in your cluster.
1. Create an Amazon ECS cluster.
1. Define the Amazon ECS container agent environment variables to launch the instance into the cluster.
1. Run a container.
For detailed information about how to integrate Amazon ECS with AWS Outposts, see [Extend Amazon ECS across two AWS Outposts racks](https://community.aws/content/2k5wK9P1oSC9I4ZzuSLWynsiJaa/extend-amazon-ecs-across-two-outposts-racks).
The following example creates an Amazon ECS cluster on an AWS Outposts.
1. Create a role and policy with rights on AWS Outposts.
The `role-policy.json` file is the policy document that contains the effect and actions for resources. For information about the file format, see [PutRolePolicy](https://docs.aws.amazon.com/IAM/latest/APIReference/API_PutRolePolicy.html) in the *IAM API Reference*
```
aws iam create-role –-role-name ecsRole \
--assume-role-policy-document file://ecs-policy.json
aws iam put-role-policy --role-name ecsRole --policy-name ecsRolePolicy \
--policy-document file://role-policy.json
```
1. Create an IAM instance profile with rights on AWS Outposts.
```
aws iam create-instance-profile --instance-profile-name outpost
aws iam add-role-to-instance-profile --instance-profile-name outpost \
--role-name ecsRole
```
1. Create a VPC.
```
aws ec2 create-vpc --cidr-block 10.0.0.0/16
```
1. Create a subnet associated with your AWS Outposts.
```
aws ec2 create-subnet \
--cidr-block 10.0.3.0/24 \
--vpc-id vpc-xxxxxxxx \
--outpost-arn arn:aws:outposts:us-west-2:123456789012:outpost/op-xxxxxxxxxxxxxxxx \
--availability-zone-id usw2-az1
```
1. Create a security group for the container instances, specifying the proper CIDR range for the AWS Outposts. (This step is different for AWS Outposts.)
```
aws ec2 create-security-group --group-name MyOutpostSG
aws ec2 authorize-security-group-ingress --group-name MyOutpostSG --protocol tcp \
--port 22 --cidr 10.0.3.0/24
aws ec2 authorize-security-group-ingress --group-name MyOutpostSG --protocol tcp \
--port 80 --cidr 10.0.3.0/24
```
1. Create the Cluster.
1. Define the Amazon ECS container agent environment variables to launch the instance into the cluster created in the previous step and define any tags you want to add to help identify the cluster (for example, `Outpost` to indicate that the cluster is for an Outpost).
```
#! /bin/bash
cat << ‘EOF’ >> /etc/ecs/ecs.config
ECS_CLUSTER=MyCluster
ECS_IMAGE_PULL_BEHAVIOR=prefer-cached
ECS_CONTAINER_INSTANCE_TAGS={“environment”: ”Outpost”}
EOF
```
**Note**
In order to avoid delays caused by pulling container images from Amazon ECR in the Region, use image caches. To do this, each time a task is run, configure the Amazon ECS agent to default to using the cached image on the instance itself by setting `ECS_IMAGE_PULL_BEHAVIOR` to `prefer-cached`.
1. Create the container instance, specifying the VPC and subnet for the AWS Outposts where this instance should run and an instance type that is available on the AWS Outposts. (This step is different for AWS Outposts.)
The `userdata.txt` file contains the user data the instance can use to perform common automated configuration tasks and even run scripts after the instance starts. For information about the file for API calls, see [Run commands on your Linux instance at launch](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html) in the *Amazon EC2 User Guide*.
```
aws ec2 run-instances --count 1 --image-id ami-xxxxxxxx --instance-type c5.large \
--key-name aws-outpost-key –-subnet-id subnet-xxxxxxxxxxxxxxxxx \
--iam-instance-profile Name outpost --security-group-id sg-xxxxxx \
--associate-public-ip-address --user-data file://userdata.txt
```
**Note**
This command is also used when adding additional instances to the cluster. Any containers deployed in the cluster will be placed on that specific AWS Outposts.
# Best practices for Amazon ECS container images
A container image is a set of instructions on how to build the container. A container image holds your application code and all the dependencies that your application code requires to run. Application dependencies include the source code packages that your application code relies on, a language runtime for interpreted languages, and binary packages that your dynamically linked code relies on.
Use the following guidelines when you design and build your container images:
+ Make your container images complete by storing all application dependencies as static files inside the container image.
If you change something in the container image, build a new container image with the changes.
+ Run a single application process within a container.
The container lifetime is as long as the application process runs. Amazon ECS replaces crashed processes and determines where to launch the replacement process. A complete image makes the overall deployment more resilient.
+ Make your application handle `SIGTERM`.
When Amazon ECS stops a task, it first sends a SIGTERM signal to the task to notify the application that it needs to finish and shut down. Amazon ECS then sends a `SIGKILL` message. When applications ignore the `SIGTERM`, the Amazon ECS service must wait to send the `SIGKILL` signal to terminate the process.
You need to identify how long it takes your application to complete its work, and ensure that your applications handles the `SIGTERM` signal. The application's signal handling needs to stop the application from taking new work and complete the work that is in-progress, or save unfinished work to storage outside of the task when the work takes too long to complete.
+ Configure containerized applications to write logs to `stdout` and `stderr`.
Decoupling log handling from your application code gives you flexibility to adjust log handling at the infrastructure level. One example of this is to change your logging system. Instead of modifying your services, and building and deploying a new container image, you can adjust the settings.
+ Use tags to version your container images.
Container images are stored in a container registry. Each image in a registry is identified by a tag. There's a tag called `latest`. This tag functions as a pointer to the latest version of the application container image, similar to the `HEAD` in a git repository. We recommend that you use the `latest` tag only for testing purposes. As a best practice, tag container images with a unique tag for each build. We recommend that you tag your images using the git SHA for the git commit that was used to build the image.
You don’t need to build a container image for every commit. However, we recommend that you build a new container image each time you release a particular code commit to the production environment. We also recommend that you tag the image with a tag that corresponds to the git commit of the code that's inside the image. If you tagged the image with the git commit, you can more quickly find which version of the code the image is running.
We also recommend that you turn on immutable image tags in Amazon Elastic Container Registry. With this setting, you can't change the container image that a tag points at. Instead Amazon ECR enforces that a new image must be uploaded to a new tag. For more information, see [Image tag mutability](https://docs.aws.amazon.com/AmazonECR/latest/userguide/image-tag-mutability.html) in the *Amazon ECR User Guide*.
When you architect your application to run on AWS Fargate, you must decide between deploying multiple containers into the same task definition and deploying containers separately in multiple task definitions. If the following conditions are required, we recommend deploying multiple containers into the same task definition:
+ Your containers share a common lifecycle (that is, they're launched and terminated together).
+ Your containers must run on the same underlying host (that is, one container references the other on a localhost port).
+ Your containers share resources.
+ Your containers share data volumes.
If these conditions aren't required, we recommend deploying containers separately in multiple task definitions. This allows you to scale, provision, and deprovision the containers separately.
# Amazon ECS networking best practices
Modern applications are typically built out of multiple distributed components that communicate with each other. For example, a mobile or web application might communicate with an API endpoint, and the API might be powered by multiple microservices that communicate over the internet.
For information about the best practices for connection applications to the internet, see [Connect Amazon ECS applications to the internet](networking-outbound.md).
For information about the best practices for receiving inbound connections to Amazon ECS from the internet, see [Best practices for receiving inbound connections to Amazon ECS from the internet](networking-inbound.md).
For information about the best practices for connecting Amazon ECS to other AWS services, see [Best practices for connecting Amazon ECS to AWS services from inside your VPC](networking-connecting-vpc.md).
For information about the best practices for connecting services within a VPC, see [Best practices for connecting Amazon ECS services in a VPC](networking-connecting-services.md).
For information about the best practices for networking services across AWS accounts and VPCs, see [Best practices for networking Amazon ECS services across AWS accounts and VPCs](networking-connecting-services-crossaccount.md).
For information about the best practices for services to trobleshoot networking issues, see [AWS services for Amazon ECS networking troubleshooting](networking-troubleshooting.md).
# Connect Amazon ECS applications to the internet
Most containerized applications have a least some components that need outbound access to the internet. For example, the backend for a mobile app requires outbound access to push notifications.
Amazon Virtual Private Cloud has two main methods for facilitating communication between your VPC and the internet.
## Public subnet and internet gateway
![\[Diagram showing architecture of a public subnet connected to an internet gateway.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/public-network.png)
When you use a public subnet that has a route to an internet gateway, your containerized application can run on a host inside a VPC on a public subnet. The host that runs your container is assigned a public IP address. This public IP address is routable from the internet. For more information, see [Internet gateways](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html) in the *Amazon VPC User Guide*.
This network architecture facilitates direct communication between the host that runs your application and other hosts on the internet. The communication is bi-directional. This means that not only can you establish an outbound connection to any other host on the internet, but other hosts on the internet might also attempt to connect to your host. Therefore, you should pay close attention to your security group and firewall rules. This ensures that other hosts on the internet can’t open any connections that you don't want to be opened.
For example, if your application runs on Amazon EC2, make sure that port 22 for SSH access is not open. Otherwise, your instance could receive constant SSH connection attempts from malicious bots on the internet. These bots trawl through public IP addresses. After they find an open SSH port, they attempt to brute-force passwords to try to access your instance. Because of this, many organizations limit the usage of public subnets and prefer to have most, if not all, of their resources inside of private subnets.
Using public subnets for networking is suitable for public applications that require large amounts of bandwidth or minimal latency. Applicable use cases include video streaming and gaming services.
This networking approach is supported both when you use Amazon ECS on Amazon EC2 and when you use it on AWS Fargate.
+ Amazon EC2 — You can launch EC2 instances on a public subnet. Amazon ECS uses these EC2 instances as cluster capacity, and any containers that are running on the instances can use the underlying public IP address of the host for outbound networking. This applies to both the `host` and `bridge` network modes. However, the `awsvpc` network mode doesn't provide task ENIs with public IP addresses. Therefore, they can’t make direct use of an internet gateway.
+ Fargate — When you create your Amazon ECS service, specify public subnets for the networking configuration of your service, and use the **Assign public IP address** option. Each Fargate task is networked in the public subnet, and has its own public IP address for direct communication with the internet.
## Private subnet and NAT gateway
![\[Diagram showing architecture of a private subnet connected to a NAT gateway.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/private-network.png)
When you use a private subnet and a NAT gateway, you can run your containerized application on a host that's in a private subnet. As such, this host has a private IP address that's routable inside your VPC, but isn't routable from the internet. This means that other hosts inside the VPC can connect to the host using its private IP address, but other hosts on the internet can't make any inbound communications to the host.
With a private subnet, you can use a Network Address Translation (NAT) gateway to allow a host inside a private subnet to connect to the internet. Hosts on the internet receive an inbound connection that appears to be coming from the public IP address of the NAT gateway that's inside a public subnet. The NAT gateway is responsible for serving as a bridge between the internet and the private subnet. This configuration is often preferred for security reasons because it means that your VPC is protected from direct access by attackers on the internet. For more information, see [NAT gateways](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) in the *Amazon VPC User Guide*.
This private networking approach is suitable for scenarios where you want to protect your containers from direct external access. Applicable scenarios include payment processing systems or containers storing user data and passwords. You're charged for creating and using a NAT gateway in your account. NAT gateway hourly usage and data processing rates also apply. For redundancy purposes, you should have a NAT gateway in each Availability Zone. This way, the loss in availability of a single Availability Zone doesn't compromise your outbound connectivity. Because of this, if you have a small workload, it might be more cost effective to use private subnets and NAT gateways.
This networking approach is supported both when using Amazon ECS on Amazon EC2 and when using it on AWS Fargate.
+ Amazon EC2 — You can launch EC2 instances on a private subnet. The containers that run on these EC2 hosts use the underlying hosts networking, and outbound requests go through the NAT gateway.
+ Fargate — When you create your Amazon ECS service, specify private subnets for the networking configuration of your service, and don't use the **Assign public IP address** option. Each Fargate task is hosted in a private subnet. Its outbound traffic is routed through any NAT gateway that you have associated with that private subnet.
# Best practices for receiving inbound connections to Amazon ECS from the internet
If you run a public service, you must accept inbound traffic from the internet. For example, your public website must accept inbound HTTP requests from browsers. In such case, other hosts on the internet must also initiate an inbound connection to the host of your application.
One approach to this problem is to launch your containers on hosts that are in a public subnet with a public IP address. However, we don't recommend this for large-scale applications. For these, a better approach is to have a scalable input layer that sits between the internet and your application. For this approach, you can use any of the AWS services listed in this section as an input.
## Application Load Balancer
An Application Load Balancer functions at the application layer. It's the seventh layer of the Open Systems Interconnection (OSI) model. This makes an Application Load Balancer suitable for public HTTP services. If you have a website or an HTTP REST API, then an Application Load Balancer is a suitable load balancer for this workload. For more information, see [What is an Application Load Balancer?](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html) in the *User Guide for Application Load Balancers*.
![\[Diagram showing architecture of a network using an Application Load Balancer.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/alb-ingress.png)
With this architecture, you create an Application Load Balancer in a public subnet so that it has a public IP address and can receive inbound connections from the internet. When the Application Load Balancer receives an inbound connection, or more specifically an HTTP request, it opens a connection to the application using its private IP address. Then, it forwards the request over the internal connection.
An Application Load Balancer has the following advantages.
+ SSL/TLS termination — An Application Load Balancer can sustain secure HTTPS communication and certificates for communications with clients. It can optionally terminate the SSL connection at the load balancer level so that you don’t have to handle certificates in your own application.
+ Advanced routing — An Application Load Balancer can have multiple DNS hostnames. It also has advanced routing capabilities to send incoming HTTP requests to different destinations based on metrics such as the hostname or the path of the request. This means that you can use a single Application Load Balancer as the input for many different internal services, or even microservices on different paths of a REST API.
+ gRPC support and websockets — An Application Load Balancer can handle more than just HTTP. It can also load balance gRPC and websocket based services, with HTTP/2 support.
+ Security — An Application Load Balancer helps protect your application from malicious traffic. It includes features such as HTTP de sync mitigations, and is integrated with AWS Web Application Firewall (AWS WAF). AWS WAF can further filter out malicious traffic that might contain attack patterns, such as SQL injection or cross-site scripting.
## Network Load Balancer
A Network Load Balancer functions at the fourth layer of the Open Systems Interconnection (OSI) model. It's suitable for non-HTTP protocols or scenarios where end-to-end encryption is necessary, but doesn’t have the same HTTP-specific features of an Application Load Balancer. Therefore, a Network Load Balancer is best suited for applications that don’t use HTTP. For more information, see [What is a Network Load Balancer?](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) in the *User Guide for Network Load Balancers*.
![\[Diagram showing architecture of a network using an Network Load Balancer.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/nlbingress.png)
When a Network Load Balancer is used as an input, it functions similarly to an Application Load Balancer. This is because it's created in a public subnet and has a public IP address that can be accessed on the internet. The Network Load Balancer then opens a connection to the private IP address of the host running your container, and sends the packets from the public side to the private side.
**Network Load Balancer features**
Because the Network Load Balancer operates at a lower level of the networking stack, it doesn't have the same set of features that Application Load Balancer does. However, it does have the following important features.
+ End-to-end encryption — Because a Network Load Balancer operates at the fourth layer of the OSI model, it doesn't read the contents of packets. This makes it suitable for load balancing communications that need end-to-end encryption.
+ TLS encryption — In addition to end-to-end encryption, Network Load Balancer can also terminate TLS connections. This way, your backend applications don’t have to implement their own TLS.
+ UDP support — Because a Network Load Balancer operates at the fourth layer of the OSI model, it's suitable for non HTTP workloads and protocols other than TCP.
**Closing connections**
Because the Network Load Balancer does not observe the application protocol at the higher layers of the OSI model, it cannot send closure messages to the clients in those protocols. Unlike the Application Load Balancer, those connections need to be closed by the application or you can configure the Network Load Balancer to close the fourth layer connections when a task is stopped or replaced. See the connection termination setting for Network Load Balancer target groups in the [Network Load Balancer documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#deregistration-delay).
Letting the Network Load Balancer close connections at the fourth layer can cause clients to display undesired error messages, if the client does not handle them. See the Builders Library for more information on recommended client configuration [here](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter).
The methods to close connections will vary by application, however one way is to ensure that the Network Load Balancer target deregistration delay is longer than client connection timeout. The client would timeout first and reconnect gracefully through the Network Load Balancer to the next task while the old task slowly drains all of its clients. For more information about the Network Load Balancer target deregistration delay, see the [Network Load Balancer documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#deregistration-delay).
## Amazon API Gateway HTTP API
Amazon API Gateway is suitable for HTTP applications with sudden bursts in request volumes or low request volumes. For more information, see [What is Amazon API Gateway?](https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html) in the *API Gateway Developer Guide*.
![\[Diagram showing architecture of a network using API Gateway.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/apigateway-ingress.png)
The pricing model for both Application Load Balancer and Network Load Balancer include an hourly price to keep the load balancers available for accepting incoming connections at all times. In contrast, API Gateway charges for each request separately. This has the effect that, if no requests come in, there are no charges. Under high traffic loads, an Application Load Balancer or Network Load Balancer can handle a greater volume of requests at a cheaper per-request price than API Gateway. However, if you have a low number of requests overall or have periods of low traffic, then the cumulative price for using the API Gateway should be more cost effective than paying a hourly charge to maintain a load balancer that's being underutilized. The API Gateway can also cache API responses, which might result in lower backend request rates.
API Gateway functions which use a VPC link that allows the AWS managed service to connect to hosts inside the private subnet of your VPC, using its private IP address. It can detect these private IP addresses by looking at AWS Cloud Map service discovery records that are managed by Amazon ECS Service Discovery.
API Gateway supports the following features.
+ The API Gateway operation is similar to a load balancer, but has additional capabilities unique to API management
+ The API Gateway provides additional capabilities around client authorization, usage tiers, and request/response modification. For more information, see [Amazon API Gateway features](https://aws.amazon.com//api-gateway/features/).
+ The API Gateway can support edge, regional, and private API gateway endpoints. Edge endpoints are available through a managed CloudFront distribution. Regional and private endpoints are both local to a Region.
+ SSL/TLS termination
+ Routing different HTTP paths to different backend microservices
Besides the preceding features, API Gateway also supports using custom Lambda authorizers that you can use to protect your API from unauthorized usage. For more information, see [Field Notes: Serverless Container-based APIs with Amazon ECS and Amazon API Gateway](https://aws.amazon.com/blogs/architecture/field-notes-serverless-container-based-apis-with-amazon-ecs-and-amazon-api-gateway/).
# Best practices for connecting Amazon ECS to AWS services from inside your VPC
For Amazon ECS to function properly, the Amazon ECS container agent that runs on each host must communicate with the Amazon ECS control plane. If you're storing your container images in Amazon ECR, the Amazon EC2 hosts must communicate to the Amazon ECR service endpoint, and to Amazon S3, where the image layers are stored. If you use other AWS services for your containerized application, such as persisting data stored in DynamoDB, double-check that these services also have the necessary networking support.
## NAT gateway
Using a NAT gateway is the easiest way to ensure that your Amazon ECS tasks can access other AWS services. For more information about this approach, see [Private subnet and NAT gateway](networking-outbound.md#networking-private-subnet).
![\[Diagram showing architecture of a network using a NAT gateway.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/natgateway.png)
The following are the disadvantages to using this approach:
+ You can't limit what destinations the NAT gateway can communicate with. You also can't limit which destinations your backend tier can communicate to without disrupting all outbound communications from your VPC.
+ NAT gateways charge for every GB of data that passes through. If you use the NAT gateway for any of the following operations, you're charged for every GB of bandwidth:
+ Downloading large files from Amazon S3
+ Doing a high volume of database queries to DynamoDB
+ Pulling images from Amazon ECR
Additionally, NAT gateways support 5 Gbps of bandwidth and automatically scale up to 45 Gbps. If you route through a single NAT gateway, applications that require very high bandwidth connections might encounter networking constraints. As a workaround, you can divide your workload across multiple subnets and give each subnet its own NAT gateway.
## AWS PrivateLink
AWS PrivateLink provides private connectivity between VPCs, AWS services, and your on-premises networks without exposing your traffic to the public internet.
A VPC endpoint allows private connections between your VPC and supported AWS services and VPC endpoint services. Traffic between your VPC and the other service doesn't leave the Amazon network. A VPC endpoint doesn't require an internet gateway, virtual private gateway, NAT device, VPN connection, or Direct Connect connection. Amazon EC2 instances in your VPC don't require public IP addresses to communicate with resources in the service.
The following diagram shows how communication to AWS services works when you are using VPC endpoints instead of an internet gateway. AWS PrivateLink provisions elastic network interfaces (ENIs) inside of the subnet, and VPC routing rules are used to send any communication to the service hostname through the ENI, directly to the destination AWS service. This traffic no longer needs to use the NAT gateway or internet gateway.
![\[Diagram showing architecture of a network using AWS PrivateLink\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/endpointaccess-multiple.png)
The following are some of the common VPC endpoints that are used with the Amazon ECS service.
+ [Gateway endpoints for Amazon S3](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html)
+ [DynamoDB VPC endpoint](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/vpc-endpoints-dynamodb.html)
+ [Amazon ECS VPC endpoint](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/vpc-endpoints.html)
+ [Amazon ECR VPC endpoint](https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html)
Many other AWS services support VPC endpoints. If you make heavy usage of any AWS service, you should look up the specific documentation for that service and how to create a VPC endpoint for that traffic.
# Best practices for connecting Amazon ECS services in a VPC
Using Amazon ECS tasks in a VPC, you can split monolithic applications into separate parts that can be deployed and scaled independently in a secure environment. This architecture is called service-oriented architecture (SOA) or microservices. However, it can be challenging to make sure that all of these parts, both in and outside of a VPC, can communicate with each other. There are several approaches for facilitating communication, all with different advantages and disadvantages.
## Using Service Connect
We recommend Service Connect, which provides Amazon ECS configuration for service discovery, connectivity, and traffic monitoring. With Service Connect, your applications can use short names and standard ports to connect to services in the same cluster, other clusters, including across VPCs in the same Region. For more information, see [Amazon ECS Service Connect](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-connect.html).
When you use Service Connect, Amazon ECS manages all of the parts of service discovery: creating the names that can be discovered, dynamically managing entries for each task as the tasks start and stop, running an agent in each task that is configured to discover the names. Your application can look up the names by using the standard functionality for DNS names and making connections. If your application does this already, you don't need to modify your application to use Service Connect.
![\[Diagram showing architecture of a network using service connect.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/serviceconnect.png)
**Changes only happen during deployments**
You provide the complete configuration inside each service and task definition. Amazon ECS manages changes to this configuration in each service deployment, to ensure that all tasks in a deployment behave in the same way. For example, a common problem with DNS as service discovery is controlling a migration. If you change a DNS name to point to the new replacement IP addresses, it might take the maximum TTL time before all the clients begin using the new service. With Service Connect, the client deployment updates the configuration by replacing the client tasks. You can configure the deployment circuit breaker and other deployment configuration to affect Service Connect changes in the same way as any other deployment.
## Using service discovery
Another approach for service-to-service communication is direct communication using service discovery. In this approach, you can use the AWS Cloud Map service discovery integration with Amazon ECS. Using service discovery, Amazon ECS syncs the list of launched tasks to AWS Cloud Map, which maintains a DNS hostname that resolves to the internal IP addresses of one or more tasks from that particular service. Other services in the Amazon VPC can use this DNS hostname to send traffic directly to another container using its internal IP address. For more information, see [Service discovery](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html).
![\[Diagram showing architecture of a network using service discovery.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/servicediscovery.png)
In the preceding diagram, there are three services. `service-a-local` has one container and communicates with `service-b-local`, which has two containers. `service-b-local` must also communicate with `service-c-local`, which has one container. Each container in all three of these services can use the internal DNS names from AWS Cloud Map to find the internal IP addresses of a container from the downstream service that it needs to communicate to.
This approach to service-to-service communication provides low latency. At first glance, it's also simple as there are no extra components between the containers. Traffic travels directly from one container to the other container.
This approach is suitable when using the `awsvpc` network mode, where each task has its own unique IP address. Most software only supports the use of DNS `A` records, which resolve directly to IP addresses. When using the `awsvpc` network mode, the IP address for each task are an `A` record. However, if you're using `bridge` network mode, multiple containers could be sharing the same IP address. Additionally, dynamic port mappings cause the containers to be randomly assigned port numbers on that single IP address. At this point, an `A` record is no longer be enough for service discovery. You must also use an `SRV` record. This type of record can keep track of both IP addresses and port numbers but requires that you configure applications appropriately. Some prebuilt applications that you use might not support `SRV` records.
Another advantage of the `awsvpc` network mode is that you have a unique security group for each service. You can configure this security group to allow incoming connections from only the specific upstream services that need to talk to that service.
The main disadvantage of direct service-to-service communication using service discovery is that you must implement extra logic to have retries and deal with connection failures. DNS records have a time-to-live (TTL) period that controls how long they are cached for. It takes some time for the DNS record to be updated and for the cache to expire so that your applications can pick up the latest version of the DNS record. So, your application might end up resolving the DNS record to point at another container that's no longer there. Your application needs to handle retries and have logic to ignore bad backends.
## Using an internal load balancer
Another approach to service-to-service communication is to use an internal load balancer. An internal load balancer exists entirely inside of your VPC and is only accessible to services inside of your VPC.
![\[Diagram showing architecture of a network using an internal load balancer.\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/images/loadbalancer-internal.png)
The load balancer maintains high availability by deploying redundant resources into each subnet. When a container from `serviceA` needs to communicate with a container from `serviceB`, it opens a connection to the load balancer. The load balancer then opens a connection to a container from `service B`. The load balancer serves as a centralized place for managing all connections between each service.
If a container from `serviceB` stops, then the load balancer can remove that container from the pool. The load balancer also does health checks against each downstream target in its pool and can automatically remove bad targets from the pool until they become healthy again. The applications no longer need to be aware of how many downstream containers there are. They just open their connections to the load balancer.
This approach is advantageous to all network modes. The load balancer can keep track of task IP addresses when using the `awsvpc` network mode, as well as more advanced combinations of IP addresses and ports when using the `bridge` network mode. It evenly distributes traffic across all the IP address and port combinations, even if several containers are actually hosted on the same Amazon EC2 instance, just on different ports.
The one disadvantage of this approach is cost. To be highly available, the load balancer needs to have resources in each Availability Zone. This adds extra cost because of the overhead of paying for the load balancer and for the amount of traffic that goes through the load balancer.
However, you can reduce overhead costs by having multiple services share a load balancer. This is particularly suitable for REST services that use an Application Load Balancer. You can create path-based routing rules that route traffic to different services. For example, `/api/user/*` might route to a container that's part of the `user` service, whereas `/api/order/*` might route to the associated `order` service. With this approach, you only pay for one Application Load Balancer, and have one consistent URL for your API. However, you can split the traffic off to various microservices on the backend.
# Best practices for networking Amazon ECS services across AWS accounts and VPCs
If you're part of an organization with multiple teams and divisions, you probably deploy services independently into separate VPCs inside a shared AWS account or into VPCs that are associated with multiple individual AWS accounts. No matter which way you deploy your services, we recommend that you supplement your networking components to help route traffic between VPCs. For this, several AWS services can be used to supplement your existing networking components.
+ AWS Transit Gateway — You should consider this networking service first. This service serves as a central hub for routing your connections between Amazon VPCs, AWS accounts, and on-premises networks. For more information, see [What is a transit gateway?](https://docs.aws.amazon.com/vpc/latest/tgw/what-is-transit-gateway.html) in the *Amazon VPC Transit Gateways Guide*.
+ Amazon VPC and VPN support — You can use this service to create site-to-site VPN connections for connecting on-premises networks to your VPC. For more information, see [What is AWS Site-to-Site VPN?](https://docs.aws.amazon.com/vpn/latest/s2svpn/VPC_VPN.html) in the *AWS Site-to-Site VPN User Guide*.
+ Amazon VPC — You can use Amazon VPC peering to help you to connect multiple VPCs, either in the same account, or across accounts. For more information, see [What is VPC peering?](https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html) in the *Amazon VPC Peering Guide*.
+ Shared VPCs — You can use a VPC and VPC subnets across multiple AWS accounts. For more information, see [Working with shared VPCs](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-sharing.html) in the *Amazon VPC User Guide*.
# AWS services for Amazon ECS networking troubleshooting
The following services and features can help you to gain insights about your network and service configurations. You can use this information to troubleshoot networking issues and better optimize your services.
## CloudWatch Container Insights
CloudWatch Container Insights collects, aggregates, and summarizes metrics and logs from your containerized applications and microservices. Metrics include the utilization of resources such as CPU, memory, disk, and network. They're available in CloudWatch automatic dashboards. For more information, see [Setting up Container Insights on Amazon ECS](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/deploy-container-insights-ECS.html) in the *Amazon CloudWatch User Guide*.
## AWS X-Ray
AWS X-Ray is a tracing service that you can use to collect information about the network requests that your application makes. You can use the SDK to instrument your application and capture timings and response codes of traffic between your services, and between your services and AWS service endpoints. For more information, see [What is AWS X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/aws-xray.html) in the *AWS X-Ray Developer Guide*.
You can also explore AWS X-Ray graphs of how your services network with each other. Or, use them to explore aggregate statistics about how each service-to-service link is performing. Last, you can dive deeper into any specific transaction to see how segments representing network calls are associated with that particular transaction.
You can use these features to identify if there's a networking bottleneck or if a specific service within your network isn't performing as expected.
## VPC Flow Logs
You can use Amazon VPC flow logs to analyze network performance and debug connectivity issues. With VPC flow logs enabled, you can capture a log of all the connections in your VPC. These include connections to networking interfaces that are associated with Elastic Load Balancing, Amazon RDS, NAT gateways, and other key AWS services that you might be using. For more information, see [VPC Flow Logs](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html) in the *Amazon VPC User Guide*.
## Network tuning tips
There are a few settings that you can fine tune in order to improve your networking.
### nofile ulimit
If you expect your application to have high traffic and handle many concurrent connections, you should take into account the system quota for the number of files allowed. When there are a lot of network sockets open, each one must be represented by a file descriptor. If your file descriptor quota is too low, it will limit your network sockets. This results in failed connections or errors. You can update the container specific quota for the number of files in the Amazon ECS task definition. If you're running on Amazon EC2 (instead of AWS Fargate), then you might also need to adjust these quotas on your underlying Amazon EC2 instance.
### sysctl net
Another category of tunable setting is the `sysctl` net settings. You should refer to the specific settings for your Linux distribution of choice. Many of these settings adjust the size of read and write buffers. This can help in some situations when running large-scale Amazon EC2 instances that have a lot of containers on them.
# Optimize Amazon ECS task launch time
In order to speed up your task launches, consider the following recommendations.
+
**Cache container images and binpack instances**
If you use EC2, you can configure the Amazon ECS container agent pull behavior to `ECS_IMAGE_PULL_BEHAVIOR`: `prefer-cached`. The image is pulled remotely if there is no cached image. Otherwise, the cached image on the instance is used. Automated image cleanup is turned off for the container to ensure that the cached image isn't removed. This reduces image pull-time for subsequent launches. The effect of caching is even greater when you have a high task density in your container instances, which you can configure using the `binpack` placement strategy. Caching container images is especially beneficial for windows-based workloads which usually have large (tens of GBs) container image sizes. When using the `binpack` placement strategy, you can also consider using Elastic Network Interface (ENI) trunking to place more tasks with the `awsvpc` network mode on each container instance. ENI trunking increases the number of tasks you can run on `awsvpc` mode. For example, a c5.large instance that may support running only 2 tasks concurrently, can run up to 10 tasks with ENI trunking.
+
**Choose an optimal network mode**
Although there are many instances where `awsvpc` network mode is ideal, this network mode can inherently increase task launch latency, because for each task in `awsvpc` mode, Amazon ECS workflows need to provision and attach an ENI by invoking Amazon EC2 APIs which adds an overhead of several seconds to your task launches. By contrast, a key advantage of using `awsvpc` network mode is that each task has a security group to allow or deny traffic. This means you have greater flexibility to control communications between tasks and services at a more granular level. If the deployment speed is your priority, you can consider using `bridge` mode to speed up task launches. For more information, see [Allocate a network interface for an Amazon ECS task](task-networking-awsvpc.md).
+
**Track your task launch lifecycle to find optimization opportunities**
It is often difficult to know the amount of time it takes for your application to start-up. Launching your container image, running start-up scripts, and other configurations during application start-up can take a surprising amount of time. You can use the Task metadata endpoint to post metrics to track application start-up time from `StartedAt` in the container metadata response to the `StartedAt` time for your task or service. With this data, you can understand how your application is contributing to the total launch time, and find areas where you can reduce unnecessary application-specific overhead and optimize your container images. For more information, see [Amazon ECS Auto scaling and capacity management best practices](capacity-availability.md).
+
**Choose an optimal instance type (for EC2)**
Choosing the correct instance type is based on the resource reservation (for example, CPU, memory) that you configure on your task. Therefore, when sizing the instance, you can calculate how many tasks can be placed on a single instance. A simple example of a well-placed task, is hosting 4 tasks requiring 0.5 vCPU and 2GB of memory reservations in an m5.large instance (supporting 2 vCPU and 8 GB memory). The reservations of this task definition take full advantage of the instance’s resources.
# Operating Amazon ECS at scale
As you begin operating Amazon ECS at scale, consider how service quotas and API throttles for Amazon ECS and the AWS services that integrate with Amazon ECS might affect you.
**Topics**
+ [
# Amazon ECS service quotas and API throttling limits
](operating-at-scale-service-quotas-best-practice.md)
+ [
# Handle Amazon ECS throttling issues
](operating-at-scale-dealing-with-throttles.md)
# Amazon ECS service quotas and API throttling limits
Amazon ECS integrates with several AWS services, including Elastic Load Balancing, AWS Cloud Map, and Amazon EC2. With this tight integration, Amazon ECS includes several features such as service load balancing, service discovery, task networking, and cluster auto scaling. Amazon ECS and the other AWS services that it integrates with all maintain service quotas and API rate limits to ensure consistent performance and utilization. These service quotas also prevent the accidental provisioning of more resources than needed and protect against malicious actions that might increase your bill.
By familiarizing yourself with your service quotas and the AWS API rate limits, you can plan for scaling your workloads without worrying about unexpected performance degradation. For more information, see [Amazon ECS service quotas](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-quotas.html) and [Request throttling for the Amazon ECS API](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/request-throttling.html).
When scaling your workloads on Amazon ECS, consider the following service quota. For instructions on how to request a service quota increase, see [Managing your Amazon ECS and AWS Fargate service quotas in the AWS Management Console](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-quotas-manage.html).
+ AWS Fargate has quotas that limit the number of concurrent running tasks in each AWS Region. There are quotas for both On-Demand and Fargate Spot tasks on Amazon ECS. Each service quota also includes any Amazon EKS pods that you run on Fargate. For more information about the Fargate quotas, see [AWS Fargate service quotas](https://docs.aws.amazon.com/AmazonECS/latest/userguide/service-quotas.html#service-quotas-fargate) in the *Amazon Elastic Container Service User Guide for AWS Fargate*.
+ For tasks that run on Amazon EC2 instances, the maximum number of Amazon EC2 instances that you can register for each cluster is 5,000. If you use Amazon ECS cluster auto scaling with an Auto Scaling group capacity provider, or if you manage Amazon EC2 instances for your cluster on your own, this quota might become a deployment bottleneck. If you require more capacity, you can create more clusters or request a service quota increase.
+ If you use Amazon ECS cluster auto scaling with an Auto Scaling group capacity provider, when scaling your services consider the `Tasks in the PROVISIONING state per cluster` quota. This quota is the maximum number of tasks in the `PROVISIONING` state for each cluster for which capacity providers can increase capacity. When you launch a large number of tasks all at the same time, you can easily meet this quota. One example is if you simultaneously deploy tens of services, each with hundreds of tasks. When this happens, the capacity provider needs to launch new container instances to place the tasks when the cluster has insufficient capacity. While the capacity provider is launching additional Amazon EC2 instances, the Amazon ECS service scheduler likely will continue to launch tasks in parallel. However, this activity might be throttled because of insufficient cluster capacity. The Amazon ECS service scheduler implements a back-off and exponential throttling strategy for retrying task placement as new container instances are launched. As a result, you might experience slower deployment or scale-out times. To avoid this situation, you can plan your service deployments in one of the following ways. Either deploy a large number of tasks that don't require increasing cluster capacity, or keep spare cluster capacity for new task launches.
In addition to considering Amazon ECS service quota when scaling your workloads, consider also the service quota for the other AWS services that are integrated with Amazon ECS. The following section covers the key rate limits for each service in detail, and provides recommendations to deal with potential throttling issues.
## Elastic Load Balancing
You can configure your Amazon ECS services to use Elastic Load Balancing to distribute traffic evenly across the tasks. For more information about how to choose a load balancer, see [Use load balancing to distribute Amazon ECS service traffic](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-load-balancing.html).
### Elastic Load Balancing service quotas
When you scale your workloads, consider the following Elastic Load Balancing service quotas. Most Elastic Load Balancing service quotas are adjustable, and you can request an increase in the Service Quotas console.
**Application Load Balancer**
When you use an Application Load Balancer, depending on your use case, you might need to request a quota increase for:
+ The `Targets per Application Load Balancer` quota which is the number of targets behind your Application Load Balancer.
+ The `Targets per Target Group per Region` quota which is the number of targets behind your Target Groups.
For more information, see [Quotas for your Application Load Balancers](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-limits.html).
**Network Load Balancer**
There are stricter limitations on the number of targets you can register with a Network Load Balancer. When using a Network Load Balancer, you often will want to enable cross-zone support, which comes with additional scaling limitations on `Targets per Availability Zone Per Network Load Balancer` the maximum number of targets per Availability Zone for each Network Load Balancer. For more information, see [Quotas for your Network Load Balancers](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-limits.html).
### Elastic Load Balancing API throttling
When you configure an Amazon ECS service to use a load balancer, the target group health checks must pass before the service is considered healthy. For performing these health checks, Amazon ECS invokes Elastic Load Balancing API operations on your behalf. If you have a large number of services configured with load balancers in your account, you might experience slower service deployments because of potential throttling specifically for the `RegisterTarget`, `DeregisterTarget`, and `DescribeTargetHealth` Elastic Load Balancing API operations. When throttling occurs, throttling errors occur in your Amazon ECS service event messages.
If you experience AWS Cloud Map API throttling, you can contact Support for guidance on how to increase your AWS Cloud Map API throttling limits. For more information about monitoring and troubleshooting such throttling errors, see [Handling throttling issues](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/operating-at-scale-dealing-with-throttles.html).
## Elastic network interfaces
When your tasks use the `awsvpc` network mode, Amazon ECS provisions a unique elastic network interface (ENI) for each task. When your Amazon ECS services use an Elastic Load Balancing load balancer, these network interfaces are also registered as targets to the appropriate target group defined in the service.
### Elastic network interface service quotas
When you run tasks that use the `awsvpc` network mode, a unique elastic network interface is attached to each task. If those tasks must be reached over the internet, assign a public IP address to the elastic network interface for those tasks. When you scale your Amazon ECS workloads, consider these two important quotas:
+ The `Network interfaces per Region` quota which is the maximum number of network interfaces in an AWS Region for your account.
+ The `Elastic IP addresses per Region` quota which is the maximum number of elastic IP addresses in an AWS Region.
Both of these service quotas are adjustable and you can request an increase from your Service Quotas console for these. For more information, see [Amazon VPC service quotas](https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html#vpc-limits-enis).
For Amazon ECS workloads that are hosted on Amazon EC2 instances, when running tasks that use the `awsvpc` network mode consider the `Maximum network interfaces` service quota, the maximum number of network instances for each Amazon EC2 instance. This quota limits the number of tasks that you can place on an instance. You cannot adjust the quota and it's not available in the Service Quotas console. For more information, see [IP addresses per network interface per instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI) in the *Amazon EC2 User Guide*.
Although you can't change the number of network interfaces that can be attached to an Amazon EC2 instance, you can use the elastic network interface trunking feature to increase the number of available network interfaces. For example, by default a `c5.large` instance can have up to three network interfaces. The primary network interface for the instance counts as one. So, you can attach an additional two network interfaces to the instance. Because each task that uses the `awsvpc` network mode requires a network interface, you can typically only run two such tasks on this instance type. This can lead to under-utilization of your cluster capacity. If you enable elastic network interface trunking, you can increase the network interface density to place a larger number of tasks on each instance. With trunking turned on, a `c5.large` instance can have up to 12 network interfaces. The instance has the primary network interface and Amazon ECS creates and attaches a "trunk" network interface to the instance. As a result, with this configuration you can run 10 tasks on the instance instead of the default two tasks. For more information, see [Elastic network interface trunking](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container-instance-eni.html).
### Elastic network interface API throttling
When you run tasks that use the `awsvpc` network mode, Amazon ECS relies on the following Amazon EC2 APIs. Each of these APIs have different API throttles. For more information, see [Request throttling for the Amazon EC2 API](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/throttling.html).
+ CreateNetworkInterface
+ AttachNetworkInterface
+ DetachNetworkInterface
+ DeleteNetworkInterface
+ DescribeNetworkInterfaces
+ DescribeVpcs
+ DescribeSubnets
+ DescribeSecurityGroups
+ DescribeInstances
If the Amazon EC2 API calls are throttled during the elastic network interface provisioning workflows, the Amazon ECS service scheduler automatically retries with exponential back-offs. These automatic retries might sometimes lead to a delay in launching tasks, which results in slower deployment speeds. When API throttling occurs, you will see the message `Operations are being throttled. Will try again later.` on your service event messages. If you consistently meet Amazon EC2 API throttles, you can contact Support for guidance on how to increase your API throttling limits. For more information about monitoring and troubleshooting throttling errors, see [Handling throttling issues](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/operating-at-scale-dealing-with-throttles.html).
## AWS Cloud Map
Amazon ECS service discovery uses AWS Cloud Map APIs to manage namespaces for your Amazon ECS services. If your services have a large number of tasks, consider the following recommendations. For more information, see [Amazon ECS service discovery considerations](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-discovery.html#service-discovery-considerations).
### AWS Cloud Map service quotas
When Amazon ECS services are configured to use service discovery, the `Tasks per service` quota which is the maximum number of tasks for the service, is affected by the AWS Cloud Map `Instances per service` service quota which is the maximum number of instances for that service. In particular, the AWS Cloud Map service quota decreases the amount of tasks that you can run to at most 1,000 tasks per service. You cannot change the AWS Cloud Map quota. For more information, see [AWS Cloud Map service quotas](https://docs.aws.amazon.com/general/latest/gr/cloud_map.html).
### AWS Cloud Map API throttling
Amazon ECS calls the `ListInstances`, `GetInstancesHealthStatus`, `RegisterInstance`, and `DeregisterInstance` AWS Cloud Map APIs on your behalf. They help with service discovery and perform health checks when you launch a task. When multiple services that use service discovery with a large number of tasks are deployed at the same time, this can result in exceeding the AWS Cloud Map API throttling limits. When this happens, you likely will see the following message: `Operations are being throttled. Will try again later` in your Amazon ECS service event messages and slower deployment and task launch speed. AWS Cloud Map doesn't document throttling limits for these APIs. If you experience throttling from these, you can contact Support for guidance on increasing your API throttling limits. For more recommendations on monitoring and troubleshooting such throttling errors, see [Handling throttling issues](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/operating-at-scale-dealing-with-throttles.html).
# Handle Amazon ECS throttling issues
Throttling errors fall into two major categories: synchronous throttling and asynchronous throttling.
## Synchronous throttling
When synchronous throttling occurs, you immediately receive an error response from Amazon ECS. This category typically occurs when you call Amazon ECS APIs while running tasks or creating services. For more information about the throttling involved and the relevant throttle limits, see [Request throttling for the Amazon ECS API](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/request-throttling.html).
When your application initiates API requests, for example, by using the AWS CLI or an AWS SDK, you can remediate API throttling. You can do this by either architecting your application to handle the errors or by implementing an exponential backoff and jitter strategy with retry logic for the API calls. For more information, see [Timeouts, retries, and backoff with jitter](https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/).
If you use an AWS SDK, the automatic retry logic is built-in and configurable.
## Asynchronous throttling
Asynchronous throttling occurs because of asynchronous workflows where Amazon ECS or CloudFormation might be calling APIs on your behalf to provision resources. It's important to know which AWS APIs that Amazon ECS invokes on your behalf. For example, the `CreateNetworkInterface` API is invoked for tasks that use the `awsvpc` network mode, and the `DescribeTargetHealth` API is invoked when performing health checks for tasks registered to a load balancer.
When your workloads reach a considerable scale, these API operations might be throttled. That is, they might be throttled enough to breach the limits enforced by Amazon ECS or the AWS service that is being called. For example, if you deploy hundreds of services, each having hundreds of tasks concurrently that use the `awsvpc` network mode, Amazon ECS invokes Amazon EC2 API operations such as `CreateNetworkInterface` and Elastic Load Balancing API operations such as `RegisterTarget` or `DescribeTargetHealth` to register the elastic network interface and load balancer, respectively. These API calls can exceed the API limits, resulting in throttling errors. The following is an example of an Elastic Load Balancing throttling error that's included in the service event message.
```
{
"userIdentity":{
"arn":"arn:aws:sts::111122223333:assumed-role/AWSServiceRoleForECS/ecs-service-scheduler",
"eventTime":"2022-03-21T08:11:24Z",
"eventSource":"elasticloadbalancing.amazonaws.com",
"eventName":" DescribeTargetHealth ",
"awsRegion":"us-east-1",
"sourceIPAddress":"ecs.amazonaws.com",
"userAgent":"ecs.amazonaws.com",
"errorCode":"ThrottlingException",
"errorMessage":"Rate exceeded",
"eventID":"0aeb38fc-229b-4912-8b0d-2e8315193e9c"
}
}
```
When these API calls share limits with other API traffic in your account, they might be difficult monitor even though they're emitted as service events.
## Monitor throttling
It's important to identify which API requests are throttled and who issues these requests. You can use AWS CloudTrail which monitors throttling, and integrates with CloudWatch, Amazon Athena, and Amazon EventBridge. You can configure CloudTrail to send specific events to CloudWatch Logs. CloudWatch Logs log insights parses and analyzes the events. This identifies details in throttling events such as the user or IAM role that made the call and the number of API calls that were made. For more information, see [Monitoring CloudTrail log files with CloudWatch Logs](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/monitor-cloudtrail-log-files-with-cloudwatch-logs.html).
For more information about CloudWatch Logs insights and instructions on how to query log files, see [Analyzing log data with CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html).
With Amazon Athena, you can create queries and analyze data using standard SQL. For example, you can create an Athena table to parse CloudTrail events. For more information, see [Using the CloudTrail console to create an Athena table for CloudTrail logs](https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html#create-cloudtrail-table-ct).
After creating an Athena table, you can use SQL queries such as the following one to investigate `ThrottlingException` errors.
Replace the *user-input* with your values.
```
select eventname, errorcode,eventsource,awsregion, useragent,COUNT(*) count
FROM cloudtrail_table-name
where errorcode = 'ThrottlingException'
AND eventtime between '2024-09-24T00:00:08Z' and '2024-09-23T23:15:08Z'
group by errorcode, awsregion, eventsource, useragent, eventname
order by count desc;
```
Amazon ECS also emits event notifications to Amazon EventBridge. There are resource state change events and service action events. They include API throttling events such as `ECS_OPERATION_THROTTLED` and `SERVICE_DISCOVERY_OPERATION_THROTTLED`. For more information, see [Amazon ECS service action events](ecs_service_events.md).
These events can be consumed by a service such as AWS Lambda to perform actions in response. For more information, see [Handling Amazon ECS events](ecs_cwet_handling.md).
If you run standalone tasks, some API operations such as `RunTask` are asynchronous, and retry operations aren't automatically performed. In such cases, you can use services such as AWS Step Functions with EventBridge integration to retry throttled or failed operations. For more information, see [Manage a container task (Amazon ECS, Amazon SNS)](https://docs.aws.amazon.com/step-functions/latest/dg/sample-project-container-task-notification.html).
## Use CloudWatch to monitor throttling
CloudWatch offers API usage monitoring on the `Usage` namespace under **By AWS Resource**. These metrics are logged with type **API** and metric name **CallCount**. You can create alarms to start whenever these metrics reach a certain threshold. For more information, see [Visualizing your service quotas and setting alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Quotas-Visualize-Alarms.html).
CloudWatch also offers anomaly detection. This feature uses machine learning to analyze and establish baselines based on the particular behavior of the metric that you enabled it on. If there's unusual API activity, you can use this feature together with CloudWatch alarms. For more information, see [Using CloudWatch anomaly detection](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Anomaly_Detection.html).
By proactively monitoring throttling errors, you can contact Support to increase the relevant throttling limits and also receive guidance for your unique application needs.
# Amazon ECS Auto scaling and capacity management best practices
You can run containerized application workloads of all sizes on Amazon ECS. This includes minimal testing environments and large production environments that operate at a global scale.
With Amazon ECS, like all AWS services, you pay only for what you use. When you architect your application appropriately, you can save costs by consuming only the resources that you need when you need them.
The following recommendations show you how to run your Amazon ECS workloads to meet your service-level objectives while operating cost-effectively.
**Topics**
+ [
# Determining the task size for Amazon ECS
](capacity-tasksize-best-practice.md)
+ [
# Optimizing Amazon ECS service auto scaling
](capacity-autoscaling-best-practice.md)
+ [
# Amazon ECS capacity and availability
](capacity-availability-best-practice.md)
+ [
# Amazon ECS cluster capacity
](capacity-cluster-best-practice.md)
+ [
# Choosing Fargate task sizes for Amazon ECS
](fargate-task-size-best-practice.md)
+ [
# Speeding up Amazon ECS cluster capacity provisioning with capacity providers on Amazon EC2
](capacity-cluster-speed-up-ec2-best-practice.md)
# Determining the task size for Amazon ECS
One of the most important choices when you deploy containers on Amazon ECS is your container and task sizes. Your container and task sizes are essential for scaling and capacity planning.
Amazon ECS uses two resource metrics for capacity: CPU and memory. Amazon ECS measures CPU in units of 1/1024 of a full vCPU (where 1024 units equals 1 whole vCPU). Amazon ECS measures memory in megabytes.
In your task definition, you can declare resource reservations and limits.
When you declare a reservation, you declare the minimum amount of resources that a task requires. Your task receives at least the amount of resources you request. Your application might be able to use more CPU or memory than the reservation you declare. However, this is subject to any limits you also declared.
Using more than the reservation amount is known as bursting. Bursting means your application uses more resources than you reserved but stays within your declared limits. Amazon ECS guarantees reservations. For example, if you use Amazon EC2 instances to provide capacity, Amazon ECS doesn't place a task on an instance where it can't fulfill the reservation.
A limit is the maximum amount of CPU units or memory that your container or task can use. If your container tries to use more CPU than this limit, Amazon ECS throttles it. If your container tries to use more memory than this limit, Amazon ECS stops your container.
Choosing these values can be challenging. The values that work best for your application depend greatly on your application's resource requirements.
Load testing your application is the key to successful resource requirement planning. Load testing helps you better understand your application's requirements.
## Stateless applications
For stateless applications that scale horizontally, such as an application behind a load balancer, we recommend that you first determine how much memory your application consumes when it serves requests.
To do this, you can use traditional tools such as `ps` or `top`. You can also use monitoring solutions such as CloudWatch Container Insights.
When you determine a CPU reservation, consider how you want to scale your application to meet your business requirements.
You can use smaller CPU reservations, such as 256 CPU units (or 1/4 vCPU), to scale out in a fine-grained way that minimizes cost. But they might not scale fast enough to meet significant spikes in demand.
You can use larger CPU reservations to scale in and out more quickly. This helps you match demand spikes more quickly. However, larger CPU reservations cost more.
## Other applications
For applications that don't scale horizontally, such as singleton workers or database servers, available capacity and cost are your most important considerations.
Choose the amount of memory and CPU based on what load testing shows you need to serve traffic and meet your service-level objective. Amazon ECS ensures that your application is placed on a host that has adequate capacity.
# Optimizing Amazon ECS service auto scaling
An Amazon ECS service is a managed collection of tasks. Each service has an associated task definition, a desired task count, and an optional placement strategy.
Amazon ECS service auto scaling works through the Application Auto Scaling service. Application Auto Scaling uses CloudWatch metrics as the source for scaling metrics. It also uses CloudWatch alarms to set thresholds on when to scale your service in or out.
You provide the thresholds for scaling. You can set a metric target, which is called *target tracking scaling*. You can also specify thresholds, which is called *step scaling*.
After you configure Application Auto Scaling, it continually calculates the appropriate desired task count for the service. It also notifies Amazon ECS when the desired task count should change, either by scaling it out or scaling it in.
To use service auto scaling effectively, you must choose an appropriate scaling metric. We discuss how to choose a metric in the following sections.
## Characterizing your application
Properly scaling an application requires you to know the conditions where you should scale your application in and when you should scale it out.
In essence, you should scale out your application if demand is forecasted to outstrip capacity. Conversely, you can scale in your application to conserve costs when resources exceed demand.
### Identifying a utilization metric
To scale effectively, you must identify a metric that indicates utilization or saturation. This metric must exhibit the following properties to be useful for scaling.
+ The metric must be correlated with demand. When you hold resources steady but demand changes, the metric value must also change. The metric should increase or decrease when demand increases or decreases.
+ The metric value must scale in proportion to capacity. When demand holds constant, adding more resources must result in a proportional change in the metric value. So, doubling the number of tasks should cause the metric to decrease by 50%.
The best way to identify a utilization metric is through load testing in a pre-production environment such as a staging environment. Commercial and open-source load testing solutions are widely available. These solutions typically can either generate synthetic load or simulate real user traffic.
To start the process of load testing, you should first build dashboards for your application’s utilization metrics. These metrics include CPU utilization, memory utilization, I/O operations, I/O queue depth, and network throughput. You can collect these metrics with a service such as CloudWatch Container Insights. You can also collect them by using Amazon Managed Service for Prometheus together with Amazon Managed Grafana. During this process, make sure that you collect and plot metrics for your application’s response times or work completion rates.
When you load test, begin with a small request or job insertion rate. Keep this rate steady for several minutes to allow your application to warm up. Then, slowly increase the rate and hold it steady for a few minutes. Repeat this cycle, increasing the rate each time until your application’s response or completion times are too slow to meet your service-level objectives (SLOs).
While you load test, examine each of the utilization metrics. The metrics that increase along with the load are the top candidates to serve as your best utilization metrics.
Next, identify the resource that reaches saturation. At the same time, also examine the utilization metrics to see which one flattens out at a high level first. Or, examine which one reaches peak and then crashes your application first. For example, if CPU utilization increases from 0% to 70-80% as you add load, then stays at that level after you add even more load, then it's safe to say that the CPU is saturated. Depending on the CPU architecture, it might never reach 100%. For example, assume that memory utilization increases as you add load, and then your application suddenly crashes when it reaches the task or Amazon EC2 instance memory limit. In this situation, it's likely the case that memory has been fully consumed. Multiple resources might be consumed by your application. Therefore, choose the metric that represents the resource that depletes first.
Last, try load testing again after you double the number of tasks or Amazon EC2 instances. Assume that the key metric increases, or decreases, at half the rate as before. If this is the case, then the metric is proportional to capacity. This is a good utilization metric for auto scaling.
Now consider this hypothetical scenario. Suppose that you load test an application and find that the CPU utilization eventually reaches 80% at 100 requests per second. When you add more load, it doesn't make CPU utilization increase anymore. However, it does make your application respond more slowly. Then, you run the load test again, doubling the number of tasks but holding the rate at its previous peak value. If you find the average CPU utilization falls to about 40%, then average CPU utilization is a good candidate for a scaling metric. On the other hand, if CPU utilization remains at 80% after increasing the number of tasks, then average CPU utilization isn't a good scaling metric. In that case, you need more research to find a suitable metric.
### Common application models and scaling properties
You can run software of all kinds on AWS. Many workloads are homegrown, whereas others are based on popular open-source software. Regardless of where they originate, we have observed some common design patterns for services. How you scale effectively depends in large part on the pattern.
#### The efficient CPU-bound server
The efficient CPU-bound server utilizes almost no resources other than CPU and network throughput. Each request can be handled by the application alone. Requests don't depend on other services such as databases. The application can handle hundreds of thousands of concurrent requests, and can efficiently utilize multiple CPUs to do so. Each request is either serviced by a dedicated thread with low memory overhead, or there's an asynchronous event loop that runs on each CPU that services requests. Each replica of the application is equally capable of handling a request. The only resource that might be depleted before CPU is network bandwidth. In CPU bound-services, memory utilization, even at peak throughput, is a fraction of the resources available.
You can use CPU-based auto scaling for this type of application. The application enjoys maximum flexibility in terms of scaling. You can scale it vertically by providing larger Amazon EC2 instances or Fargate vCPUs to it. And, you can also scale it horizontally by adding more replicas. Adding more replicas, or doubling the instance size, cuts the average CPU utilization relative to capacity by half.
If you're using Amazon EC2 capacity for this application, consider placing it on compute-optimized instances such as the `c5` or `c6g` family.
#### The efficient memory-bound server
The efficient memory-bound server allocates a significant amount of memory per request. At maximum concurrency, but not necessarily throughput, memory is depleted before the CPU resources are depleted. Memory associated with a request is freed when the request ends. Additional requests can be accepted as long as there is available memory.
You can use memory-based auto scaling for this type of application. The application enjoys maximum flexibility in terms of scaling. You can scale it both vertically by providing larger Amazon EC2 or Fargate memory resources to it. And, you can also scale it horizontally by adding more replicas. Adding more replicas, or doubling the instance size, can cut the average memory utilization relative to capacity by half.
If you're using Amazon EC2 capacity for this application, consider placing it on memory-optimized instances such as the `r5` or `r6g` family.
Some memory-bound applications don't free the memory that's associated with a request when it ends, so that a reduction in concurrency doesn't result in a reduction in the memory used. For this, we don't recommend that you use memory-based scaling.
#### The worker-based server
The worker-based server processes one request for each individual worker thread one after another. The worker threads can be lightweight threads, such as POSIX threads. They can also be heavier-weight threads, such as UNIX processes. No matter which thread they are, there's always a maximum concurrency that the application can support. Usually the concurrency limit is set proportionally to the memory resources that are available. If the concurrency limit is reached, the application places additional requests into a backlog queue. If the backlog queue overflows, the application immediately rejects additional incoming requests. Common applications that fit this pattern include Apache web server and Gunicorn.
Request concurrency is usually the best metric for scaling this application. Because there's a concurrency limit for each replica, it's important to scale out before the average limit is reached.
The best way to obtain request concurrency metrics is to have your application report them to CloudWatch. Each replica of your application can publish the number of concurrent requests as a custom metric at a high frequency. We recommend that the frequency is set to be at least once every minute. After several reports are collected, you can use the average concurrency as a scaling metric. You calculate this metric by taking the total concurrency and dividing it by the number of replicas. For example, if total concurrency is 1000 and the number of replicas is 10, then the average concurrency is 100.
If your application is behind an Application Load Balancer, you can also use the `ActiveConnectionCount` metric for the load balancer as a factor in the scaling metric. You must divide the `ActiveConnectionCount` metric by the number of replicas to obtain an average value. You must use the average value for scaling, as opposed to the raw count value.
For this design to work best, the standard deviation of response latency should be small at low request rates. We recommend that, during periods of low demand, most requests are answered within a short time, and there isn't a lot of requests that take significantly longer than average time to respond. The average response time should be close to the 95th percentile response time. Otherwise, queue overflows might occur as result. This leads to errors. We recommend that you provide additional replicas where necessary to mitigate the risk of overflow.
#### The waiting server
The waiting server does some processing for each request, but it is highly dependent on one or more downstream services to function. Container applications often make heavy use of downstream services like databases and other API services. It can take some time for these services to respond, particularly in high capacity or high concurrency scenarios. This is because these applications tend to use few CPU resources and utilize their maximum concurrency in terms of available memory.
The waiting service is suitable either in the memory-bound server pattern or the worker-based server pattern, depending on how the application is designed. If the application’s concurrency is limited only by memory, then average memory utilization should be used as a scaling metric. If the application’s concurrency is based on a worker limit, then average concurrency should be used as a scaling metric.
#### The Java-based server
If your Java-based server is CPU-bound and scales proportionally to CPU resources, then it might be suitable for the efficient CPU-bound server pattern. If that is the case, average CPU utilization might be appropriate as a scaling metric. However, many Java applications aren't CPU-bound, making them challenging to scale.
For the best performance, we recommend that you allocate as much memory as possible to the Java Virtual Machine (JVM) heap. Recent versions of the JVM, including Java 8 update 191 or later, automatically set the heap size as large as possible to fit within the container. This means that, in Java, memory utilization is rarely proportional to application utilization. As the request rate and concurrency increases, memory utilization remains constant. Because of this, we don't recommend scaling Java-based servers based on memory utilization. Instead, we typically recommend scaling on CPU utilization.
In some cases, Java-based servers encounter heap exhaustion before exhausting CPU. If your application is prone to heap exhaustion at high concurrency, then average connections are the best scaling metric. If your application is prone to heap exhaustion at high throughput, then average request rate is the best scaling metric.
#### Servers that use other garbage-collected runtimes
Many server applications are based on runtimes that perform garbage collection such as .NET and Ruby. These server applications might fit into one of the patterns described earlier. However, as with Java, we don't recommend scaling these applications based on memory, because their observed average memory utilization is often uncorrelated with throughput or concurrency.
For these applications, we recommend that you scale on CPU utilization if the application is CPU bound. Otherwise, we recommend that you scale on average throughput or average concurrency, based on your load testing results.
#### Job processors
Many workloads involve asynchronous job processing. They include applications that don't receive requests in real time, but instead subscribe to a work queue to receive jobs. For these types of applications, the proper scaling metric is almost always queue depth. Queue growth is an indication that pending work outstrips processing capacity, whereas an empty queue indicates that there's more capacity than work to do.
AWS messaging services, such as Amazon SQS and Amazon Kinesis Data Streams, provide CloudWatch metrics that can be used for scaling. For Amazon SQS, `ApproximateNumberOfMessagesVisible` is the best metric. For Kinesis Data Streams, consider using the `MillisBehindLatest` metric, published by the Kinesis Client Library (KCL). This metric should be averaged across all consumers before using it for scaling.
# Amazon ECS capacity and availability
Application availability is crucial for providing an error-free experience and for minimizing application latency. Availability depends on having resources that are accessible and have enough capacity to meet demand. AWS provides several mechanisms to manage availability. For applications that you host on Amazon ECS, these include autoscaling and Availability Zones (AZs). Autoscaling manages the number of tasks or instances based on metrics you define, while Availability Zones allow you to host your application in isolated but geographically-close locations.
As with task sizes, capacity and availability present certain trade-offs you must consider. Ideally, capacity would be perfectly aligned with demand. There would always be just enough capacity to serve requests and process jobs to meet Service Level Objectives (SLOs) including a low latency and error rate. Capacity would never be too high, leading to excessive cost; nor would it never be too low, leading to high latency and error rates.
Autoscaling is a latent process. First, CloudWatch must receive real-time metrics. Then, CloudWatch needs to aggregate them for analysis, which can take up to several minutes depending on the granularity of the metric. CloudWatch compares the metrics against alarm thresholds to identify a shortage or excess of resources. To prevent instability, you should configure alarms to require the set threshold be crossed for a few minutes before the alarm goes off. It also takes time to provision new tasks and to terminate tasks that are no longer needed.
Because of these potential delays in the system, you should maintain some headroom by over-provisioning. Over-provisioning can help accommodate short-term bursts in demand. This also helps your application service additional requests without reaching saturation. As a good practice, you can set your scaling target between 60-80% of utilization. This helps your application better handle bursts of extra demand while additional capacity is still in the process of being provisioned.
Another reason we recommend that you over-provision is so that you can quickly respond to Availability Zone failures. AWS recommends that production workloads be served from multiple Availability Zones. This is because, if an Availability Zone failure occurs, your tasks that are running in the remaining Availability Zones can still serve the demand. If your application runs in two Availability Zones, you need to double your normal task count. This is so that you can provide immediate capacity during any potential failure. If your application runs in three Availability Zones, we recommend that you run 1.5 times your normal task count. That is, run three tasks for every two that are needed for ordinary serving.
## Maximizing scaling speed
Autoscaling is a reactive process that takes time to take effect. However, there are some ways to help minimize the time that's needed to scale out.
**Minimize image size.** Larger images take longer to download from an image repository and unpack. Therefore, keeping image sizes smaller reduces the amount of time that's needed for a container to start. To reduce the image size, you can follow these specific recommendations:
+ If you can build a static binary or use Golang, build your image `FROM` scratch and include only your binary application in the resulting image.
+ Use minimized base images from upstream distro vendors, such as Amazon Linux or Ubuntu.
+ Don’t include any build artifacts in your final image. Using multi-stage builds can help with this.
+ Compact `RUN` stages wherever possible. Each `RUN` stage creates a new image layer, leading to an additional round trip to download the layer. A single `RUN` stage that has multiple commands joined by `&&` has fewer layers than one with multiple `RUN` stages.
+ If you want to include data, such as ML inference data, in your final image, include only the data that's needed to start up and begin serving traffic. If you fetch data on demand from Amazon S3 or other storage without impacting service, then store your data in those places instead.
**Keep your images close.** The higher the network latency, the longer it takes to download the image. Host your images in a repository in the same AWS Region that your workload is in. Amazon ECR is a high-performance image repository that's available in every Region that Amazon ECS is available in. Avoid traversing the Internet or a VPN link to download container images. Hosting your images in the same Region improves overall reliability. It mitigating the risk of network connectivity issues and availability issues in a different Region. Alternatively, you can also implement Amazon ECR cross-region replication to help with this.
**Reduce load balancer health check thresholds.** Load balancers perform health checks before sending traffic to your application. The default health check configuration for a target group can take 90 seconds or longer. During this, the load balancer checks the health status and receives requests. Lowering the health check interval and threshold count can make your application accept traffic quicker and reduce load on other tasks.
**Consider cold-start performance.** Some application use runtimes such as Java perform Just-In-Time (JIT) compilation. The compilation process at least as it starts can show application performance. A workaround is to rewrite the latency-critical parts of your workload in languages that don't impose a cold-start performance penalty.
**Use step scaling, not target-tracking scaling policies.** You have several Application Auto Scaling options for Amazon ECS tasks. Target tracking is the easiest mode to use. With it, all you need to do is set a target value for a metric, such as CPU average utilization. Then, the auto scaler automatically manages the number of tasks that are needed to attain that value. With step scaling you can more quickly react to changes in demand, because you define the specific thresholds for your scaling metrics, and how many tasks to add or remove when the thresholds are crossed. And, more importantly, you can react very quickly to changes in demand by minimizing the amount of time a threshold alarm is in breach. For more information, see [Service Auto Scaling](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-auto-scaling.html) in the *Amazon Elastic Container Service Developer Guide*.
If you're using Amazon EC2 instances to provide cluster capacity, consider the following recommendations:
**Use larger Amazon EC2 instances and faster Amazon EBS volumes.** You can improve image download and preparation speeds by using a larger Amazon EC2 instance and faster Amazon EBS volume. Within a given Amazon EC2 instance family, the network and Amazon EBS maximum throughput increases as the instance size increases (for example, from `m5.xlarge` to `m5.2xlarge`). Additionally, you can also customize Amazon EBS volumes to increase their throughput and IOPS. For example, if you’re using `gp2` volumes, use larger volumes that offer more baseline throughput. If you're using `gp3` volumes, specify throughput and IOPS when you create the volume.
**Use bridge network mode for tasks running on Amazon EC2 instances.** Tasks that use `bridge` network mode on Amazon EC2 start faster than tasks that use the `awsvpc` network mode. When `awsvpc` network mode is used, Amazon ECS attaches an elastic network interface (ENI) to the instance before launching the task. This introduces additional latency. There are several tradeoffs for using bridge networking though. These tasks don't get their own security group, and there are some implications for load balancing. For more information, see [Load balancer target groups](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html) in the *Elastic Load Balancing User Guide*.
## Handling demand shocks
Some applications experience sudden large shocks in demand. This happens for a variety of reasons: a news event, big sale, media event, or some other event that goes viral and causes traffic to quickly and significantly increase in a very short span of time. If unplanned, this can cause demand to quickly outstrip available resources.
The best way to handle demand shocks is to anticipate them and plan accordingly. Because autoscaling can take time, we recommend that you scale out your application before the demand shock begins. For the best results, we recommend having a business plan that involves tight collaboration between teams that use a shared calender. The team that's planning the event should work closely with the team in charge of the application in advance. This gives that team enough time to have a clear scheduling plan. They can schedule capacity to scale out before the event and to scale in after the event. For more information, see [Scheduled scaling](https://docs.aws.amazon.com/autoscaling/application/userguide/application-auto-scaling-scheduled-scaling.html) in the *Application Auto Scaling User Guide*.
If you have an Enterprise Support plan, be sure also to work with your Technical Account Manager (TAM). Your TAM can verify your service quotas and ensure that any necessary quotas are raised before the event begins. This way, you don't accidentally hit any service quotas. They can also help you by prewarming services such as load balancers to make sure your event goes smoothly.
Handling unscheduled demand shocks is a more difficult problem. Unscheduled shocks, if large enough in amplitude, can quickly cause demand to outstrip capacity. It can also outpace the ability for autoscaling to react. The best way to prepare for unscheduled shocks is to over-provision resources. You must have enough resources to handle maximum anticipated traffic demand at any time.
Maintaining maximum capacity in anticipation of unscheduled demand shocks can be costly. To mitigate the cost impact, find a leading indicator metric or event that predicts a large demand shock is imminent. If the metric or event reliably provides significant advance notice, begin the scale-out process immediately when the event occurs or when the metric crosses the specific threshold that you set.
If your application is prone to sudden unscheduled demand shocks, consider adding a high-performance mode to your application that sacrifices non-critical functionality but retains crucial functionality for a customer. For example, assume that your application can switch from generating expensive customized responses to serving a static response page. In this scenario, you can increase throughput significantly without scaling the application at all.
Last, you can consider breaking apart monolithic services to better deal with demand shocks. If your application is a monolithic service that's expensive to run and slow to scale, you might be able to extract or rewrite performance-critical pieces and run them as separate services. These new services then can be scaled independently from less-critical components. Having the flexibility to scale out performance-critical functionality separately from other parts of your application can both reduce the time it takes to add capacity and help conserve costs.
# Amazon ECS cluster capacity
You can provide capacity to an Amazon ECS cluster in several ways. For example, you can launch Amazon EC2 instances and register them with the cluster at start-up using the Amazon ECS container agent. However, this method can be challenging because you need to manage scaling on your own. Therefore, we recommend that you use Amazon ECS capacity providers. Capacity providers manage resource scaling for you. There are three kinds of capacity providers: Amazon EC2, Fargate, and Fargate Spot. For more information about Fargate capacity providers, see [Amazon ECS clusters for Fargate workloads](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-capacity-providers.html) and for EC2 workloads, see [Amazon ECS clusters for EC2 workloads](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/asg-capacity-providers.html).
The Fargate and Fargate Spot capacity providers handle the lifecycle of Fargate tasks for you. Fargate provides on-demand capacity, and Fargate Spot provides Spot capacity. When you launch a task, Amazon ECS provisions a Fargate resource for you. This Fargate resource comes with the memory and CPU units that directly correspond to the task-level limits that you declared in your task definition. Each task receives its own Fargate resource, making a 1:1 relationship between the task and compute resources.
Tasks that run on Fargate Spot are subject to interruption. Interruptions come after a two-minute warning. These occur during periods of heavy demand. Fargate Spot works best for interruption-tolerant workloads such as batch jobs, development or staging environments. They're also suitable for any other scenario where high availability and low latency isn't a requirement.
You can run Fargate Spot tasks alongside Fargate on-demand tasks. By using them together, you receive provision “burst” capacity at a lower cost.
Amazon ECS can also manage the Amazon EC2 instance capacity for your tasks. Each Amazon EC2 capacity provider is associated with an Amazon EC2 Auto Scaling group that you specify. When you use the Amazon EC2 capacity provider, cluster auto scaling maintains the size of the Amazon EC2 Auto Scaling group to ensure all scheduled tasks can be placed.
# Choosing Fargate task sizes for Amazon ECS
For AWS Fargate task definitions, you're required to specify CPU and memory at the task level, and do not need to account for any overhead. You can also specify CPU and memory at the container level for Fargate tasks. However, doing so isn't required. The resource limits must be greater than or equal to any reservations that you declared. In most cases, you can set them to the sum of the reservations of each of the container that's declared in your task definition. Then, round the number up to the nearest Fargate size. For more information about the available sizes, see [Task CPU and memory](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/fargate-tasks-services.html#fargate-tasks-size).
# Speeding up Amazon ECS cluster capacity provisioning with capacity providers on Amazon EC2
Customers who run Amazon ECS on Amazon EC2 can take advantage of [Amazon ECS Cluster Auto Scaling (CAS)](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-auto-scaling.html) to manage the scaling of Amazon EC2 Auto Scaling groups (ASG). With CAS, you can configure Amazon ECS to scale your ASG automatically, and just focus on running your tasks. Amazon ECS will ensure the ASG scales in and out as needed with no further intervention required. Amazon ECS capacity providers are used to manage the infrastructure in your cluster by ensuring there are enough container instances to meet the demands of your application. To learn how Amazon ECS CAS works, see [Deep Dive on Amazon ECS Cluster Auto Scaling](https://aws.amazon.com/blogs/containers/deep-dive-on-amazon-ecs-cluster-auto-scaling/).
Since CAS relies on a CloudWatch based integration with ASG for adjusting cluster capacity, it has inherent latency associated with publishing the CloudWatch metrics, the time taken for the metric `CapacityProviderReservation` to breach CloudWatch alarms (both high and low), and the time taken by a newly launched Amazon EC2 instance to warm-up. You can take the following actions to make CAS more responsive for faster deployments:
## Capacity provider step scaling sizes
Amazon ECS capacity providers will eventually grow/shrink the container instances to meet the demands of your application. The minimum number of instances that Amazon ECS will launch is set to 1 by default. This may add additional time to your deployments, if several instances are required for placing your pending tasks. You can increase the [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ManagedScaling.html](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ManagedScaling.html) by using the Amazon ECS API to increase the minimum number of instances that Amazon ECS scales in or out at a time. A [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ManagedScaling.html](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ManagedScaling.html) that is too low can limit how many container instances are scaled in or out at a time, which can slow down your deployments.
**Note**
This configuration is currently only available by using the [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateCapacityProvider.html](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateCapacityProvider.html) or [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateCapacityProvider.html](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateCapacityProvider.html) APIs.
## Instance warm-up period
The instance warm-up period is the period of time after which a newly launched Amazon EC2 instance can contribute to CloudWatch metrics for the Auto Scaling group. After the specified warm-up period expires, the instance is counted toward the aggregated metrics of the ASG, and CAS proceeds with its next iteration of calculations to estimate the number instances required.
The default value for [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ManagedScaling.html#ECS-Type-ManagedScaling-instanceWarmupPeriod](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ManagedScaling.html#ECS-Type-ManagedScaling-instanceWarmupPeriod) is 300 seconds, which you can configure to a lower value by using the [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateCapacityProvider.html](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateCapacityProvider.html) or [https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateCapacityProvider.html](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateCapacityProvider.html) APIs for more responsive scaling.
## Spare capacity
If your capacity provider has no container instances available for placing tasks, then it needs to increase (scale out) cluster capacity by launching Amazon EC2 instances on the fly, and wait for them to boot up before it can launch containers on them. This can significantly lower the task launch rate. You have two options here.
In this case, having spare Amazon EC2 capacity already launched and ready to run tasks will increase the effective task launch rate. You can use the `Target Capacity` configuration to indicate that you wish to maintain spare capacity in your clusters. For example, by setting `Target Capacity` at 80%, you indicate that your cluster needs 20% spare capacity at all times. This spare capacity can allow any standalone tasks to be immediately launched, ensuring task launches are not throttled. The trade-off for this approach is potential increased costs of keeping spare cluster capacity.
An alternate approach you can consider is adding headroom to your service, not to the capacity provider. This means that instead of reducing `Target Capacity` configuration to launch spare capacity, you can increase the number of replicas in your service by modifying the target tracking scaling metric or the step scaling thresholds of the service auto scaling. Note that this approach will only be helpful for spiky workloads, but won't have an effect when you’re deploying new services and going from 0 to N tasks for the first time. For more information about the related scaling policies, see [Target Tracking Scaling Policies](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-autoscaling-targettracking.html) or [Step Scaling Policies](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-autoscaling-stepscaling.html)
# Access Amazon ECS features with account settings
You can go into Amazon ECS account settings to opt in or out of specific features. For each AWS Region, you can opt in to, or opt out of, each account setting at the account-level or for a specific user or role.
You might want to opt in or out of specific features if any of the following is relevant to you:
+ A user or role can opt in or opt out specific account settings for their individual account.
+ A user or role can set the default opt-in or opt-out setting for all users on the account.
+ The root user or a user with administrator privileges can opt in to, or opt out of, any specific role or user on the account. If the account setting for the root user is changed, it sets the default for all the users and roles that no individual account setting was selected for.
**Note**
Federated users assume the account setting of the root user and can't have explicit account settings set for them separately.
The following account settings are available. You must separately opt-in and opt-out to each account setting.
| Resource name | Learn more |
| --- | --- |
| containerInsights | [Container Insights](#container-insights-setting) |
| serviceLongArnFormat`taskLongArnFormat` `containerInstanceLongArnFormat` | [Amazon Resource Names (ARNs) and IDs](#ecs-resource-ids) |
| tagResourceAuthorization | [Tagging authorization](#tag-resources-setting) |
| fargateFIPSMode | [AWS Fargate Federal Information Processing Standard (FIPS-140) compliance](#fips-setting) |
| fargateTaskRetirementWaitPeriod | [AWS Fargate task retirement wait time](#fargate-retirement-wait-time) |
| fargateEventWindows | [AWS Fargate task retirements using EC2 event windows](#fargate-event-windows) |
| guardDutyActivate | [Runtime Monitoring (Amazon GuardDuty integration)](#guard-duty-integration) |
| dualStackIPv6 | [Dual stack IPv6 VPC](#dual-stack-setting) |
| awsvpcTrunking | [Increase Linux container instance network interfaces](#vpc-trunking-setting) |
| defaultLogDriverMode | [Default log driver mode](#default-log-driver-mode) |
## Amazon Resource Names (ARNs) and IDs
When Amazon ECS resources are created, each resource is assigned a unique Amazon Resource Name (ARN) and resource identifier (ID). If you use a command line tool or the Amazon ECS API to work with Amazon ECS, resource ARNs or IDs are required for certain commands. For example, if you use the [stop-task](https://docs.aws.amazon.com/cli/latest/reference/ecs/stop-task.html) AWS CLI command to stop a task, you must specify the task ARN or ID in the command.
Amazon ECS introduced a new format for Amazon Resource Names (ARNs) and resource IDs for Amazon ECS services, tasks, and container instances. The opt-in status for each resource type determines the Amazon Resource Name (ARN) format the resource uses. You must opt in to the new ARN format to use features such as resource tagging for that resource type.
You can opt in to and opt out of the new Amazon Resource Name (ARN) and resource ID format on a per-Region basis. Currently, any new account created is opted in by default.
You can opt in or opt out of the new Amazon Resource Name (ARN) and resource ID format at any time. After you opt in, any new resources that you create use the new format.
**Note**
A resource ID doesn't change after it's created. Therefore, opting in or out of the new format doesn't affect your existing resource IDs.
The following sections describe how ARN and resource ID formats are changing. For more information about the transition to the new formats, see [Amazon Elastic Container Service FAQ](https://aws.amazon.com/ecs/faqs/).
**Amazon Resource Name (ARN) format**
Some resources have a user-friendly name, such as a service named `production`. In other cases, you must specify a resource using the Amazon Resource Name (ARN) format. The new ARN format for Amazon ECS tasks, services, and container instances includes the cluster name. For information about opting in to the new ARN format, see [Modifying Amazon ECS account settings](ecs-modifying-longer-id-settings.md).
The following table shows both the current format and the new format for each resource type.
| Resource type | ARN |
| --- | --- |
| Container instance | Current: `arn:aws:ecs:region:aws_account_id:container-instance/container-instance-id` New: `arn:aws:ecs:region:aws_account_id:container-instance/cluster-name/container-instance-id` |
| Amazon ECS service | Current: `arn:aws:ecs:region:aws_account_id:service/service-name` New: `arn:aws:ecs:region:aws_account_id:service/cluster-name/service-name` |
| Amazon ECS task | Current: `arn:aws:ecs:region:aws_account_id:task/task-id` New: `arn:aws:ecs:region:aws_account_id:task/cluster-name/task-id` |
**Resource ID length**
A resource ID takes the form of a unique combination of letters and numbers. New resource ID formats include shorter IDs for Amazon ECS tasks and container instances. The current resource ID format is 36 characters long. The new IDs are in a 32-character format that doesn't include any hyphens. For information about opting in to the new resource ID format, see [Modifying Amazon ECS account settings](ecs-modifying-longer-id-settings.md).
The default is `enabled`.
Only resources launched after opting in receive the new ARN and resource ID format. All existing resources aren't affected. For Amazon ECS services and tasks to transition to the new ARN and resource ID formats, you must recreate the service or task. To transition a container instance to the new ARN and resource ID format, the container instance must be drained and a new container instance must be launched and registered to the cluster.
**Note**
Tasks launched by an Amazon ECS service can only receive the new ARN and resource ID format if the service was created on or after November 16, 2018, and the user who created the service has opted in to the new format for tasks.
## ARN and resource ID format timeline
The timeline for the opt-in and opt-out periods for the new Amazon Resource Name (ARN) and resource ID format for Amazon ECS resources ended on April 1, 2021. By default, all accounts are opted in to the new format. All new resources created receive the new format, and you can no longer opt out.
## Container Insights
On December 2, 2024, AWS released Container Insights with enhanced observability for Amazon ECS. This version supports enhanced observability for Amazon ECS clusters using Fargate Amazon ECS Managed Instances, and EC2. After you configure Container Insights with enhanced observability on Amazon ECS, Container Insights auto-collects detailed infrastructure telemetry from cluster level down to the container level in your environment and displays your data in dashboards that show you a variety of metrics and dimensions. You can then use these out-of-the-box dashboards on the Container Insights console to better understand your container health and performance, and to mitigate issues faster by identifying anomalies.
We recommend that you use Container Insights with enhanced observability instead of Container Insights because it provides detailed visibility in your container environment, reducing the mean time to resolution. For more information, see [Amazon ECS Container Insights with enhanced observability metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-enhanced-observability-metrics-ECS.html) in the *Amazon CloudWatch User Guide*.
The default for the `containerInsights` account setting is `disabled`.
### Container Insights with enhanced observability
Use the following command to turn on Container Insights with enhanced observability.
Set the `containerInsights` account setting to `enhanced`.
```
aws ecs put-account-setting --name containerInsights --value enhanced
```
Example output
```
{
"setting": {
"name": "containerInsights",
"value": "enhanced",
"principalArn": "arn:aws:iam::123456789012:johndoe",
"type": user
}
}
```
After you set this account setting, all new clusters automatically use Container Insights with enhanced observability. Use the `update-cluster-settings` command to add Container Insights with enhanced observability to an existing cluster, or to upgrade a cluster from Container Insights to Container Insights with enhanced observability.
```
aws ecs update-cluster-settings --cluster cluster-name --settings name=containerInsights,value=enhanced
```
You can also use the console to configure Container Insights with enhanced observability. For more information, see [Modifying Amazon ECS account settings](ecs-modifying-longer-id-settings.md).
### Container Insights
When you set the `containerInsights` account setting to `enabled`, all new clusters have Container Insights enabled by default. You can modify existing clusters by using `update-cluster-settings`.
To use Container Insights, set the `containerInsights` account setting to `enabled`. Use the following command to turn on Container Insights.
```
aws ecs put-account-setting --name containerInsights --value enabled
```
Example output
```
{
"setting": {
"name": "containerInsights",
"value": "enabled",
"principalArn": "arn:aws:iam::123456789012:johndoe",
"type": user
}
}
```
When you set the `containerInsights` account setting to `enabled`, all new clusters have Container Insights enabled by default. Use the `update-cluster-settings` command to add Container Insights to an existing cluster.
```
aws ecs update-cluster-settings --cluster cluster-name --settings name=containerInsights,value=enabled
```
You can also use the console to configure Container Insights. For more information, see [Modifying Amazon ECS account settings](ecs-modifying-longer-id-settings.md).
## AWS Fargate Federal Information Processing Standard (FIPS-140) compliance
Fargate supports the Federal Information Processing Standard (FIPS-140) which specifies the security requirements for cryptographic modules that protect sensitive information. It is the current United States and Canadian government standard, and is applicable to systems that are required to be compliant with Federal Information Security Management Act (FISMA) or Federal Risk and Authorization Management Program (FedRAMP).
The resource name is `fargateFIPSMode`.
The default is `disabled`.
You must turn on Federal Information Processing Standard (FIPS-140) compliance on Fargate. For more information, see [AWS Fargate Federal Information Processing Standard (FIPS-140)](ecs-fips-compliance.md).
**Important**
The `fargateFIPSMode` account setting can only be changed using either the Amazon ECS API or the AWS CLI. For more information, see [Modifying Amazon ECS account settings](ecs-modifying-longer-id-settings.md).
Run `put-account-setting-default` with the `fargateFIPSMode` option set to `enabled`. For more information, see, [put-account-setting-default](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting-default.html) in the *Amazon Elastic Container Service API Reference*.
+ You can use the following command to turn on FIPS-140 compliance.
```
aws ecs put-account-setting-default --name fargateFIPSMode --value enabled
```
Example output
```
{
"setting": {
"name": "fargateFIPSMode",
"value": "enabled",
"principalArn": "arn:aws:iam::123456789012:root",
"type": user
}
}
```
You can run `list-account-settings` to view the current FIPS-140 compliance status. Use the `effective-settings` option to view the account level settings.
```
aws ecs list-account-settings --effective-settings
```
## Tagging authorization
Amazon ECS is introducing tagging authorization for resource creation. Users must have tagging permissions for actions that create the resource, such as `ecsCreateCluster`. When you create a resource and specify tags for that resource, AWS performs additional authorization to verify that there are permissions to create tags. Therefore, you must grant explicit permissions to use the `ecs:TagResource` action. For more information, see [Grant permission to tag resources on creation](supported-iam-actions-tagging.md).
In order to opt in to tagging authorization, run `put-account-setting-default` with the `tagResourceAuthorization` option set to `on`. For more information, see, [put-account-setting-default](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting-default.html) in the *Amazon Elastic Container Service API Reference*. You can run `list-account-settings` to view the current tagging authorization status.
+ You can use the following command to enable tagging authorization.
```
aws ecs put-account-setting-default --name tagResourceAuthorization --value on --region region
```
Example output
```
{
"setting": {
"name": "tagResourceAuthorization",
"value": "on",
"principalArn": "arn:aws:iam::123456789012:root",
"type": user
}
}
```
After you enable tagging authorization, you must configure the appropriate permissions to allow users to tag resources on creation. For more information, see [Grant permission to tag resources on creation](supported-iam-actions-tagging.md).
You can run `list-account-settings` to view the current tagging authorization status. Use the `effective-settings` option to view the account level settings.
```
aws ecs list-account-settings --effective-settings
```
## Tagging authorization timeline
You can confirm whether tagging authorization is active by running `list-account-settings` to view the `tagResourceAuthorization` value. When the value is `on`, it means that the tagging authorization is in use. For more information, see, [list-account-settings](https://docs.aws.amazon.com/cli/latest/reference/ecs/list-account-settings.html) in the *Amazon Elastic Container Service API Reference*.
The following are the important dates related to tagging authorization.
+ April 18, 2023 – Tagging authorization is introduced. All new and existing accounts must opt in to use the feature. You can opt in to start using tagging authorization. By opting in, you must grant the appropriate permissions.
+ February 9, 2024 - March 6, 2024 – All new accounts and non-impacted existing accounts have tagging authorization on by default. You can enable or disable the `tagResourceAuthorization` account setting to verify your IAM policy.
AWS has notified impacted accounts.
To disable the feature, run `put-account-setting-default` with the `tagResourceAuthorization` option set to `off`.
+ March 7, 2024 – If you have enabled tagging authorization, you can no longer disable the account setting.
We recommend that you complete your IAM policy testing before this date.
+ March 29, 2024 – All accounts use tagging authorization. The account-level setting will no longer be available in the Amazon ECS console or AWS CLI.
## AWS Fargate task retirement wait time
AWS sends out notifications when you have Fargate tasks running on a platform version revision marked for retirement. For more information, see [Task retirement and maintenance for AWS Fargate on Amazon ECS](task-maintenance.md).
AWS is responsible for patching and maintaining the underlying infrastructure for AWS Fargate. When AWS determines that a security or infrastructure update is needed for an Amazon ECS task hosted on Fargate, the tasks need to be stopped and new tasks launched to replace them. You can configure the wait period before tasks are retired for patching. You have the option to retire the task immediately, to wait 7 calendar days, or to wait 14 calendar days.
This setting is at the account-level.
You can configure the time that Fargate starts the task retirement. The default wait period is 7 days. For workloads that require immediate application of the updates, choose the immediate setting (`0`). If you need more time, configure the `7`, or `14` day option.
We recommend that you choose a shorter waiting period in order to pick up newer platform versions revisions sooner.
Configure the wait period by running `put-account-setting-default` or `put-account-setting` as the root user or an administrative user. Use the `fargateTaskRetirementWaitPeriod` option for the `name` and the `value` option set to one of the following values:
+ `0` - AWS sends the notification, and immediately starts to retire the affected tasks.
+ `7` - AWS sends the notification, and waits 7 calendar days before starting to retire the affected tasks. This is the default.
+ `14` - AWS sends the notification, and waits 14 calendar days before starting to retire the affected tasks.
For more information, see, [put-account-setting-default](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting-default.html) and [put-account-setting](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting.html) in the *Amazon Elastic Container Service API Reference*.
You can run the following command to set the wait period to 14 days.
```
aws ecs put-account-setting-default --name fargateTaskRetirementWaitPeriod --value 14
```
Example output
```
{
"setting": {
"name": "fargateTaskRetirementWaitPeriod",
"value": "14",
"principalArn": "arn:aws:iam::123456789012:root",
"type: user"
}
}
```
You can run `list-account-settings` to view the current Fargate task retirement wait time. Use the `effective-settings` option.
```
aws ecs list-account-settings --effective-settings
```
## AWS Fargate task retirements using EC2 event windows
AWS is responsible for patching and maintaining the underlying infrastructure for AWS Fargate. When AWS determines that a security or infrastructure update is needed for an Amazon ECS task hosted on Fargate, the tasks need to be stopped and new tasks launched to replace them. By enabling this account setting AWS Fargate will use EC2 event windows to determine the time and day when your tasks will be retired. Note that this is a one time enablement for an account.
To enable usage of EC2 event windows for Fargate task retirements, you can use the following CLI command:
```
aws ecs put-account-setting-default --name fargateEventWindows --value enabled
```
This is an account level setting. You can associate EC2 event windows with Fargate tasks at the account, cluster and service levels using the following instance tags:
+ `aws:ecs:serviceArn` : for service tasks
+ `aws:ecs:clusterArn` : for tasks belonging to a cluster
+ `aws:ecs:fargateTask` : set to true to target all Fargate tasks in the account
To learn more about how [ Amazon EC2 event windows](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/event-windows.html) work for your Amazon ECS tasks running on Fargate, see [Step 1: Set the task wait time or use Amazon EC2 event windows](prepare-task-retirement.md#prepare-task-retirement-set-time)
## Increase Linux container instance network interfaces
Each Amazon ECS task that uses the `awsvpc` network mode receives its own elastic network interface (ENI), which is attached to the container instance that hosts it. There is a default limit to the number of network interfaces that can be attached to an Amazon EC2 instance, and the primary network interface counts as one. For example, by default a `c5.large` instance may have up to three ENIs attached to it. The primary network interface for the instance counts as one, so you can attach an additional two ENIs to the instance. Because each task using the `awsvpc` network mode requires an ENI, you can typically only run two such tasks on this instance type.
Amazon ECS supports launching container instances with increased ENI density using supported Amazon EC2 instance types. When you use these instance types and turn on the `awsvpcTrunking` account setting, additional ENIs are available on newly launched container instances. This configuration allows you to place more tasks on each container instance.
For example, a `c5.large` instance with `awsvpcTrunking` has an increased ENI limit of twelve. The container instance will have the primary network interface and Amazon ECS creates and attaches a "trunk" network interface to the container instance. So this configuration allows you to launch ten tasks on the container instance instead of the current two tasks.
## Runtime Monitoring (Amazon GuardDuty integration)
Runtime Monitoring is an intelligent threat detection service that protects workloads running on Fargate and EC2 container instances by continuously monitoring AWS log and networking activity to identify malicious or unauthorized behavior.
The `guardDutyActivate` parameter is read-only in Amazon ECS and indicates whether Runtime Monitoring is turned on or off by your security administrator in your Amazon ECS account. GuardDuty controls this account setting on your behalf. For more information, see [Protecting Amazon ECS workloads with Runtime Monitoring](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-guard-duty-integration.html).
You can run `list-account-settings` to view the current GuardDuty integration setting.
```
aws ecs list-account-settings
```
Example output
```
{
"setting": {
"name": "guardDutyActivate",
"value": "on",
"principalArn": "arn:aws:iam::123456789012:doej",
"type": aws-managed"
}
}
```
## Dual stack IPv6 VPC
Amazon ECS supports providing tasks with an IPv6 address in addition to the primary private IPv4 address.
For tasks to receive an IPv6 address, the task must use the `awsvpc` network mode, must be launched in a VPC configured for dual-stack mode, and the `dualStackIPv6` account setting must be enabled. For more information about other requirements, see [Using a VPC in dual-stack mode](task-networking-awsvpc.md#task-networking-vpc-dual-stack) for EC2 capacity, [Using a VPC in dual-stack mode](managed-instances-awsvpc-mode.md#managed-instance-networking-vpc-dual-stack) for Amazon ECS Managed Instances capacity, and [Using a VPC in dual-stack mode](fargate-task-networking.md#fargate-task-networking-vpc-dual-stack) for Fargate capacity.
**Important**
The `dualStackIPv6` account setting can only be changed using either the Amazon ECS API or the AWS CLI. For more information, see [Modifying Amazon ECS account settings](ecs-modifying-longer-id-settings.md).
If you had a running task using the `awsvpc` network mode in an IPv6 enabled subnet between the dates of October 1, 2020 and November 2, 2020, the default `dualStackIPv6` account setting in the Region that the task was running in is `disabled`. If that condition isn't met, the default `dualStackIPv6` setting in the Region is `enabled`.
The default is `disabled`.
## Default log driver mode
Amazon ECS supports setting a default delivery mode of log messages from a container to the chosen log driver. The delivery mode affects application stability when the flow of logs from the container to the log driver is interrupted.
The `defaultLogDriverMode` setting supports two values: `blocking` and `non-blocking`. For more information about these delivery modes, see [LogConfiguration](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LogConfiguration.html) in the *Amazon Elastic Container Service API Reference*.
If you don't specify a delivery mode in your container definition's `logConfiguration`, the mode you specify using this account setting will be used as the default.
The default delivery mode is `non-blocking`.
**Note**
On June 25, 2025, Amazon ECS changed the default log driver mode from `blocking` to `non-blocking` to prioritize task availability over logging. To continue using the `blocking` mode after this change, do one of the following:
Set the `mode` option in your container definition's `logConfiguration` as `blocking`.
Set the `defaultLogDriverMode` account setting to `blocking`.
To set a default log driver mode to `blocking`, you can run the following command.
```
aws ecs put-account-setting-default --name defaultLogDriverMode --value "blocking"
```
# Viewing Amazon ECS account settings using the console
View your account settings in the console to see which features you have access to.
**Important**
The `dualStackIPv6`, `fargateFIPSMode`, `fargateTaskRetirementWaitPeriod` and the `fargateEventWindows` account settings can only be viewed or changed using the AWS CLI.
1. Open the console at [https://console.aws.amazon.com/ecs/v2](https://console.aws.amazon.com/ecs/v2).
1. In the navigation bar at the top, select the Region for which to view your account settings.
1. In the navigation page, choose **Account Settings**.
# Modifying Amazon ECS account settings
Modify your account settings to access Amazon ECS features.
The `guardDutyActivate` parameter is read-only in Amazon ECS and indicates whether Runtime Monitoring is turned on or off by your security administrator in your Amazon ECS account. GuardDuty controls this account setting on your behalf. For more information, see [Protecting Amazon ECS workloads with Runtime Monitoring](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-guard-duty-integration.html).
**Important**
The `dualStackIPv6`, `fargateFIPSMode`, `fargateTaskRetirementWaitPeriod` and the `fargateEventWindows` account settings can only be viewed or changed using the AWS CLI.
1. Open the console at [https://console.aws.amazon.com/ecs/v2](https://console.aws.amazon.com/ecs/v2).
1. In the navigation bar at the top, select the Region for which to view your account settings.
1. In the navigation page, choose **Account Settings**.
1. Choose **Update**.
1. (Optional) To increase or decrease the number of tasks that you can run in the awsvpc network mode for each EC2 instance, under **AWSVPC Trunking**, select **AWSVPC Trunking**.
1. (Optional) To use, or stop using CloudWatch Container Insights by default for clusters, under **CloudWatch Container Insights observability**, choose one of the following options:
+ To use Container Insights with enhanced observability, choose **Container Insights with enhanced observability**.
+ To use Container Insights, choose **Container Insights**.
+ To stop using Container Insights, choose **Turned off**.
1. (Optional) To enable or disable tagging authorization, under **Resource Tagging Authorization**, select or clear **Resource Tagging Authorization**.
1. (Optional) To configure a default log driver mode for when a log delivery mode isn't defined in a container's `logConfiguration`, under **Default log driver mode**, choose one of the following options:
+ To set the default log driver mode as `blocking`, choose **Blocking**.
+ To set the default log driver mode as `non-blocking`, choose **Non-blocking**.
1. Choose **Save changes**.
1. On the confirmation screen, choose **Confirm** to save the selection.
## Next steps
If you turned on Container Insights with enhanced observability or Container Insights, you can optionally update your existing clusters to use the feature. For more information, see [Updating an Amazon ECS cluster](update-cluster-v2.md).
# Reverting to the default Amazon ECS account settings
You can use the AWS Management Console to revert your Amazon ECS account settings to the default.
The **Revert to account default** option is only available when your account settings are no longer the default settings.
1. Open the console at [https://console.aws.amazon.com/ecs/v2](https://console.aws.amazon.com/ecs/v2).
1. In the navigation bar at the top, select the Region for which to view your account settings.
1. In the navigation page, choose **Account Settings**.
1. Choose **Update**.
1. Choose **Revert to account default**.
1. On the confirmation screen, choose **Confirm** to save the selection.
# Managing Amazon ECS account settings using the AWS CLI
You can manage your account settings using the Amazon ECS API, AWS CLI or SDKs. The `dualStackIPv6`, `fargateFIPSMode`, `fargateTaskRetirementWaitPeriod` and the `fargateEventWindows` account settings can only be viewed or changed using those tools.
**Note**
You can use dual-stack service endpoints to interact with Amazon ECS from the AWS CLI, SDKs, and the Amazon ECS API over both IPv4 and IPv6. For more information, see [Using Amazon ECS dual-stack endpoints](dual-stack-endpoint.md).
For information about the available API actions for task definitions see [Account setting actions](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/OperationList-query-account.html) in the *Amazon Elastic Container Service API Reference*.
Use one of the following commands to modify the default account setting for all users or roles on your account. These changes apply to the entire AWS account unless a user or role explicitly overrides these settings for themselves.
+ [put-account-setting-default](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting-default.html) (AWS CLI)
```
aws ecs put-account-setting-default --name serviceLongArnFormat --value enabled --region us-east-2
```
You can also use this command to modify other account settings. To do this, replace the `name` parameter with the corresponding account setting.
+ [Write-ECSAccountSetting](https://docs.aws.amazon.com/powershell/latest/reference/items/Write-ECSAccountSetting.html) (AWS Tools for Windows PowerShell)
```
Write-ECSAccountSettingDefault -Name serviceLongArnFormat -Value enabled -Region us-east-1 -Force
```
**To modify the account settings for your user account (AWS CLI)**
Use one of the following commands to modify the account settings for your user. If you’re using these commands as the root user, changes apply to the entire AWS account unless a; user or role explicitly overrides these settings for themselves.
+ [put-account-setting](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting.html) (AWS CLI)
```
aws ecs put-account-setting --name serviceLongArnFormat --value enabled --region us-east-1
```
You can also use this command to modify other account settings. To do this, replace the `name` parameter with the corresponding account setting.
+ [Write-ECSAccountSetting](https://docs.aws.amazon.com/powershell/latest/reference/items/Write-ECSAccountSetting.html) (AWS Tools for Windows PowerShell)
```
Write-ECSAccountSetting -Name serviceLongArnFormat -Value enabled -Force
```
**To modify the account settings for a specific user or role (AWS CLI)**
Use one of the following commands and specify the ARN of a user, role, or root user in the request to modify the account settings for a specific user or role.
+ [put-account-setting](https://docs.aws.amazon.com/cli/latest/reference/ecs/put-account-setting.html) (AWS CLI)
```
aws ecs put-account-setting --name serviceLongArnFormat --value enabled --principal-arn arn:aws:iam::aws_account_id:user/principalName --region us-east-1
```
You can also use this command to modify other account settings. To do this, replace the `name` parameter with the corresponding account setting.
+ [Write-ECSAccountSetting](https://docs.aws.amazon.com/powershell/latest/reference/items/Write-ECSAccountSetting.html) (AWS Tools for Windows PowerShell)
```
Write-ECSAccountSetting -Name serviceLongArnFormat -Value enabled -PrincipalArn arn:aws:iam::aws_account_id:user/principalName -Region us-east-1 -Force
```
# IAM roles for Amazon ECS
An IAM role is an IAM identity that you can create in your account that has specific permissions. In Amazon ECS, you can create roles to grant permissions to Amazon ECS resource such as containers or services.
The roles Amazon ECS requires depend on the task definition launch type and the features that you use. Use the following table to determine which IAM roles you need for Amazon ECS.
| Role | Definition | When required | More information |
| --- | --- | --- | --- |
| Task execution role | This role allows Amazon ECS to use other AWS services on your behalf. | Your task is hosted on AWS Fargate or on external instances and: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-iam-role-overview.html) Your task is hosted on either AWS Fargate or Amazon EC2 instances and: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-iam-role-overview.html) | [Amazon ECS task execution IAM role](task_execution_IAM_role.md) |
| Task role | This role allows your application code (on the container) to use other AWS services. | Your application accesses other AWS services, such as Amazon S3. | [Amazon ECS task IAM role](task-iam-roles.md) |
| Container instance role | This role allows your EC2 instances or external instances to register with the cluster. | Your task is hosted on Amazon EC2 instances or an external instance. | [Amazon ECS container instance IAM role](instance_IAM_role.md) |
| Amazon ECS Anywhere role | This role allows your external instances to access AWS APIs. | Your task is hosted on external instances. | [Amazon ECS Anywhere IAM role](iam-role-ecsanywhere.md) |
| Amazon ECS infrastructure for load balancers role | This role allows Amazon ECS to manage load balancer resources in your clusters on your behalf for blue/green deployments. | You want to use Amazon ECS blue/green deployments. | [Amazon ECS infrastructure IAM role for load balancers](AmazonECSInfrastructureRolePolicyForLoadBalancers.md) |
| Amazon ECS CodeDeploy role | This role allows CodeDeploy to make updates to your services. | You use the CodeDeploy blue/green deployment type to deploy services. | [Amazon ECS CodeDeploy IAM Role](codedeploy_IAM_role.md) |
| Amazon ECS EventBridge role | This role allows EventBridge to make updates to your services. | You use the EventBridge rules and targets to schedule your tasks. | [Amazon ECS EventBridge IAM Role](CWE_IAM_role.md) |
| Amazon ECS infrastructure role | This role allows Amazon ECS to manage infrastructure resources in your clusters. | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-iam-role-overview.html) | [Amazon ECS infrastructure IAM role](infrastructure_IAM_role.md) |
| Instance profile | This role allows allows Amazon ECS Managed Instances to assume the infrastructure role securely. | You use Amazon ECS Managed Instances in your clusters. | [Amazon ECS Managed Instances instance profile](managed-instances-instance-profile.md) |