Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

Preferences
Contact Us
Feedback
AWS Documentation
Create an AWS Account
Manage blue/green deployments of microservices to multiple accounts and Regions by using AWS code services and AWS KMS multi-Region keys - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpicsTroubleshootingRelated resources

Manage blue/green deployments of microservices to multiple accounts and Regions by using AWS code services and AWS KMS multi-Region keys

PDF

Created by Balaji Vedagiri (AWS), Ashish Kumar (AWS), Faisal Shahdad (AWS), Anand Krishna Varanasi (AWS), Vanitha Dontireddy (AWS), and Vivek Thangamuthu (AWS)

Summary

Notice: AWS CodeCommit is no longer available to new customers. Existing customers of AWS CodeCommit can continue to use the service as normal. Learn more

This pattern describes how to deploy a global microservices application from a central AWS account to multiple workload accounts and Regions in accordance with a blue/green deployment strategy. The pattern supports the following:

  • Software is developed in a central account, whereas workloads and applications are spread across multiple accounts and AWS Regions.

  • A single AWS Key Management System (AWS KMS) multi-Region key is used for encryption and decryption to cover disaster recovery.

  • The KMS key is Region-specific and has to be maintained or created in three different Regions for pipeline artifacts. A KMS multi-Region key helps retain the same key ID across Regions.

  • The Git workflow branching model is implemented with two branches (development and main) and code is merged by using pull requests (PRs). The AWS Lambda function that is deployed from this stack creates a PR from the development branch to the main branch. The PR merge to the main branch initiates an AWS CodePipeline pipeline, which orchestrates the continuous integration and continuous delivery (CI/CD) flow and deploys the stacks across accounts.

This pattern  provides a sample infrastructure as code (IaC) setup through AWS CloudFormation stacks to demonstrate this use case. The blue/green deployment of microservices is implemented by using AWS CodeDeploy.

Prerequisites and limitations

Prerequisites

  • Four active AWS accounts:

    • A tools account to manage the code pipeline and maintain the AWS CodeCommit repository.

    • Three workload (test) accounts for deploying the microservices workload.

  • This pattern uses the following Regions. If  you want to use other Regions, you must make the appropriate modifications to the AWS CodeDeploy and AWS KMS multi-Region stacks.

    • Tools (AWS CodeCommit) account: ap-south-1

    • Workload (test) account 1: ap-south-1

    • Workload (test) account 2: eu-central-1

    • Workload (test) account 3: us-east-1

  • Three Amazon Simple Storage Service (Amazon S3) buckets for the deployment Regions in each workload account. (These are called S3BUCKETNAMETESTACCOUNT1, S3BUCKETNAMETESTACCOUNT2 and S3BUCKETNAMETESTACCOUNT3 later in this pattern.)

    For example, you can create these buckets in specific accounts and Regions with unique bucket names as follows (replace xxxx with a random number):

    ##In Test Account 1
aws s3 mb s3://ecs-codepipeline-xxxx-ap-south-1 --region ap-south-1
##In Test Account 2
aws s3 mb s3://ecs-codepipeline-xxxx-eu-central-1 --region eu-central-1
##In Test Account 3
aws s3 mb s3://ecs-codepipeline-xxxx-us-east-1 --region us-east-1

#Example
##In Test Account 1
aws s3 mb s3://ecs-codepipeline-18903-ap-south-1 --region ap-south-1
##In Test Account 2
aws s3 mb s3://ecs-codepipeline-18903-eu-central-1 --region eu-central-1
##In Test Account 3
aws s3 mb s3://ecs-codepipeline-18903-us-east-1 --region us-east-1

Limitations

The pattern uses AWS CodeBuild and other configuration files to deploy a sample microservice. If you have a different workload type (for example, serverless), you must update all relevant configurations.

Architecture

Target technology stack

  • AWS CloudFormation

  • AWS CodeCommit

  • AWS CodeBuild

  • AWS CodeDeploy

  • AWS CodePipeline

Target architecture

Target architecture for deploying microservices to multiple accounts and Regions

Automation and scale

The setup is automated by using AWS CloudFormation stack templates (IaC). It can be easily scaled for multiple environments and accounts.

Tools

AWS services

  • AWS CloudFormation helps you set up AWS resources, provision them quickly and consistently, and manage them throughout their lifecycle across AWS accounts and Regions.

  • AWS CodeBuild is a fully managed build service that helps you compile source code, run unit tests, and produce artifacts that are ready to deploy.

  • AWS CodeCommit is a version control service that helps you privately store and manage Git repositories, without needing to manage your own source control system.

  • AWS CodeDeploy automates deployments to Amazon Elastic Compute Cloud (Amazon EC2) or on-premises instances, AWS Lambda functions, or Amazon Elastic Container Service (Amazon ECS) services.

  • AWS CodePipeline helps you quickly model and configure the different stages of a software release and automate the steps required to release software changes continuously.

  • Amazon Elastic Container Registry (Amazon ECR) is a managed container image registry service that’s secure, scalable, and reliable.

  • Amazon Elastic Container Service (Amazon ECS) is a fast and scalable container management service that helps you run, stop, and manage containers on a cluster.

  • AWS Key Management Service (AWS KMS) helps you create and control cryptographic keys to help protect your data.

  • Amazon Simple Storage Service (Amazon S3) is a cloud-based object storage service that helps you store, protect, and retrieve any amount of data.

Additional tools

  • Git is an open-source, distributed version control system that works with the AWS CodeCommit repository.

  • Docker is a set of platform as a service (PaaS) products that use virtualization at the operating-system level to deliver software in containers. This pattern uses Docker to build and test container images locally.

  • cfn-lint and cfn-nag are open-source tools that help you review CloudFormation stacks for any errors and security issues.

Code repository

The code for this pattern is available in the GitHub Global Blue/Green deployments in multiple regions and accounts repository.

Epics

TaskDescriptionSkills required

Export environment variables for CloudFormation stack deployment.

Define environment variables that will be used as input to the CloudFormation stacks later in this pattern.

  1. Update the bucket names that you created in the three accounts and Regions as explained earlier in the Prerequisites section:

    export S3BUCKETNAMETESTACCOUNT1=<S3BUCKETACCOUNT1>
export S3BUCKETNAMETESTACCOUNT2=<S3BUCKETACCOUNT2>
export S3BUCKETNAMETESTACCOUNT3=<S3BUCKETACCOUNT3>

  2. Define a random string to create artifact buckets, because bucket names must be unique globally:

    export BUCKETSTARTNAME=ecs-codepipeline-artifacts-19992

  3. Define and export the account IDs and Regions:

    export TOOLSACCOUNT=<TOOLSACCOUNT>
export CODECOMMITACCOUNT=<CODECOMMITACCOUNT>
export CODECOMMITREGION=ap-south-1
export CODECOMMITREPONAME=Poc
export TESTACCOUNT1=<TESTACCOUNT1>
export TESTACCOUNT2=<TESTACCOUNT2>
export TESTACCOUNT3=<TESTACCOUNT3>
export TESTACCOUNT1REGION=ap-south-1
export TESTACCOUNT2REGION=eu-central-1
export TESTACCOUNT3REGION=us-east-1
export TOOLSACCOUNTREGION=ap-south-1
export ECRREPOSITORYNAME=web
AWS DevOps

Set up environment variables

TaskDescriptionSkills required

Export environment variables for CloudFormation stack deployment.

Define environment variables that will be used as input to the CloudFormation stacks later in this pattern.

  1. Update the bucket names that you created in the three accounts and Regions as explained earlier in the Prerequisites section:

    export S3BUCKETNAMETESTACCOUNT1=<S3BUCKETACCOUNT1>
export S3BUCKETNAMETESTACCOUNT2=<S3BUCKETACCOUNT2>
export S3BUCKETNAMETESTACCOUNT3=<S3BUCKETACCOUNT3>

  2. Define a random string to create artifact buckets, because bucket names must be unique globally:

    export BUCKETSTARTNAME=ecs-codepipeline-artifacts-19992

  3. Define and export the account IDs and Regions:

    export TOOLSACCOUNT=<TOOLSACCOUNT>
export CODECOMMITACCOUNT=<CODECOMMITACCOUNT>
export CODECOMMITREGION=ap-south-1
export CODECOMMITREPONAME=Poc
export TESTACCOUNT1=<TESTACCOUNT1>
export TESTACCOUNT2=<TESTACCOUNT2>
export TESTACCOUNT3=<TESTACCOUNT3>
export TESTACCOUNT1REGION=ap-south-1
export TESTACCOUNT2REGION=eu-central-1
export TESTACCOUNT3REGION=us-east-1
export TOOLSACCOUNTREGION=ap-south-1
export ECRREPOSITORYNAME=web
AWS DevOps
TaskDescriptionSkills required

Clone the repository.

Clone the sample repository into a new repository in your work location:

##In work location
git clone https://github.com/aws-samples/ecs-blue-green-global-deployment-with-multiregion-cmk-codepipeline.git
AWS DevOps

Package the Cloudformation resources.

In this step, you package the local artifacts that the CloudFormation templates reference to create the infrastructure resources required for services such as Amazon Virtual Private Cloud (Amazon VPC) and Application Load Balancer.

The templates are available in the Infra folder of the code repository.

##In TestAccount1##
aws cloudformation package \
    --template-file mainInfraStack.yaml \
    --s3-bucket $S3BUCKETNAMETESTACCOUNT1 \
    --s3-prefix infraStack \
    --region $TESTACCOUNT1REGION \
    --output-template-file infrastructure_${TESTACCOUNT1}.template
##In TestAccount2##
aws cloudformation package \
    --template-file mainInfraStack.yaml \
    --s3-bucket $S3BUCKETNAMETESTACCOUNT2 \
    --s3-prefix infraStack \
    --region $TESTACCOUNT2REGION \
    --output-template-file infrastructure_${TESTACCOUNT2}.template
##In TestAccount3##
aws cloudformation package \
    --template-file mainInfraStack.yaml \
    --s3-bucket $S3BUCKETNAMETESTACCOUNT3 \
    --s3-prefix infraStack \
    --region $TESTACCOUNT3REGION \
    --output-template-file infrastructure_${TESTACCOUNT3}.template
AWS DevOps

Validate the package templates.

Validate the package templates:

aws cloudformation validate-template \
    --template-body file://infrastructure_${TESTACCOUNT1}.template

aws cloudformation validate-template \
    --template-body file://infrastructure_${TESTACCOUNT2}.template

aws cloudformation validate-template \
    --template-body file://infrastructure_${TESTACCOUNT3}.template
AWS DevOps

Deploy the package files into the workload accounts,

  1. Update the placeholder values and account names in the infraParameters.json script based on your setup.

  2. Deploy the package templates into your three workload accounts.

    ##In TestAccount1##
aws cloudformation deploy \
    --template-file infrastructure_${TESTACCOUNT1}.template \
    --stack-name mainInfrastack \
    --parameter-overrides file://infraParameters.json \
    --region $TESTACCOUNT1REGION \
    --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

##In TestAccount2##
aws cloudformation deploy \
    --template-file infrastructure_${TESTACCOUNT2}.template \
    --stack-name mainInfrastack \
    --parameter-overrides file://infraParameters.json \
    --region $TESTACCOUNT2REGION \
    --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

##In TestAccount3##
aws cloudformation deploy \
    --template-file infrastructure_${TESTACCOUNT3}.template \
    --stack-name mainInfrastack \
    --parameter-overrides file://infraParameters.json \
    --region $TESTACCOUNT3REGION \
    --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM
AWS DevOps

Package and deploy the CloudFormation stacks for the infrastructure

TaskDescriptionSkills required

Clone the repository.

Clone the sample repository into a new repository in your work location:

##In work location
git clone https://github.com/aws-samples/ecs-blue-green-global-deployment-with-multiregion-cmk-codepipeline.git
AWS DevOps

Package the Cloudformation resources.

In this step, you package the local artifacts that the CloudFormation templates reference to create the infrastructure resources required for services such as Amazon Virtual Private Cloud (Amazon VPC) and Application Load Balancer.

The templates are available in the Infra folder of the code repository.

##In TestAccount1##
aws cloudformation package \
    --template-file mainInfraStack.yaml \
    --s3-bucket $S3BUCKETNAMETESTACCOUNT1 \
    --s3-prefix infraStack \
    --region $TESTACCOUNT1REGION \
    --output-template-file infrastructure_${TESTACCOUNT1}.template
##In TestAccount2##
aws cloudformation package \
    --template-file mainInfraStack.yaml \
    --s3-bucket $S3BUCKETNAMETESTACCOUNT2 \
    --s3-prefix infraStack \
    --region $TESTACCOUNT2REGION \
    --output-template-file infrastructure_${TESTACCOUNT2}.template
##In TestAccount3##
aws cloudformation package \
    --template-file mainInfraStack.yaml \
    --s3-bucket $S3BUCKETNAMETESTACCOUNT3 \
    --s3-prefix infraStack \
    --region $TESTACCOUNT3REGION \
    --output-template-file infrastructure_${TESTACCOUNT3}.template
AWS DevOps

Validate the package templates.

Validate the package templates:

aws cloudformation validate-template \
    --template-body file://infrastructure_${TESTACCOUNT1}.template

aws cloudformation validate-template \
    --template-body file://infrastructure_${TESTACCOUNT2}.template

aws cloudformation validate-template \
    --template-body file://infrastructure_${TESTACCOUNT3}.template
AWS DevOps

Deploy the package files into the workload accounts,

  1. Update the placeholder values and account names in the infraParameters.json script based on your setup.

  2. Deploy the package templates into your three workload accounts.

    ##In TestAccount1##
aws cloudformation deploy \
    --template-file infrastructure_${TESTACCOUNT1}.template \
    --stack-name mainInfrastack \
    --parameter-overrides file://infraParameters.json \
    --region $TESTACCOUNT1REGION \
    --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

##In TestAccount2##
aws cloudformation deploy \
    --template-file infrastructure_${TESTACCOUNT2}.template \
    --stack-name mainInfrastack \
    --parameter-overrides file://infraParameters.json \
    --region $TESTACCOUNT2REGION \
    --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM

##In TestAccount3##
aws cloudformation deploy \
    --template-file infrastructure_${TESTACCOUNT3}.template \
    --stack-name mainInfrastack \
    --parameter-overrides file://infraParameters.json \
    --region $TESTACCOUNT3REGION \
    --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM
AWS DevOps
TaskDescriptionSkills required

Push a sample image to the Amazon ECR repository.

Push a sample (NGINX) image to the Amazon Elastic Container Registry (Amazon ECR) repository named web (as set in parameters). You can customize the image as required.

To log in and set the credentials for pushing an image to Amazon ECR, follow the instructions in the Amazon ECR documentation.

The commands are:

  docker pull nginx
  docker images
  docker tag <imageid> aws_account_id.dkr.ecr.region.amazonaws.com/<web>:latest
  docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<web>:tag
AWS DevOps

Scale Amazon ECS and verify access.

  1. Scale Amazon ECS to create two replicas:

    aws ecs update-service --cluster QA-Cluster --service Poc-Service --desired-count 2

    where Poc-Service refers to your sample application.

  2. Verify that the services are accessible from the Application Load Balancer by using a fully qualified domain name (FQDN) or DNS from a browser or by using the curl command.

AWS DevOps

Push a sample image and scale Amazon ECS

TaskDescriptionSkills required

Push a sample image to the Amazon ECR repository.

Push a sample (NGINX) image to the Amazon Elastic Container Registry (Amazon ECR) repository named web (as set in parameters). You can customize the image as required.

To log in and set the credentials for pushing an image to Amazon ECR, follow the instructions in the Amazon ECR documentation.

The commands are:

  docker pull nginx
  docker images
  docker tag <imageid> aws_account_id.dkr.ecr.region.amazonaws.com/<web>:latest
  docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/<web>:tag
AWS DevOps

Scale Amazon ECS and verify access.

  1. Scale Amazon ECS to create two replicas:

    aws ecs update-service --cluster QA-Cluster --service Poc-Service --desired-count 2

    where Poc-Service refers to your sample application.

  2. Verify that the services are accessible from the Application Load Balancer by using a fully qualified domain name (FQDN) or DNS from a browser or by using the curl command.

AWS DevOps
TaskDescriptionSkills required

Create a CodeCommit repository in the tools account.

Create a CodeCommit repository in the tools account by using the codecommit.yaml template, which is in the code folder of the GitHub repository. You must create this repository only in the single Region where you plan to develop the code.

aws cloudformation deploy --stack-name codecommitrepoStack --parameter-overrides  CodeCommitReponame=$CODECOMMITREPONAME \
ToolsAccount=$TOOLSACCOUNT --template-file codecommit.yaml  --region $TOOLSACCOUNTREGION \
--capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Create an S3 bucket for managing artifacts generated by CodePipeline.

Create an S3 bucket for managing artifacts generated by CodePipeline by using the pre-reqs-bucket.yaml template, which is in the code folder of the GitHub repository. The stacks must be deployed in all three workload (test) and tools accounts and Regions.

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TOOLSACCOUNTREGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Set up a multi-Region KMS key.

  1. Create a multi-Region KMS key with primary and replica keys that CodePipeline will use. In our example, ToolsAccount1region - ap-south-1 will be the primary Region.

    aws cloudformation deploy --stack-name ecs-codepipeline-pre-reqs-KMS \
--template-file pre-reqs_KMS.yaml --parameter-overrides \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT --region $TOOLSACCOUNTREGION

  2. Set the CMKARN variables to pass to CodeBuild projects. The values are available in the output of the ecs-codepipeline-pre-reqs-KMS template stack (the key ID will be same in all Regions and starts with mrk-). Or, you can get the CMKARN values from the tools account. Export them in all account sessions: 

    export CMKARN1=arn:aws:kms:ap-south-1:<TOOLSACCOUNTID>:key/mrk-xxx
export CMKARN2=arn:aws:kms:eu-central-1:<TOOLSACCOUNTID>:key/mrk-xxx
export CMKARN3=arn:aws:kms:us-east-1:<TOOLSACCOUNTID>:key/mrk-xxx
export CMARNTOOLS=arn:aws:kms:ap-south-1:<TOOLSACCOUNTID>:key/mrk-xxx
AWS DevOps

Set up the CodeBuild project in the tools account.

  1. Use the codebuild_IAM.yaml template from the code folder of the GitHub repository to set up AWS Identity and Access Management (IAM) for AWS CodeBuild in a single Region in the tools account:

    #In ToolsAccount
aws cloudformation deploy --stack-name ecs-codebuild-iam \
--template-file codebuild_IAM.yaml --region $TOOLSACCOUNTREGION \
--capabilities CAPABILITY_NAMED_IAM

  2. Use the codebuild.yaml template to set up CodeBuild for your build project. Deploy this template in all three Regions as follows:

    aws cloudformation deploy --stack-name ecscodebuildstack --parameter-overrides  ToolsAccount=$TOOLSACCOUNT \
CodeCommitRepoName=$CODECOMMITREPONAME ECRRepositoryName=$ECRREPOSITORYNAME APPACCOUNTID=$TESTACCOUNT1 \
TestAccount3=$TESTACCOUNT3 CodeCommitRegion=$CODECOMMITREGION CMKARN=$CMKARN1 \
--template-file codebuild.yaml  --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name ecscodebuildstack --parameter-overrides  ToolsAccount=$TOOLSACCOUNT \
CodeCommitRepoName=$CODECOMMITREPONAME ECRRepositoryName=$ECRREPOSITORYNAME APPACCOUNTID=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitRegion=$CODECOMMITREGION CMKARN=$CMKARN2 \
--template-file codebuild.yaml  --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name ecscodebuildstack --parameter-overrides  ToolsAccount=$TOOLSACCOUNT \
CodeCommitRepoName=$CODECOMMITREPONAME ECRRepositoryName=$ECRREPOSITORYNAME APPACCOUNTID=$TESTACCOUNT3 \
CodeCommitRegion=$CODECOMMITREGION CMKARN=$CMKARN3 \
--template-file codebuild.yaml  --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Set up CodeDeploy in workload accounts.

Use the codedeploy.yaml template in the code folder of the GitHub repository to set up CodeDeploy in all three workload accounts. The output of mainInfraStack includes the Amazon Resource Names (ARNs) of the Amazon ECS cluster and Application Load Balancer listener.

Note

The values from the infrastructure stacks are exported already, so they are imported by the CodeDeploy stack templates.

##WorkloadAccount1##
aws cloudformation deploy --stack-name ecscodedeploystack \
--parameter-overrides  ToolsAccount=$TOOLSACCOUNT mainInfrastackname=mainInfrastack \
--template-file codedeploy.yaml  --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

##WorkloadAccount2##
aws cloudformation deploy --stack-name ecscodedeploystack \
--parameter-overrides ToolsAccount=$TOOLSACCOUNT mainInfrastackname=mainInfrastack \
--template-file codedeploy.yaml  --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

##WorkloadAccount3##
aws cloudformation deploy --stack-name ecscodedeploystack \
--parameter-overrides ToolsAccount=$TOOLSACCOUNT mainInfrastackname=mainInfrastack \
--template-file codedeploy.yaml  --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Set up code services and resources

TaskDescriptionSkills required

Create a CodeCommit repository in the tools account.

Create a CodeCommit repository in the tools account by using the codecommit.yaml template, which is in the code folder of the GitHub repository. You must create this repository only in the single Region where you plan to develop the code.

aws cloudformation deploy --stack-name codecommitrepoStack --parameter-overrides  CodeCommitReponame=$CODECOMMITREPONAME \
ToolsAccount=$TOOLSACCOUNT --template-file codecommit.yaml  --region $TOOLSACCOUNTREGION \
--capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Create an S3 bucket for managing artifacts generated by CodePipeline.

Create an S3 bucket for managing artifacts generated by CodePipeline by using the pre-reqs-bucket.yaml template, which is in the code folder of the GitHub repository. The stacks must be deployed in all three workload (test) and tools accounts and Regions.

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TOOLSACCOUNTREGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Set up a multi-Region KMS key.

  1. Create a multi-Region KMS key with primary and replica keys that CodePipeline will use. In our example, ToolsAccount1region - ap-south-1 will be the primary Region.

    aws cloudformation deploy --stack-name ecs-codepipeline-pre-reqs-KMS \
--template-file pre-reqs_KMS.yaml --parameter-overrides \
TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT --region $TOOLSACCOUNTREGION

  2. Set the CMKARN variables to pass to CodeBuild projects. The values are available in the output of the ecs-codepipeline-pre-reqs-KMS template stack (the key ID will be same in all Regions and starts with mrk-). Or, you can get the CMKARN values from the tools account. Export them in all account sessions: 

    export CMKARN1=arn:aws:kms:ap-south-1:<TOOLSACCOUNTID>:key/mrk-xxx
export CMKARN2=arn:aws:kms:eu-central-1:<TOOLSACCOUNTID>:key/mrk-xxx
export CMKARN3=arn:aws:kms:us-east-1:<TOOLSACCOUNTID>:key/mrk-xxx
export CMARNTOOLS=arn:aws:kms:ap-south-1:<TOOLSACCOUNTID>:key/mrk-xxx
AWS DevOps

Set up the CodeBuild project in the tools account.

  1. Use the codebuild_IAM.yaml template from the code folder of the GitHub repository to set up AWS Identity and Access Management (IAM) for AWS CodeBuild in a single Region in the tools account:

    #In ToolsAccount
aws cloudformation deploy --stack-name ecs-codebuild-iam \
--template-file codebuild_IAM.yaml --region $TOOLSACCOUNTREGION \
--capabilities CAPABILITY_NAMED_IAM

  2. Use the codebuild.yaml template to set up CodeBuild for your build project. Deploy this template in all three Regions as follows:

    aws cloudformation deploy --stack-name ecscodebuildstack --parameter-overrides  ToolsAccount=$TOOLSACCOUNT \
CodeCommitRepoName=$CODECOMMITREPONAME ECRRepositoryName=$ECRREPOSITORYNAME APPACCOUNTID=$TESTACCOUNT1 \
TestAccount3=$TESTACCOUNT3 CodeCommitRegion=$CODECOMMITREGION CMKARN=$CMKARN1 \
--template-file codebuild.yaml  --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name ecscodebuildstack --parameter-overrides  ToolsAccount=$TOOLSACCOUNT \
CodeCommitRepoName=$CODECOMMITREPONAME ECRRepositoryName=$ECRREPOSITORYNAME APPACCOUNTID=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitRegion=$CODECOMMITREGION CMKARN=$CMKARN2 \
--template-file codebuild.yaml  --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name ecscodebuildstack --parameter-overrides  ToolsAccount=$TOOLSACCOUNT \
CodeCommitRepoName=$CODECOMMITREPONAME ECRRepositoryName=$ECRREPOSITORYNAME APPACCOUNTID=$TESTACCOUNT3 \
CodeCommitRegion=$CODECOMMITREGION CMKARN=$CMKARN3 \
--template-file codebuild.yaml  --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Set up CodeDeploy in workload accounts.

Use the codedeploy.yaml template in the code folder of the GitHub repository to set up CodeDeploy in all three workload accounts. The output of mainInfraStack includes the Amazon Resource Names (ARNs) of the Amazon ECS cluster and Application Load Balancer listener.

Note

The values from the infrastructure stacks are exported already, so they are imported by the CodeDeploy stack templates.

##WorkloadAccount1##
aws cloudformation deploy --stack-name ecscodedeploystack \
--parameter-overrides  ToolsAccount=$TOOLSACCOUNT mainInfrastackname=mainInfrastack \
--template-file codedeploy.yaml  --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

##WorkloadAccount2##
aws cloudformation deploy --stack-name ecscodedeploystack \
--parameter-overrides ToolsAccount=$TOOLSACCOUNT mainInfrastackname=mainInfrastack \
--template-file codedeploy.yaml  --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

##WorkloadAccount3##
aws cloudformation deploy --stack-name ecscodedeploystack \
--parameter-overrides ToolsAccount=$TOOLSACCOUNT mainInfrastackname=mainInfrastack \
--template-file codedeploy.yaml  --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps
TaskDescriptionSkills required

Create a code pipeline in the tools account.

In the tools account, run the command:

aws cloudformation deploy --stack-name ecscodepipelinestack --parameter-overrides  \
TestAccount1=$TESTACCOUNT1 TestAccount1Region=$TESTACCOUNT1REGION \
TestAccount2=$TESTACCOUNT2 TestAccount2Region=$TESTACCOUNT2REGION \
TestAccount3=$TESTACCOUNT3 TestAccount3Region=$TESTACCOUNT3REGION \
CMKARNTools=$CMKTROOLSARN CMKARN1=$CMKARN1 CMKARN2=$CMKARN2 CMKARN3=$CMKARN3 \
CodeCommitRepoName=$CODECOMMITREPONAME BucketStartName=$BUCKETSTARTNAME \
--template-file codepipeline.yaml --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Provide access for CodePipeline and CodeBuild roles in the AWS KMS key policy and S3 bucket policy.

  1. Provide access for CodePipeline and CodeBuild roles in the AWS KMS key policy:

    aws cloudformation deploy --stack-name ecs-codepipeline-pre-reqs-KMS \
--template-file pre-reqs_KMS.yaml --parameter-overrides \
CodeBuildCondition=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT --region $TOOLSACCOUNTREGION

  2. Update the S3 bucket policy to allow access for CodePipeline and CodeDeploy roles:

    aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TOOLSACCOUNTREGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Set up CodePipeline in the tools account

TaskDescriptionSkills required

Create a code pipeline in the tools account.

In the tools account, run the command:

aws cloudformation deploy --stack-name ecscodepipelinestack --parameter-overrides  \
TestAccount1=$TESTACCOUNT1 TestAccount1Region=$TESTACCOUNT1REGION \
TestAccount2=$TESTACCOUNT2 TestAccount2Region=$TESTACCOUNT2REGION \
TestAccount3=$TESTACCOUNT3 TestAccount3Region=$TESTACCOUNT3REGION \
CMKARNTools=$CMKTROOLSARN CMKARN1=$CMKARN1 CMKARN2=$CMKARN2 CMKARN3=$CMKARN3 \
CodeCommitRepoName=$CODECOMMITREPONAME BucketStartName=$BUCKETSTARTNAME \
--template-file codepipeline.yaml --capabilities CAPABILITY_NAMED_IAM
AWS DevOps

Provide access for CodePipeline and CodeBuild roles in the AWS KMS key policy and S3 bucket policy.

  1. Provide access for CodePipeline and CodeBuild roles in the AWS KMS key policy:

    aws cloudformation deploy --stack-name ecs-codepipeline-pre-reqs-KMS \
--template-file pre-reqs_KMS.yaml --parameter-overrides \
CodeBuildCondition=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT --region $TOOLSACCOUNTREGION

  2. Update the S3 bucket policy to allow access for CodePipeline and CodeDeploy roles:

    aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT1REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT2REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TESTACCOUNT3REGION --capabilities CAPABILITY_NAMED_IAM

aws cloudformation deploy --stack-name pre-reqs-artifacts-bucket --parameter-overrides BucketStartName=$BUCKETSTARTNAME \
PutS3BucketPolicy=true TestAccount1=$TESTACCOUNT1 TestAccount2=$TESTACCOUNT2 \
TestAccount3=$TESTACCOUNT3 CodeCommitAccount=$CODECOMMITACCOUNT ToolsAccount=$TOOLSACCOUNT \
--template-file pre-reqs_bucket.yaml --region $TOOLSACCOUNTREGION --capabilities CAPABILITY_NAMED_IAM
AWS DevOps
TaskDescriptionSkills required

Push changes to the CodeCommit repository.

  1. Clone the CodeCommit repository that was created in the codecommitrepoStack by using the git clone command, as described in the AWS CodeCommit documentation.

  2. Update the input artifacts with the required details:

    • JSON file: Update AccountID in the file in three places of this file. Rename the three files to include the account IDs.

    • YAML files: Update the task definition ARN and version. Rename the three files to include the account IDs.

  3. Modify the index.html file to make a few minor changes to the home page.

  4. Copy the following files to the repository and commit:

     index.html
 Dockerfile
 buildspec.yaml
 appspec_<accountid>.yaml (3 files - one per account )
 taskdef<accountid>.json (3 files - one per account)

  5. Start or restart the pipeline and verify the results.

  6. Access the service from the Application Load Balancer using by using an FQDN or DNS, and verify that the updates have been deployed.

Call and test the pipeline

TaskDescriptionSkills required

Push changes to the CodeCommit repository.

  1. Clone the CodeCommit repository that was created in the codecommitrepoStack by using the git clone command, as described in the AWS CodeCommit documentation.

  2. Update the input artifacts with the required details:

    • JSON file: Update AccountID in the file in three places of this file. Rename the three files to include the account IDs.

    • YAML files: Update the task definition ARN and version. Rename the three files to include the account IDs.

  3. Modify the index.html file to make a few minor changes to the home page.

  4. Copy the following files to the repository and commit:

     index.html
 Dockerfile
 buildspec.yaml
 appspec_<accountid>.yaml (3 files - one per account )
 taskdef<accountid>.json (3 files - one per account)

  5. Start or restart the pipeline and verify the results.

  6. Access the service from the Application Load Balancer using by using an FQDN or DNS, and verify that the updates have been deployed.

TaskDescriptionSkills required

Clean up all the deployed resources.

  1. Scale down Amazon ECS to zero instances:

    aws ecs update-service --cluster QA-Cluster --service Poc-Service --desired-count 0

  2. Delete the CloudFormation stacks in each account and Region:

    ##In Tools Account##
aws cloudformation delete-stack --stack-name ecscodepipelinestack --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name ecscodebuildstack --region $TESTACCOUNT1REGION
aws cloudformation delete-stack --stack-name ecscodebuildstack --region $TESTACCOUNT2REGION
aws cloudformation delete-stack --stack-name ecscodebuildstack --region $TESTACCOUNT3REGION
aws cloudformation delete-stack --stack-name ecs-codepipeline-pre-reqs-KMS --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name codecommitrepoStack --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TESTACCOUNT1REGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TESTACCOUNT2REGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TESTACCOUNT3REGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name ecs-codebuild-iam --region $TOOLSACCOUNTREGION

##NOTE: Artifact buckets will not get deleted if there are artifacts so it has to be emptied manually before deleting.##
    ##In Workload / Test Accounts##
##Account:1##
aws cloudformation delete-stack --stack-name ecscodedeploystack --region $TESTACCOUNT1REGION
aws cloudformation delete-stack --stack-name mainInfrastack --region $TESTACCOUNT1REGION
##Account:2##
aws cloudformation delete-stack --stack-name ecscodedeploystack --region $TESTACCOUNT2REGION
aws cloudformation delete-stack --stack-name mainInfrastack --region $TESTACCOUNT2REGION
##Account:3##
aws cloudformation delete-stack --stack-name ecscodedeploystack --region $TESTACCOUNT3REGION
aws cloudformation delete-stack --stack-name mainInfrastack --region $TESTACCOUNT3REGION
##NOTE: Amazon ECR (web) will not get deleted if the registry still includes images. It can be manually cleaned up if not required.

Clean up

TaskDescriptionSkills required

Clean up all the deployed resources.

  1. Scale down Amazon ECS to zero instances:

    aws ecs update-service --cluster QA-Cluster --service Poc-Service --desired-count 0

  2. Delete the CloudFormation stacks in each account and Region:

    ##In Tools Account##
aws cloudformation delete-stack --stack-name ecscodepipelinestack --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name ecscodebuildstack --region $TESTACCOUNT1REGION
aws cloudformation delete-stack --stack-name ecscodebuildstack --region $TESTACCOUNT2REGION
aws cloudformation delete-stack --stack-name ecscodebuildstack --region $TESTACCOUNT3REGION
aws cloudformation delete-stack --stack-name ecs-codepipeline-pre-reqs-KMS --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name codecommitrepoStack --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TESTACCOUNT1REGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TESTACCOUNT2REGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TESTACCOUNT3REGION
aws cloudformation delete-stack --stack-name pre-reqs-artifacts-bucket --region $TOOLSACCOUNTREGION
aws cloudformation delete-stack --stack-name ecs-codebuild-iam --region $TOOLSACCOUNTREGION

##NOTE: Artifact buckets will not get deleted if there are artifacts so it has to be emptied manually before deleting.##
    ##In Workload / Test Accounts##
##Account:1##
aws cloudformation delete-stack --stack-name ecscodedeploystack --region $TESTACCOUNT1REGION
aws cloudformation delete-stack --stack-name mainInfrastack --region $TESTACCOUNT1REGION
##Account:2##
aws cloudformation delete-stack --stack-name ecscodedeploystack --region $TESTACCOUNT2REGION
aws cloudformation delete-stack --stack-name mainInfrastack --region $TESTACCOUNT2REGION
##Account:3##
aws cloudformation delete-stack --stack-name ecscodedeploystack --region $TESTACCOUNT3REGION
aws cloudformation delete-stack --stack-name mainInfrastack --region $TESTACCOUNT3REGION
##NOTE: Amazon ECR (web) will not get deleted if the registry still includes images. It can be manually cleaned up if not required.

Troubleshooting

IssueSolution

Changes that you committed to the repository aren’t getting deployed.

  • Check the CodeBuild logs for errors in the Docker build action. For more information, see the CodeBuild documentation.

  • Check the CodeDeploy deployment for any Amazon ECS deployment issues.

Related resources

Need help?

PrivacySite termsCookie preferences
© 2025, Amazon Web Services, Inc. or its affiliates. All rights reserved.