# Create an API-driven resource orchestration framework using GitHub Actions and Terragrunt
*Tamilselvan P, Abhigyan Dandriyal, Sandeep Gawande, and Akash Kumar, Amazon Web Services*
## Summary
This pattern leverages GitHub Actions workflows to automate resource provisioning through standardized JSON payloads, eliminating the need for manual configuration. This automated pipeline manages the complete deployment lifecycle and can seamlessly integrate with various frontend systems, from custom UI components to ServiceNow. The solution’s flexibility allows users to interact with the system through their preferred interfaces while maintaining standardized processes.
The configurable pipeline architecture can be adapted to meet different organizational requirements. The example implementation focuses on Amazon Virtual Private Cloud (Amazon VPC) and Amazon Simple Storage Service (Amazon S3) provisioning. The pattern effectively addresses common cloud resources management challenges by standardizing requests across the organization and providing consistent integration points. This approach makes it easier for teams to request and manage resources while ensuring standardization.
## Prerequisites and limitations
**Prerequisites**
+ An active AWS account
+ An active GitHub account with access to the configured repository
**Limitations**
+ New resources require manual addition of `terragrunt.hcl` files to the repository configuration.
+ Some AWS services aren’t available in all AWS Regions. For Region availability, see [AWS Services by Region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). For specific endpoints, see [Service endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/aws-service-information.html), and choose the link for the service.
## Architecture
The following diagram shows the components and workflow of this pattern.
The architecture diagram shows the following actions:
1. The user submits a JSON payload to GitHub Actions, triggering the automation pipeline.
1. The GitHub Actions pipeline retrieves the required resources code from the Terragrunt and Terraform repositories, based on the payload specifications.
1. The pipeline assumes the appropriate AWS Identity and Access Management (IAM) role using the specified AWS account ID. Then, the pipeline deploys the resources to the target AWS account and manages Terraform state using the account-specific Amazon S3 bucket and Amazon DynamoDB table.
Each AWS account contains IAM roles for secure access, an Amazon S3 bucket for Terraform state storage, and a DynamoDB table for state locking. This design enables controlled, automated resource deployment across AWS accounts. The deployment process maintains proper state management and access control through dedicated Amazon S3 buckets and IAM roles in each account.
## Tools
**AWS services**
+ [Amazon DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html) is a fully managed NoSQL database service that provides fast, predictable, and scalable performance.
+ [AWS Identity and Access Management (IAM)](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) helps you securely manage access to your AWS resources by controlling who is authenticated and authorized to use them.
+ [Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) is a cloud-based object storage service that helps you store, protect, and retrieve any amount of data.
+ [Amazon Virtual Private Cloud (Amazon VPC)](https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html) helps you launch AWS resources into a virtual network that you’ve defined. This virtual network resembles a traditional network that you’d operate in your own data center, with the benefits of using the scalable infrastructure of AWS.
**Other tools**
+ [GitHub Actions](https://docs.github.com/en/actions) is a continuous integration and continuous delivery (CI/CD) platform that’s tightly integrated with GitHub repositories. You can use GitHub Actions to automate your build, test, and deployment pipeline.
+ [Terraform](https://www.terraform.io/) is an infrastructure as code (IaC) tool from HashiCorp that helps you create and manage cloud and on-premises resources.
+ [Terragrunt](https://terragrunt.gruntwork.io/docs/getting-started/overview/) is an orchestration tool that extends both OpenTofu and Terraform capabilities. It manages how generic infrastructure patterns are applied, making it easier to scale and maintain large infrastructure estates.
**Code repository**
The code for this pattern is available in the GitHub [sample-aws-orchestration-pipeline-terraform](https://github.com/aws-samples/sample-aws-orchestration-pipeline-terraform) repository.
## Best practices
+ Store AWS credentials and sensitive data using GitHub repository secrets for secure access.
+ Configure the OpenID Connect (OIDC) provider for GitHub Actions to assume the IAM role, avoiding static credentials.
+ Follow the principle of least privilege and grant the minimum permissions required to perform a task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html#grant-least-priv) and [Security best practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) in the IAM documentation.
## Epics
### Create and configure repository
| Task | Description | Skills required |
| --- | --- | --- |
| Initialize the GitHub repository. | To initialize the GitHub repository, use the following steps:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html) | DevOps engineer |
| Configure the IAM roles and permissions. | To configure the IAM roles and permissions, use the following steps:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html) | DevOps engineer |
| Set up GitHub secrets and variables. | For instructions about how to set up repository secrets and variables in the GitHub repository, see [Creating configuration variables for a repository](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-variables#creating-configuration-variables-for-a-repository) in the GitHub documentation. Configure the following variables:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html) | DevOps engineer |
| Create the repository structure. | To create the repository structure, use the following steps:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html) | DevOps engineer |
### Trigger the pipeline and validate results
| Task | Description | Skills required |
| --- | --- | --- |
| Execute the pipeline using curl. | To execute the pipeline by using [curl](https://curl.se/), use the following steps:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html)
For more information about the pipeline execution process, see [Additional information](#create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt-additional). | DevOps engineer |
| Validate results of the pipeline execution | To validate the results, use the following steps:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html)
You can also cross-verify the created resources by using the `output.json` file created in the repository that’s inside the same resource as the `terragrunt.hcl` file. | DevOps engineer |
### Clean up resources
| Task | Description | Skills required |
| --- | --- | --- |
| Submit a cleanup request. | To delete resources that are no longer required, use the following steps:[See the AWS documentation website for more details](http://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/create-an-api-driven-resource-orchestration-framework-using-github-actions-and-terragrunt.html) | DevOps engineer |
## Related resources
**AWS Blogs**
+ [Use IAM roles to connect GitHub Actions to actions in AWS](https://aws.amazon.com/blogs/security/use-iam-roles-to-connect-github-actions-to-actions-in-aws/)
**AWS service documentation**
+ [IAM role creation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create.html)
+ [Monitoring CloudTrail log files with Amazon CloudWatch Logs](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/monitor-cloudtrail-log-files-with-cloudwatch-logs.html)
+ [Security best practices for Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html)
**GitHub resources**
+ [Create a repository dispatch event](https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#create-a-repository-dispatch-event)
+ [Creating webhooks](https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks#payload)
+ [Implement strong access controls on the GitHub repository](https://docs.github.com/en/get-started/learning-about-github/access-permissions-on-github)
+ [Regularly audit repository access](https://docs.github.com/en/organizations/keeping-your-organization-secure/managing-security-settings-for-your-organization)
+ [Security checks in the CI/CD pipeline](https://github.com/marketplace/actions/checkov-github-action)
+ [Use multi-factor authentication for GitHub accounts](https://docs.github.com/en/authentication/securing-your-account-with-two-factor-authentication-2fa/configuring-two-factor-authentication)
## Additional information
**Pipeline execution process**
Following are the steps of the pipeline execution:
1. **Validates JSON payload format** - Ensures that the incoming JSON configuration is properly structured and contains all required parameters
1. **Assumes specified IAM role** - Authenticates and assumes the required IAM role for AWS operations
1. **Downloads required Terraform and Terragrunt code** - Retrieves the specified version of resource code and dependencies
1. **Executes resource deployment** - Applies the configuration to deploy or update AWS resources in the target environment
**Sample payload used for VPC creation**
Following is example code for Terraform backend state bucket creation:
```
state_bucket_name = "${local.payload.ApplicationName}-${local.payload.EnvironmentId}-tfstate"
```
```
lock_table_name = "${local.payload.ApplicationName}-${local.payload.EnvironmentId}-tfstate-lock"
```
Following is an example payload for creating a VPC with Amazon VPC, where `vpc_cidr` defines the [CIDR block](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-cidr-blocks.html) specifications for the VPC. The Terraform state bucket is mapped to a variable defined in the `terraform` files. The `ref` parameter contains the branch name of the code to execute.
```
{
"ref": "main",
"inputs": {
"RequestParameters": {
"RequestId": "1111111",
"RequestType": "create",
"ResourceType": "vpc",
"AccountId": "1234567890",
"AccountAlias": "account-alias",
"RegionId": "us-west-2",
"ApplicationName": "myapp",
"DivisionName": "division-name",
"EnvironmentId": "dev",
"Suffix": "poc"
},
"ResourceParameters": [
{
"VPC": {
"vpc_cidr": "10.0.0.0/16"
}
}
]
}
}
```
`RequestParameters` are used to track the request status in the pipeline section and `tfstate` is created based on this information. The following parameters contain metadata and control information:
+ `RequestId` – Unique identifier for the request
+ `RequestType` – Type of operation (create, update, or delete)
+ `ResourceType` – Type of resource to be provisioned
+ `AccountId` – Target AWS account for deployment
+ `AccountAlias` – Friendly name for the AWS account
+ `RegionId` – AWS Region for resource deployment
+ `ApplicationName` – Name of the application
+ `DivisionName` – Organization division
+ `EnvironmentId` – Environment (for example, dev and prod)
+ `Suffix` – Additional identifier for the resources
`ResourceParameters` contain resource-specific configuration that maps to variables defined in the Terraform files. Any custom variables that need to be passed to the Terraform modules should be included in `ResourceParameters`. The parameter `vpc_cidr` is mandatory for Amazon VPC.