Created by Dr. Rahul Sharad Gaikwad (AWS) and Tamilselvan P (AWS)
Environment: PoC or pilot | Technologies: Infrastructure; DevOps | Workload: All other workloads |
AWS services: AWS Service Catalog; Amazon EC2 |
AWS Service Catalog supports self-service provisioning with governance for your HashiCorp Terraform configurations. If you use Terraform, you can use Service Catalog as the single tool to organize, govern, and distribute your Terraform configurations within AWS at scale. You can access Service Catalog key features, including cataloging of standardized and pre-approved infrastructure as code (IaC) templates, access control, cloud resources provisioning with least privilege access, versioning, sharing to thousands of AWS accounts, and tagging. End users, such as engineers, database administrators, and data scientists, see a list of products and versions they have access to, and they can deploy them through a single action.
This pattern helps you deploy AWS resources by using Terraform code. The Terraform code in the GitHub repository is accessed through Service Catalog. Using this approach, you integrate the products with your existing Terraform workflows. Administrators can create Service Catalog portfolios and add AWS Launch Wizard products to them by using Terraform.
The following are the benefits of this solution:
Because of the rollback feature in Service Catalog, if any issues occur during deployment, you can revert the product to a previous version.
You can easily identify the differences between product versions. This helps you resolve issues during deployment.
You can configure a repository connection in Service Catalog, such as to GitHub or GitLab. You can make product changes directly through the repository.
For information about the overall benefits of AWS Service Catalog, see What is Service Catalog.
Prerequisites
An active AWS account.
A GitHub, BitBucket, or other repository that contains Terraform configuration files in ZIP format.
AWS Serverless Application Model Command Line Interface (AWS SAM CLI), installed.
AWS Command Line Interface (AWS CLI), installed and configured.
Go, installed.
Python version 3.9 , installed. AWS SAM CLI requires this version of Python.
Permissions to write and run AWS Lambda functions and permissions to access and manage Service Catalog products and portfolios.
The diagram shows the following workflow:
When a Terraform configuration is ready, a developer creates a .zip file that contains all of the Terraform code. The developer uploads the .zip file into the code repository that is connected to Service Catalog.
An administrator associates the Terraform product to a portfolio in Service Catalog. The administrator also creates a launch constraint that allows end users to provision the product.
In Service Catalog, end users launch AWS resources by using the Terraform configuration. They can choose which product version to deploy.
AWS services
AWS Lambda is a compute service that helps you run code without needing to provision or manage servers. It runs your code only when needed and scales automatically, so you pay only for the compute time that you use.
AWS Service Catalog helps you centrally manage catalogs of IT services that are approved for AWS. End users can quickly deploy only the approved IT services they need, following the constraints set by your organization.
Other services
Code repository
If you require sample Terraform configurations that you can deploy through Service Catalog, you can use the configurations in the GitHub Amazon Macie Organization Setup Using Terraform repository. Use of the code samples in this repository is not required.
Instead of providing the values for variables in the Terraform configuration file (terraform.tfvars), configure variable values when launching product through Service Catalog.
Grant access to the portfolio only to specific users or administrators.
Follow the principle of least privilege and grant the minimum permissions required to perform a task. For more information, see Grant least privilege and Security best practices in the AWS Identity and Access Management (IAM) documentation.
| Task | Description | Skills required |
|---|
(Optional) Install Docker. | If you want to run the AWS Lambda functions in your development environment, install Docker. For instructions, see Install Docker Engine in the Docker documentation. | DevOps engineer |
Install the AWS Service Catalog Engine for Terraform. | Enter the following command to clone the AWS Service Catalog Engine for Terraform repository. git clone https://github.com/aws-samples/service-catalog-engine-for-terraform-os.git
Navigate to the the root directory of the cloned repository. Enter the following command. This installs the engine. run ./bin/bash/deploy-tre.sh -r
The AWS Region set in your default profile is not used during the automated installation. Instead, you provide the Region when you run this command.
| DevOps engineer, AWS administrator |
| Task | Description | Skills required |
|---|
Create a connection to the GitHub repository. | Sign in to the AWS Management Console, and then open the Developer Tools console. You can access the Developer Tools console by choosing a service such as AWS CodePipeline or AWS CodeDeploy. In the left navigation pane, choose Settings, and then choose Connections. Choose Create connection. Select the repository where you maintain the Terraform source code. For example, you can choose Bitbucket, GitHub, or GitHub Enterprise Server. Enter a name for the connection, and then choose Connect. When you are prompted, authenticate the repository. After authentication is complete, the connection is created and the status changes to active.
| AWS administrator |
| Task | Description | Skills required |
|---|
Create the Service Catalog product. | Open the AWS Service Catalog console. Navigate to the Administration section, and then choose Product list. Choose Create product. On the Create product page in the Product details section, choose the External product type. Service Catalog uses this product type to support Terraform Community Edition products. Enter a name and owner for the Service Catalog product. Select Specify your code repository using a CodeStar provider. Enter the following information for your repository: Connect to your provider using AWS CodeConnections – Select the connection you created previously. Repository – Select the repository. Branch – Select the branch. Template file path – Choose the path where the code template file is stored. The file name should end with tar.gz.
Under Version name and description, provide information about the product version. Choose Create product.
| AWS administrator |
Create a portfolio. | Open the AWS Service Catalog console. Navigate to the Administration section, and then choose, choose Portfolios. Choose Create portfolio. Enter the following values: Portfolio name – Sample terraform Portfolio description – Sample portfolio for Terraform configurations Owner – Your contact information, such as an email address
Choose Create.
| AWS administrator |
Add the Terraform product to the portfolio. | Open the AWS Service Catalog console. Navigate to the Administration section, and then choose Product list. Select the Terraform product that you created previously. Choose Actions, and then choose Add product to portfolio. Choose the Sample terraform portfolio. Choose Add product to portfolio.
| AWS administrator |
Create the access policy. | Open the AWS Identity and Access Management (IAM) console. In the navigation pane, choose Policies. In the content pane, choose Create policy. Choose the JSON option. Enter the sample JSON policy in Access policy in the Additional information section of this pattern. Choose Next. On the Review and create page, in the Policy name box, enter TerraformResourceCreationAndArtifactAccessPolicy. Choose Create policy.
| AWS administrator |
Create a custom trust policy. | Open the IAM console. In the navigation pane, choose Roles. Choose Create role. Under Trusted entity type, choose Custom trust policy. In the JSON policy editor, enter the sample JSON policy in Trust policy in the Additional information section of this pattern. Choose Next. Under Permissions policies, choose the TerraformResourceCreationAndArtifactAccessPolicy that you previously created. Choose Next. Under Role details, in the Role name box, enter SCLaunch-product. Important: The role name must begin with SCLaunch. Choose Create role.
| AWS administrator |
Add a launch constraint to the Service Catalog product. | Sign in to the AWS Management Console as a user with administrative permissions. Open the AWS Service Catalog console. In the navigation pane, choose Portfolios. Choose the portfolio that you created previously. On the Portfolio details page, choose the Constraints tab, and then choose Create constraint. For Product, select the Terraform product you created previously. Under Launch constraint, for Method, choose Enter role name. In the Role name box, enter SCLaunch-product. Choose Create.
| AWS administrator |
Grant access to the product. | Open the AWS Service Catalog console. In the navigation pane, choose Portfolios. Choose the portfolio that you created previously. Choose the Access tab, and then choose Grant access. Choose the Roles tab, and then select the role that should have access to deploy this product. Choose Grant Access.
| AWS administrator |
Launch the product. | Sign in to the AWS Management Console as a user with permissions to deploy the Service Catalog product. Open the AWS Service Catalog console. In the navigation pane, choose Products. Choose the produce you created previously, and then choose Launch product. Enter a product name and define any required parameters. Choose Launch product.
| DevOps engineer |
| Task | Description | Skills required |
|---|
Validate the deployment. | There are two AWS Step Functions state machines for the Service Catalog provisioning workflow: ManageProvisionedProductStateMachine – Service Catalog invokes this state machine when provisioning a new Terraform product and when updating an existing Terraform provisioned product.
TerminateProvisionedProductStateMachine – Service Catalog invokes this state machine when terminating an existing Terraform provisioned product.
You check the logs for the ManageProvisionedProductStateMachine state machine to confirm that the product was provisioned. Sign in to the AWS Management Console, and then open the AWS Step Functions console. In the left navigation pane, choose State machines. Choose ManageProvisionedProductStateMachine. In the Executions list, enter the provisioned product ID to locate the execution. Note: The state file backend bucket names start with sc-terraform-engine-state-. Validate that all the required resources have been created in the account.
| DevOps engineer |
| Task | Description | Skills required |
|---|
Delete provisioned products. | Sign in to the AWS Management Console as a user with permissions to deploy the Service Catalog product. Open the AWS Service Catalog console. In the left navigation, choose Provisioned products. Select the product you created. In the Actions list, choose Terminate. In the confirmation text box, enter terminate, and then choose Terminate provisioned product. Repeat these steps to terminate all provisioned products.
| DevOps engineer |
Remove the AWS Service Catalog Engine for Terraform. | Sign in to the AWS Management Console as a user with administrative permissions. Open the Amazon Simple Storage Service (Amazon S3) console. In the navigation pane, choose Buckets. Select the sc-terraform-engine-logging-XXXX bucket. Choose Empty. Repeat steps 4–5 for the following buckets: Open the AWS CloudFormation console, and then validate you're in the correct AWS Region. In the left navigation, choose Stacks. Select SAM-TRE, and then choose Delete. Wait until the stack has been deleted. Select Bootstrap-TRE, and then choose Delete. Wait until the stack has been deleted.
| AWS administrator |
AWS documentation
Terraform documentation
Access policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "s3:GetObject",
"Resource": "*",
"Condition": {
"StringEquals": {
"s3:ExistingObjectTag/servicecatalog:provisioning": "true"
}
}
},
{
"Action": [
"s3:CreateBucket*",
"s3:DeleteBucket*",
"s3:Get*",
"s3:List*",
"s3:PutBucketTagging"
],
"Resource": "arn:aws:s3:::*",
"Effect": "Allow"
},
{
"Action": [
"resource-groups:CreateGroup",
"resource-groups:ListGroupResources",
"resource-groups:DeleteGroup",
"resource-groups:Tag"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"tag:GetResources",
"tag:GetTagKeys",
"tag:GetTagValues",
"tag:TagResources",
"tag:UntagResources"
],
"Resource": "*",
"Effect": "Allow"
}
]
}
Trust policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GivePermissionsToServiceCatalog",
"Effect": "Allow",
"Principal": {
"Service": "servicecatalog.amazonaws.com"
},
"Action": "sts:AssumeRole"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::account_id:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringLike": {
"aws:PrincipalArn": [
"arn:aws:iam::accounti_id:role/TerraformEngine/TerraformExecutionRole*",
"arn:aws:iam::accounti_id:role/TerraformEngine/ServiceCatalogExternalParameterParserRole*",
"arn:aws:iam::accounti_id:role/TerraformEngine/ServiceCatalogTerraformOSParameterParserRole*"
]
}
}
}
]
}
