Centralize software package distribution in AWS Organizations by using Terraform - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsBest practicesEpicsTroubleshootingRelated resources

Centralize software package distribution in AWS Organizations by using Terraform

Created by Pradip kumar Pandey (AWS), Aarti Rajput (AWS), Chintamani Aphale (AWS), T.V.R.L.Phani Kumar Dadi (AWS), Mayuri Shinde (AWS), and Pratap Kumar Nanda (AWS)

Environment: Production

Technologies: Management & governance; Infrastructure

AWS services: AWS Organizations; AWS Systems Manager

Summary

Enterprises often maintain multiple AWS accounts that are spread across multiple AWS Regions in order to create a strong isolation barrier between workloads. To stay secure and compliant, their administration teams install agent-based tools such as CrowdStrike, SentinelOne, or TrendMicro tools for security scanning, and the Amazon CloudWatch agent, Datadog Agent, or AppDynamics agents for monitoring. These teams often face challenges when they want to centrally automate software package management and distribution across this large landscape.

Distributor, a capability of AWS Systems Manager, automates the process of packaging and publishing software to managed Microsoft Windows and Linux instances across the cloud and on-premises servers through a single simplified interface. This pattern demonstrates how you can use Terraform to further simplify the process of managing the installation of software and to run scripts across a large number of instances and member accounts within AWS Organizations with minimal effort.

This solution works for Amazon, Linux, and Windows instances that are managed by Systems Manager.

Prerequisites and limitations

Architecture

Resource details

This pattern uses Account Factory for Terraform (AFT) to create all required AWS resources and the code pipeline to deploy the resources in a deployment account. The code pipeline runs in two repositories:

  • Global customization contains Terraform code that will run across all accounts registered with AFT.

  • Account customizations contains Terraform code that will run in the deployment account.

You can also deploy this solution without using AFT, by running Terraform commands in the account customizations folder.

The Terraform code deploys the following resources:

  • AWS Identity and Access Management (IAM) role and policies

  • Compressed files and manifest.json for the package

    • In Systems Manager, a package includes at least one .zip file of software or installable assets.

    • The JSON manifest includes pointers to your package code files.

  • S3 bucket

    • The distributed package that is shared across the organization is securely stored in an Amazon S3 bucket.

  • AWS Systems Manager documents (SSM documents)

    • DistributeSoftwarePackage contains the logic to distribute the software package to every target instance in the member accounts.

    • AddSoftwarePackageToDistributor contains the logic to package the installable software assets and add it to Automation, a capability of AWS Systems Manager.

  • Systems Manager association

    • A Systems Manager association is used to deploy the solution.

Architecture and workflow

Architecture diagram for centralizing software package distribution in AWS Organizations

The diagram illustrates the following steps:

  1. To run the solution from a centralized account, you upload your packages or software along with deployment steps to an S3 bucket.

  2. Your customized package becomes available in the Systems Manager console Documents section, in the Owned by me tab.

  3. State Manager, a capability of Systems Manager, creates, schedules, and runs an association for the package across the organization. The association specifies that the software package must be installed and running on a managed node before it can be installed on the target node.

  4. The association instructs Systems Manager to install the package on the target node.

  5. For any subsequent installations or changes, users can run the same association periodically or manually from a single location to perform deployments across accounts.

  6. In member accounts, Automation sends deployment commands to Distributor.

  7. Distributor distributes software packages across instances.

This solution uses the management account within AWS Organizations, but you can also designate an account (delegated administrator) to manage this on behalf of the organization.

Tools

AWS services

  • Amazon Simple Storage Service (Amazon S3) is a cloud-based object storage service that helps you store, protect, and retrieve any amount of data. This pattern uses Amazon S3 to centralize and securely store the distributed package.

  • AWS Systems Manager helps you manage your applications and infrastructure running in the AWS Cloud. It simplifies application and resource management, shortens the time to detect and resolve operational problems, and helps you manage your AWS resources securely at scale. This pattern uses the following Systems Manager capabilities:

    • Distributor helps you package and publish software to Systems Manager managed instances.

    • Automation simplifies common maintenance, deployment, and remediation tasks for many AWS services.

    • Documents performs actions on your Systems Manager managed instances across your organization and accounts.

  • AWS Organizations is an account management service that helps you consolidate multiple AWS accounts into an organization that you create and centrally manage.

Other tools

  • Terraform is an infrastructure as code (IaC) tool from HashiCorp that helps you create and manage cloud and on-premises resources.

Code repository

The instructions and code for this pattern are available in the GitHub Centralized package distribution repository.

Best practices

  • To assign tags to an association, use the AWS Command Line Interface (AWS CLI) or the AWS Tools for PowerShell. Adding tags to an association by using the Systems Manager console isn't supported. For more information, see Tagging Systems Manager resources in the Systems Manager documentation.

  • To run an association by using a new version of a document shared from another account, set the document version to default.

  • To tag only the target node, use one tag key. If you want to target your nodes by using multiple tag keys, use the resource group option.

Epics

TaskDescriptionSkills required

Clone the repository.

  1. Clone the GitHub Centralized package distribution repository:

    git clone https://github.com/aws-samples/aws-organization-centralised-package-distribution

  2. The Terraform code repository requires two customization folders that are managed by AFT. Confirm that your local copy of the repository contains these folders:

    $ cd centralised-package-distribution
$ ls
global-customization
account-customization
DevOps engineer

Update global variables.

Update the following input parameters in the global-customization/variables.tf file. These variables apply to all accounts that are created and managed by AFT.

  • account_id: The ID of the account where the Distributor solution will be deployed.

  • aws_region: The AWS Region where the association will be deployed.

DevOps engineer

Update account variables.

Update the following input parameters in the account-customization/variables.tf file. These variables apply only to specific accounts that are created and managed by AFT.

  • package_bucket_name: The name of the S3 bucket that contains the package distribution file.

  • package_name: The name of the package distribution file.

  • package_version: The package version of the installer.

DevOps engineer
TaskDescriptionSkills required

Update input parameters for the State Manager association.

Update the following input parameters in the account-customization/association.tf file to define the state you want to maintain on your instances. You can use the default parameter values if they support your use case.

  • targetAccounts: The organizational unit (OU) IDs within AWS Organizations that represent accounts with the target instances for distribution. OU IDs start with “ou”.

  • targetRegions: The AWS Regions (for example, “us-east-1″ or ”ap-southeast-2”) where the target instances are running.

  • action: Specify whether to install or uninstall the package.

  • installationType: One of the following installation types:

    • uninstall: The package is uninstalled.

    • reinstall: The application is taken offline until the reinstallation process is complete.

    • In-place update: The application is available while new or updated files are added to the installation.

  • name: The name of the package to install or uninstall.

  • version: The version of the package to install or uninstall. If no version of the package is installed, the system returns an error.

  • bucketName: The S3 bucket name the package has been deployed to. This bucket should consist of the packages and the manifest file only.

  • bucketPrefix: The S3 prefix where the package assets are stored.

  • AutomationAssumeRole: The Amazon Resource Name (ARN) of SystemsManager-AutomationAdministrationRole.

DevOps engineer

Prepare compressed files and the manifest.json file for the package.

This pattern provides sample PowerShell installable files (.msi for Windows and .rpm for Linux) with install and uninstall scripts in the account-customization/package folder.

  1. Replace the PowerShell installable files with your own files, or provide your installable file, install and uninstall scripts, and manifest file to create a package in the account-customization folder in your account.

  2. Customize the default manifest.json file that Terraform generates in the account-customization folder according to your requirements.

DevOps engineer
TaskDescriptionSkills required

Initialize the Terraform configuration.

To deploy the solution automatically with AFT, push the code to AWS CodeCommit:

$ git add *
$ git commit -m "message"
$ git push

You can also deploy this solution without using AFT by running a Terraform command from the account-customization folder. To initialize the working directory that contains the Terraform files, run:

$ terraform init
DevOps engineer

Preview changes.

To preview the changes that Terraform will make to the infrastructre, run the command:

$ terraform plan

This command evaluates the Terraform configuration to determine the desired state of the resources that have been declared. It also compares the desired state with the actual infrastructure to provision within the workspace.

DevOps engineer

Apply changes.

Run the following command to implement the changes that you made to the variables.tf files:

$ terraform apply
DevOps engineer
TaskDescriptionSkills required

Validate the creation of SSM documents.

  1. On the Systems Manager console, in the left navigation pane, choose Documents.

  2. Choose the Owned by me tab.

You should see the DistributeSoftwarePackage and AddSoftwarePackageToDistributor packages.

DevOps engineer

Validate the successful deployment of automations.

  1. On the Systems Manager console, in the left navigation pane, choose Automation.

  2. In the Automation executions list, you should see the most recent DistributeSoftwarePackage and AddSoftwarePackageToDistributor deployments.

  3. Choose Execution ID to validate that they completed successfully.

DevOps engineer

Validate that the package deployed to the targeted member account instances.

  1. On the Systems Manager console, in the navigation pane, choose Run Command.

  2. In Command history, you will see each invocation and its status.

  3. Choose any Command ID to see the deployment history for each target instance.

  4. Choose the Instance ID and check the Output section for the distribution.

DevOps engineer

Troubleshooting

IssueSolution

The State Manager association failed or is stuck in pending status.

See the troubleshooting information in the AWS Knowledge Center.

A scheduled association failed to run.

Your schedule specification might be invalid. State Manager doesn't currently support specifying months in cron expressions for associations. Use cron or rate expressions to confirm the schedule.

Related resources