Remove Amazon EC2 entries across AWS accounts from AWS Managed Microsoft AD by using AWS Lambda automation - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsBest practicesEpicsTroubleshootingRelated resources

Remove Amazon EC2 entries across AWS accounts from AWS Managed Microsoft AD by using AWS Lambda automation

Created by Dr. Rahul Sharad Gaikwad (AWS) and Tamilselvan P (AWS)

Environment: PoC or pilot

Technologies: DevOps; Infrastructure; Management & governance

Workload: All other workloads

AWS services: AWS Directory Service; Amazon EC2; Amazon EventBridge; AWS Lambda; AWS Systems Manager; AWS Auto Scaling

Summary

Active Directory (AD) is a Microsoft scripting tool that manages domain information and user interactions with network services. It’s widely used among managed services providers (MSPs) to manage employee credentials and access permissions. Because AD attackers can use inactive accounts to try and hack into an organization, it’s important to find inactive accounts and disable them on a routine maintenance schedule. With AWS Directory Service for Microsoft Active Directory, you can run Microsoft Active Directory as a managed service. This pattern can help you to configure AWS Lambda automation to quickly find and remove inactive accounts.

If the following scenarios apply to your organization, this pattern can assist you:

  • Centralized AD management – If your organization has multiple AWS accounts, each with its own AD deployment, it can be challenging to manage user accounts and access permissions consistently across all accounts. With an across-accounts AD cleanup solution, you can disable or remove inactive accounts from all AD instances in a centralized manner.

  • AD restructuring or migration – If your organization plans to restructure or migrate its AD deployment, an across-accounts AD cleanup solution can help you prepare the environment. The solution can help you remove unnecessary or inactive accounts, simplify the migration process, and reduce potential conflicts or issues.

When you use this pattern, you can get the following benefits:

  • Improve database and server performance, and fix vulnerabilities in your security from inactive accounts.

  • If your AD server is hosted in the cloud, removing inactive accounts can also reduce storage costs while improving performance. Your monthly bills might decrease because bandwidth charges and compute resources can both drop.

  • Keep potential attackers at bay with a clean Active Directory.

Prerequisites and limitations

Prerequisites

  • An active parent AWS account and one or multiple child accounts. In this pattern, a parent account is where Active Directory is created. Child accounts host Windows servers and are joined through the parent account Active Directory.

  • Git installed and configured on a local workstation.

  • Terraform installed and configured on a local workstation.

  • AWS Managed Microsoft AD directory configured in the parent account and shared to all child accounts. For more details, see Tutorial: Sharing your AWS Managed Microsoft AD directory for seamless EC2 domain-join in the AWS Directory Service Administration Guide.

  • A virtual private cloud (VPC) peering connection or AWS Transit Gateway connection available between the VPC of AWS Directory Service (parent account) and the VPC of the Amazon Elastic Compute Cloud (Amazon EC2) instances (child accounts). For more details, see Configure a VPC peering connection between the directory owner and the directory consumer account in the AWS Directory Service Administration Guide.

  • A Windows machine configured with the EC2WindowsUserdata script on all the parent and child accounts. The script file is available in the root of this pattern’s code repository.

  • A cross-account AWS Identity and Access Management (IAM) role available on each child account that’s configured with a trust policy to allow the use of an AWS Lambda function from the parent account. For more information, see Sending and receiving events between AWS accounts in Amazon EventBridge in the Amazon EventBridge User Guide.

  • The following secrets values available in AWS Systems Manager Parameter Store of the parent account:

    • domainJoinUser – Username of the directory service

    • domainJoinPassword – Password of the directory service

    For more information about secrets, see Create an AWS Secrets Manager secret in the AWS Secrets Manager User Guide.

Limitations

  • Creating a resource in a child account isn’t automated with Terraform. You must create the following resources manually by using the AWS Management Console:

    • Amazon EventBridge rule to send the Amazon EC2 termination events to the parent account

    • Amazon EC2 cross-account role creation in the child account with trust policy

    • VPC peering or Transit Gateway connection

  • Some AWS services aren’t available in all AWS Regions. For Region availability, see AWS services by Region. For specific endpoints, see Service endpoints and quotas, and choose the link for the service.

Product versions

Architecture

The following diagram displays the high-level architecture of the solution.

Process to use Lambda automation to remove EC2 entries from across AWS accounts.

The architecture diagram illustrates the following process:

  1. In child accounts, the EventBridge rule collects all the Amazon EC2 termination events. The rule sends those events to EventBridge which is present in the parent account.

  2. From the parent account, EventBridge collects all the events and contains the rule to trigger the Lambda function ADcleanup-Lambda.

  3. The parent account receives any termination events from the parent or child account and triggers the Lambda function.

  4. The Lambda function makes the call to Amazon EC2 Auto Scaling groups using the Python boto module and gets the random instance ID. The instance ID is used to execute Systems Manager commands.

  5. The Lambda function makes another call to Amazon EC2 using the boto module. The Lambda function gets the private IP addresses of the running Windows servers and stores the addresses in a temporary variable. In steps 5.1 and 5.2, the running Windows EC2 instances are collected from child accounts.

  6. The Lambda function makes another call to Systems Manager to get the computer information that is connected to AWS Directory Service.

  7. An AWS Systems Manager document helps to execute the PowerShell command on Amazon EC2 Windows servers to get the private IP addresses of the computers which are connected to AD. (The Systems Manager document uses the instance ID that was obtained in step 4.)

  8. The AD domain username and passwords are stored in the AWS Systems Manager Parameter Store. AWS Lambda and Systems Manager make a call to Parameter Store and get the username and password values to use to connect to AD.

  9. Using the Systems Manager document, the PowerShell script is executed on the Amazon EC2 Windows server using the instance id obtained earlier in step 4.

  10. Amazon EC2 connects to AWS Directory Service by using PowerShell commands and remove the computers that are not in use or inactive.

Tools

AWS services

  • AWS Directory Service provides multiple ways to use Microsoft Active Directory (AD) with other AWS services such as Amazon Elastic Compute Cloud (Amazon EC2), Amazon Relational Database Service (Amazon RDS) for SQL Server, and Amazon FSx for Windows File Server.

  • AWS Directory Service for Microsoft Active Directory enables your directory-aware workloads and AWS resources to use Microsoft Active Directory in the AWS Cloud.

  • Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the AWS Cloud. You can launch as many virtual servers as you need and quickly scale them up or down.

  • Amazon EventBridge is a serverless event bus service that that helps you connect your applications with real-time data from a variety of sources. For example, AWS Lambda functions, HTTP invocation endpoints using API destinations, or event buses in other AWS accounts.

  • AWS Identity and Access Management (IAM) helps you securely manage access to your AWS resources by controlling who is authenticated and authorized to use them. With IAM, you can specify who or what can access services and resources in AWS, centrally manage fine-grained permissions, and analyze access to refine permissions across AWS.

  • AWS Lambda is a compute service that helps you run code without needing to provision or manage servers. It runs your code only when needed and scales automatically, so you pay only for the compute time that you use.

  • AWS Systems Manager helps you manage your applications and infrastructure running in the AWS Cloud. It simplifies application and resource management, shortens the time to detect and resolve operational problems, and helps you manage your AWS resources securely at scale.

  • AWS Systems Manager documents define the actions that Systems Manager performs on your managed instances. Systems Manager includes more than 100 pre-configured documents that you can use by specifying parameters at runtime.

  • AWS Systems Manager Parameter Store is a capability of AWS Systems Manager and provides secure, hierarchical storage for configuration data management and secrets management.

Other tools

  • HashiCorp Terraform is an open source infrastructure as code (IaC) tool that helps you use code to provision and manage cloud infrastructure and resources.

  • PowerShell is a Microsoft automation and configuration management program that runs on Windows, Linux, and macOS.

  • Python is a general-purpose computer programming language.

Code repository

The code for this pattern is available in the GitHub aws-lambda-ad-cleanup-terraform-samples repository.

Best practices

  • Automatically join domains. When you launch a Windows instance that’s to be part of an AWS Directory Service domain, join the domain during the instance creation process instead of manually adding the instance later. To automatically join a domain, select the correct directory from the Domain join directory dropdown list when launching a new instance. For more details, see Seamlessly join an Amazon EC2 Windows instance to your AWS Managed Microsoft AD Active Directory in the AWS Directory Service Administration Guide.

  • Delete unused accounts. It’s common to find accounts in AD that have never been used. Like disabled or inactive accounts that remain in the system, neglected unused accounts can slow down your AD system or make your organization vulnerable to data breaches.

  • Automate Active Directory cleanups. To help mitigate security risks and prevent obsolete accounts from impacting AD performance, conduct AD cleanups should at regular intervals. You can accomplish most AD management and cleanup tasks by writing scripts. Example tasks include removing disabled and inactive accounts, deleting empty and inactive groups, and locating expired user accounts and passwords.

Epics

TaskDescriptionSkills required

Create a cross-account role in the child account.

To create a cross-account role in a child account, do the following:

  1. For each child account, create a role named ec2crossaccountrole by using the managed policy named AmazonEC2ReadOnlyAccess. (For more information, see Create a role using custom trust policies in the IAM documentation.)

  2. In the Custom trust policy section, add the following code:

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::${Parentaccountid}:role/ADcleanuprole"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
DevOps engineer

Create an event rule in the child account.

To create an EventBridge rule for each child account, do the following:

  1. Sign in to the child AWS account, and then open the Amazon EventBridge console at https://console.aws.amazon.com/events/.

  2. In the navigation pane, choose Rules.

  3. Choose Create rule.

  4. Enter a Name and, optionally, a description for the rule.

  5. For Event bus, select AWS default event bus.

  6. For Rule type, choose Rule with an event pattern.

  7. Choose Next.

  8. For Event pattern, paste the following code:

    {
  "source": ["aws.ec2"],
  "detail-type": ["EC2 Instance State-change Notification"],
  "detail": {
    "state": ["terminated"]
  }
}

  9. Choose Next.

  10. For Target type, choose Event bus in a different account or Region. For Event bus as target, enter the event bus Amazon Resource Name (ARN) for the parent account.

  11. For Execution role, choose Create a new role for this specific resource.

  12. Choose Next to review the details for the new rule, and then choose Create.

For more details, see Creating rules that react to events in Amazon EventBridge in the Amazon EventBridge User Guide.

DevOps engineer

Create an EC2 instance and join to AD.

To create an EC2 instance for Windows, do the following:

  1. Use the EC2WindowsUserdata script that’s available in this pattern's code repository.

  2. In the user data script, revise the following code to use your Directory service addresses values from the parent account:

    set-DnsClientServerAddress -InterfaceIndex 6 -ServerAddresses $(Directory service addresses)

DevOps engineer
TaskDescriptionSkills required

Create a project folder and add the files.

To clone the repository and create a project folder, do the following:

  1. Open this pattern’s GitHub repository.

  2. Choose the Code button to see the options to clone in the Clone dropdown.

  3. On the HTTPS tab, copy the URL provided in Clone using the web URL.

  4. Create a folder on your machine, and name it with your project name.

  5. Open a terminal in your local machine, and navigate to this folder.

  6. To clone the git repository, use the following command.

    git clone <repository-URL>.git

  7. After the repository has been cloned, use the following command to go to the cloned directory.

    cd <directory name>/terraform-aws-lambda-ad-cleanup/multiple-account-cleanup

  8. In the cloned repository, open this project in an integrated development environment (IDE) of your choice.

DevOps engineer

Build the adcleanup.zip file.

To compress the lambda_function.py file, run the following command:

zip -r adcleanup.zip lambda_function.py

DevOps engineer
TaskDescriptionSkills required

Provide values for the Terraform variables.

For the child account, provide values for the following arn variables as string types in the terraform.tfvars file:

  • lambda_env_cross_role_arn

  • child_account_cross_role_arn

DevOps engineer

Initialize the Terraform configuration.

To initialize your working directory that contains the Terraform files, run the following command:

terraform init

DevOps engineer

Preview changes.

You can preview the changes that Terraform will make to the infrastructure before your infrastructure is deployed. To validate that Terraform will make the changes as required, run the following command:

terraform plan —-var-file=examples/terraform.tfvars

DevOps engineer

Execute the proposed actions.

To verify that the results from the terraform plan command are as expected, do the following:

  1. Run the following command: terraform apply

  2. Sign in to the AWS Management Console, and verify that the resources are present.

DevOps engineer
TaskDescriptionSkills required

Execute and test the Lambda function.

To verify that the deployment occurred successfully, do the following:

  1. Sign in to the AWS Management Console, and open the Lambda console. Open the Functions page, and select the function name that begins with ADcleanup-Lambda-*.

  2. On the function overview page, choose Test on the Code tab in the Code source section.

  3. To save the test event, provide a name for the event and choose Save. To test the event, choose Test again.

The execution results show the output of the function.

DevOps engineer

View results of EventBridge rule execution from parent account.

To view the results of the EventBridge rule that’s based on Amazon EC2 termination events from the parent account, do the following:

  1. Terminate EC2 instances from the parent account.

  2. Open the Lambda console of the parent account. Open the Functions page, and select the function name that begins with ADcleanup-Lambda-*.

  3. Choose the Monitor tab, and choose View CloudWatch logs.

In the CloudWatch console, the Log groups page shows the results of the Lambda function.

DevOps engineer

View results of EventBridge rule execution from the child account.

To view the results of the EventBridge rule that’s based on Amazon EC2 termination events from the child account, do the following:

  1. Terminate EC2 instances from the child account.

  2. Open the Lambda console of the parent account. Open the Functions page, and select the function name that begins with ADcleanup-Lambda-*.

  3. Choose the Monitor tab, and choose View CloudWatch logs.

In the CloudWatch console, the Log groups page shows the results of the Lambda function.

DevOps engineer
TaskDescriptionSkills required

Clean up the infrastructure.

To clean up the infrastructure that you created, use the following command:

terraform destroy

To confirm the destroy command, type yes.

DevOps engineer

Verify after cleanup.

Verify that the resources are successfully removed.

DevOps engineer

Troubleshooting

IssueSolution

Connection issue between AWS Directory Service (parent account) and Amazon EC2 instance (child account) – You are unable to join the child account’s computers to AD even though VPC peering is available.

Add routing in the VPCs. For instructions, see Configure a VPC peering connection between the directory owner and the directory consumer account in the AWS Directory Service documentation.

Related resources

AWS documentation

Other resources