Using the AWS OpsWorks Stacks Detach in Place tool - AWS OpsWorks
How the process worksLimitationsGetting started

Using the AWS OpsWorks Stacks Detach in Place tool

Important

The AWS OpsWorks Stacks service reached end of life on May 26, 2024 and has been disabled for both new and existing customers. We strongly recommend customers migrate their workloads to other solutions as soon as possible.

This section describes how to use the AWS OpsWorks Stacks Detach in Place tool to detach your OpsWorks instances from the OpsWorks Stacks service.

The instances you detach will remain in your AWS account, but you'll no longer be able to manage them using OpsWorks. Instead, you'll use Amazon EC2, AWS Systems Manager, or any EC2 compatible approach to configure and manage the instances.

At a high level, the detachment process involves the following steps:

  1. The tool performs validation checks to ensure the resources are ready for detachment.

  2. The tool exports the Custom JSON from your OpsWorks stack and stores it as an object in Amazon S3.

  3. The tool creates Systems Manager Automation documents representing each OpsWorks Stacks lifecycle event.

  4. The tool creates an AWS Service Catalog AppRegistry Catalog for all instances that are being detached and detaches any Elastic Load Balancing (ELB) load balancers from the OpsWorks layers.

  5. Finally, the tool detaches and deregisters other resources including Amazon Relational Database Service (Amazon RDS) instances.

How the process works

The Detach In Place tool provides the following 3 commands and a wizard like experience that guides you through a series of steps to check and configure your instances before proceeding to detach your layer.

Command Description

handle-prerequisites

This command analyses whether all instances in a layer are eligible for detachment and resolves prerequisites. The instances must be in a healthy state in OpsWorks, they cannot have time or load based auto scalers, and must have the latest OpsWorks Agent version installed.

Additionally, the command checks if all instances have the permissions required to support the SSM Agent and if the latest SSM Agent version is installed. The command will install the SSM Agent if it is not present and will update the SSM Agent if it's not using the latest version. The command will also add any necessary permissions.

detach

This command detaches all OpsWorks instances for the specified layer.

First, the command will run a prerequisites check to ensure the layer qualifies for detachment. If you do not want to resolve the prerequisites, you're given the option to force detach.

Next, the command will indicate that all tags added to your instances through OpsWorks tagging APIs or through propagation of tags from your layers and stacks, will be retained. You can remove any of these tags using relevant EC2 APIs after the detachment is complete.

Then, the command will check if you want to export the Chef related configuration to SSM parameters.

If you have a Classic Load Balancer attached to the layer, the command will ask if it can detach the load balancer in order to prevent any downtime.

cleanup

This command deletes all entities in OpsWorks from your account. It will terminate the instances and delete all stacks. This should be used for resources that are no longer needed as a last step to clean up the account.

Note

We recommend you run the new setup for a few days prior to running the cleanup command. This ensures that any necessary configurations from the stack are readily available if needed.

Limitations

The main purpose of the Detach In Place tool is to safely detach the OpsWorks Stacks instances. This section summarises the limitations of the tool.

  • Windows SSM Agent – If the SSM Agent is not installed on the instance, you'll need to manually install it. The same applies if the Agent is not updated to the latest version.

  • Time/Load Auto Scaling instances – The detachment tool doesn't support instances with Auto Scaling enabled. You must disable Auto Scaling on instances you want to detach.

  • Permissions – The detachment tool doesn’t create or generate IAM entities specified on the Permissions page of the OpsWorks console.

  • Apps – The detachment tool doesn’t create or generate apps outside of OpsWorks.

Getting started

Step 1: Verify the prerequisites are met

All 3 commands of the Detach In Place tool are Python scripts, which you can run locally, on EC2 instance, or by using AWS CloudShell.

AWS CloudShell is a browser-based shell that gives you command-line access to the AWS resources in the selected AWS Region. AWS CloudShell comes pre-installed with popular tools (such as AWS CLI and Python). When using AWS CloudShell, you use the same credentials that you use to sign in to the console.

This walkthrough assumes that you're using AWS CloudShell.

Step 2: Download the script

  1. Download the zip file that contains the migration script and all the relevant files by running the following command:

    aws s3api get-object \
--bucket detach-in-place-bucket-prod-us-east-1 \
--key detach_in_place_script.zip detach_in_place_script.zip

  2. Unzip the file by running the following command.

    unzip detach_in_place_script.zip

    After the file is unzipped, the following files are available:

    • README.md

    • LICENSE

    • NOTICE

    • requirements.txt

    • TODO.py

  3. If necessary, install pipenv by running the following command.

    pip install pipenv

Step 3: Run the script

First, set up your environment so that you can run the script by running the following commands.

pipenv install -r requirements.txt
pipenv shell

Then, review the script parameters.

Command Parameter Description Type Required Default

handle-prerequisites

--layer-id

The ID of the layer you want to detach.

String

Yes

-

--region

The Region of the OpsWorks stack. If your OpsWorks stack Region and API endpoint Region are different, use the stack Region. This is the same Region as the other resources part of your OpsWorks stack (for example, EC2 instances and subnets).

String

No

us-east-1

detach

--layer-id

ID of the layer you want to detach.

String

Yes

-

--batch-size

Number of instances to detach from a layer (for example, 5).

String

No

-

--region

The Region of the OpsWorks stack. If your OpsWorks stack Region and API endpoint Region are different, use the stack Region. This is the same Region as the other resources part of your OpsWorks stack (for example, EC2 instances and subnets).

String

No

us-east-1

cleanup

--stack-id

ID of the stack you want to delete.

String

No

Mutually exclusive, you must specify either a layer ID or a stack ID

--layer-id

ID of the layer you want to delete

String

No

--region

The Region of the OpsWorks stack. If your OpsWorks stack Region and API endpoint Region are different, use the stack Region. This is the same Region as the other resources part of your OpsWorks stack (for example, EC2 instances and subnets).

String

No

us-east-1

You can see the available options for the detach, handle-prerequisites and cleanup commands by running the commands with the --help option as follows:

python3 layer_detacher.py detach --help
python3 layer_detacher.py handle-prerequisites --help
python3 layer_detacher.py cleanup --help

You're now ready to get started. The following examples show how you can run the commands for different use cases.

Example 1: Check if a layer fulfills all prerequisites and is eligible for detachment

The following command reads information about an OpsWorks layer (and the instances it includes) and checks if the following prerequisites are met:

  • All instances are online.

  • There are no Load/Time Auto Scaling instances.

  • All instances have the latest OpsWorks Agent.

  • All instances have the latest SSM Agent installed and configured.

  • All instances have an SSH key pair.

  • Every instance belongs to exactly one layer.

python3 layer_detacher.py handle-prerequisites \
--layer-id opsworks-layer-id \
--region opsworks-stack-region

Example 2: Detach all instances of a layer

The following command will iterate over all instances of the layer, check if the instances meet the prerequisites and try to detach in parallel all instances that meet the prerequisites. If one or more prerequisites are not met, the command will provide a force detach option for the remaining non-compliant instances.

Before detaching any instance the command will:

  1. Save the Custom JSON and upload it to S3.

  2. Create SSM Automation documents for every OpsWorks lifecycle event for the layer and upload the execution logs for the Automation documents to S3.

  3. Create an AppRegistry application for all instances that will be detached. The application has a Resource Group associated with it which holds all detached instances and resources. The resources include SSM Automation documents and SSM parameters that hold information about lifecycle events and custom Chef recipes.

  4. Detaches the Classic Load Balancer from the layer, if one exists.

This command will modify only OpsWorks resources. The status of EC2 instances will remain the same.

python3 layer_detacher.py detach \
--layer-id opsworks-layer-id \
--region opsworks-stack-region

Example 3: Detach all instances of a layer in batches

The following command does the same as the previous example. The only difference is that it detaches the instances in batches.

This command will modify only OpsWorks resources. The status of EC2 instances will remain the same.

python3 layer_detacher.py detach \
--layer-id opsworks-layer-id \
--region opsworks-stack-region \
--batch-size 5

Example 4: Clean up all resources for a layer and delete the layer

The following command will iterate over all resources for a layer and delete them. In more detail, it will stop and delete all instances in OpsWorks and EC2, detach the load balancer and deregister Amazon RDS instances, elastic IPs and volumes. After cleaning up the resources, it will delete the layer.

This command will delete OpsWorks resources and EC2 instances. If you want your EC2 instances to remain untouched, use the detach command before using the cleanup command. That way the cleanup command will delete all the remaining resources.

python3 layer_detacher.py cleanup \
--layer-id opsworks-layer-id \
--region opsworks-stack-region

Example 5: Clean up all resources for a stack and delete the stack

The following command will iterate over all layers and then iterate over the resources of each layer. For each layer, the command will stop and delete all instances in OpsWorks and EC2, detach load balancers, and deregister Amazon RDS instances, elastic IPs and volumes. Then, the command will delete the layer. The same process will be performed in every layer that belongs to this stack. Finally, after all layers are deleted, the stack will be removed.

This command will delete OpsWorks resources and EC2 instances. If you want your EC2 instances to remain untouched, use the detach command before using the cleanup command. That way the cleanup command will delete all the remaining resources.

python3 layer_detacher.py cleanup \
--stack-id opsworks-stack-id \
--region opsworks-stack-region

Step 4: Continue to operate your resources after detaching from OpsWorks

After running the detach command, the tool creates a new AWS Service Catalog AppRegistry application corresponding to the detached layer. The application name follows the format layer-name---layer-id. It also adds the OpsWorksLayerId tag to uniquely identify the application matching the detached layer.

To add new AWS resources to this application (for example, new EC2 instances), you can do one of the following:

  1. Tag the resource with the unique application tag of the AppRegistry application:

    Tag key: awsApplication

    Value: arn:aws:resource-groups:region:account-id:group/application-name/application-id>

  2. Run the associate-resource command.

Additionally, for each AppRegistry application, a Resource Group is created. The Resource Group contains the following tags.

Tag key Value

EnableAWSServiceCatalogAppRegistry

TRUE

aws:servicecatalog:applicationName

application-name

aws:servicecatalog:applicationId

application-id

aws:servicecatalog:applicationArn

arn:aws:servicecatalog:region:account-id:/applications/application-id

Performing tasks after detachment

The following table provides information about how to perform tasks after detachment:

Task Description

Executing lifecycle events

After running the detach command and if you selected the option, the script creates 5 Automation documents matching the 5 OpsWorks lifecycle events.

The name of each Automation document follows this format: layer-id_lifecycle-event_automation_document.

To simulate OpsWorks behavior in Systems Manager, you'll need to manually trigger Automation executions when provisioning, terminating EC2 instances, or deploying/removing recipes.

Updating Custom JSON

Custom JSON for the stack and layer are stored in an S3 bucket specified during detachment, or alternatively in a new S3 bucket that gets created.

The file names stored for the JSON files are the following:

  • layercustomjson.json

  • stackcustomjson.json

Changing your run list for lifecycle events

The run list for each lifecycle event is defined in the corresponding Automation document. To change the run list, find the Automation documents in the AppRegistry application and modify the RunList parameter.

The process to update recipes and cookbooks is unchanged because AWS-ApplyChefRecipes, which the Automation documents trigger, supports the same source as OpsWorks.

Managing auto healing / auto scaling

When you detach an instance, the OpsWorks Agent uninstalls. Without the agent, OpsWorks cannot automatically heal or replace unhealthy instances, nor can it auto scale your fleet. To continue auto scaling and replacing failed instances, create an Amazon EC2 Auto Scaling group. The group will launch new instances to maintain its desired capacity when Amazon EC2 detects unhealthy instances that need replacing.

Managing Load Balancer

If your layer uses a Classic Load Balancer, the detach command will detach it prior to deregistering the instances. This is done to ensure that all ELB instance associations remain preserved on Amazon EC2 throughout the course of detachment, thus leading to zero downtime. After the process is completed you will be able to manage your ELB on EC2.

Connecting to your instances

When you run the handle-prerequisites or detach command, two checks occur:

  • The version of the SSM Agent and permissions

  • SSH keys

The commands also offer you the option to update the SSM Agent and add required permissions so you can connect to instances using Session Manager. If SSH keys exist, you also have the option to SSH into the instance.

Using the Systems Manager Application Manager Instances tab

After detachment, you'll be able to view and manage your instances on the Application Manager Instances tab.

The Instances tab provides aggregate information about an application’s EC2 instances, such as their status, health state, and last command status. Using this tab, you can view detailed information about individual instances such as command history, alarm states, Systems Manager agent health, and more. The Instances tab also provides a variety of actions, such as the ability to apply Chef recipes, start or stop an instance, or add or remove an instance from an Auto Scaling group.