Continue rolling back an update - AWS CloudFormation
Continue rolling back from failed nested stack updates

Continue rolling back an update

Sometimes, when CloudFormation tries to roll back a stack update, it can't roll back all the changes it made during the update process. This is called the UPDATE_ROLLBACK_FAILED state. For example, you might have a stack that begins to roll back to an old database instance that was deleted outside of CloudFormation. Because CloudFormation doesn't know that the database was deleted, it assumes that the database instance still exists and attempts to roll back to it, causing the update rollback to fail.

A stack in the UPDATE_ROLLBACK_FAILED state can't be updated, but it can be rolled back to a working state (UPDATE_ROLLBACK_COMPLETE). After returning the stack to its original settings, you can try to update it again.

In most cases, you must fix the error that causes the update rollback to fail before you can continue to roll back your stack. In other cases, you can continue to roll back the update without any changes, for example when a stack operation times out.

Note

If you use nested stacks, rolling back the parent stack will attempt to roll back all the child stacks as well.

To continue rolling back an update (console)

  1. Sign in to the AWS Management Console and open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  2. On the navigation bar at the top of the screen, choose the AWS Region where the stack is located.

  3. On the Stacks page, choose the stack that you want to update, choose Stack actions, and then choose Continue update rollback.

    If none of the solutions in the Troubleshooting errors worked, you can use the advanced option to skip the resources that CloudFormation can't successfully roll back. You must look up and type the logical IDs of the resources that you want to skip. Specify only resources that went into the UPDATE_FAILED state during the UpdateRollback and not during the forward update.

    Warning

    CloudFormation sets the status of the specified resources to UPDATE_COMPLETE and continues to roll back the stack. After the rollback is complete, the state of the skipped resources will be inconsistent with the state of the resources in the stack template. Before performing another stack update, you must update the stack or resources to be consistent with each other. If you don't, subsequent stack updates might fail, and the stack will become unrecoverable.

    Specify the minimum number of resources required to successfully roll back your stack. For example, a failed resource update might cause dependent resources to fail. In this case, it might not be necessary to skip the dependent resources.

    To skip resources that are part of nested stacks, use the following format: NestedStackName.ResourceLogicalID. If you want to specify the logical ID of a stack resource (Type: AWS::CloudFormation::Stack) in the ResourcesToSkip list, then its corresponding embedded stack must be in one of the following states: DELETE_IN_PROGRESS, DELETE_COMPLETE, or DELETE_FAILED.

To continue rolling back an update (AWS CLI)

  • Use the continue-update-rollback command with the --stack-name option to specify the ID of the stack that you want to continue to roll back.

Continue rolling back from failed nested stack updates

When you have multiple stacks nested within each other, you may need to skip resources across multiple nested levels to get the full stack hierarchy back to a working state.

For example, you have a root stack called WebInfra that contains two smaller stacks inside it: WebInfra-Compute and WebInfra-Storage. These two stacks also have their own nested stacks within them.

If something goes wrong during an update, and the update process fails, the entire stack hierarchy may end up in the UPDATE_ROLLBACK_FAILED state, as shown in the following diagram.

A diagram showing a three-level nested stack hierarchy.
Note

The stack names in this example are truncated for simplicity. Child stack names are typically generated by CloudFormation and contain unique random strings, so actual names might not be user-friendly.

To get the root stack into an operable state using the continue-update-rollback command, you must use the --resources-to-skip option to skip resources that failed to rollback.

The following continue-update-rollback example resumes a rollback operation from a previously failed stack update. In this example, the --resources-to-skip option includes the following items:

  • myCustom

  • WebInfra-Compute-Asg.myAsg

  • WebInfra-Compute-LB.myLoadBalancer

  • WebInfra-Storage.DB

For the resources of the root stack, you only need to provide the logical ID, for example, myCustom. However, for the resources that are contained in nested stacks, you must provide both the nested stack name and its logical ID, separated by a period. For example, WebInfra-Compute-Asg.myAsg.

aws cloudformation continue-update-rollback --stack-name WebInfra \
    --resources-to-skip myCustom WebInfra-Compute-Asg.myAsg WebInfra-Compute-LB.myLoadBalancer WebInfra-Storage.DB
To find the stack name of a nested stack

You can locate it within the child stack's stack ID or Amazon Resource Name (ARN).

The following ARN example refers to a stack named WebInfra-Storage-Z2VKC706XKXT.

arn:aws:cloudformation:us-east-1:123456789012:stack/WebInfra-Storage-Z2VKC706XKXT/ea9e7f90-54f7-11e6-a032-028f3d2330bd
To find the logical ID of a nested stack

You can find a child stack's logical ID in the template definition of its parent. In the diagram, the LogicalId of the WebInfra-Storage-DB child stack is DB in its parent WebInfra-Storage.

In the CloudFormation console, you can also find the logical ID in the Logical ID column for the stack resource on the Resources tab or the Events tab. For more information, see View stack information from the CloudFormation console.