Continue rolling back an update

Sometimes, when CloudFormation tries to roll back a stack update, it can't roll back all the changes it made during the update process. This is called the UPDATE_ROLLBACK_FAILED state. For example, you might have a stack that begins to roll back to an old database instance that was deleted outside of CloudFormation. Because CloudFormation doesn't know that the database was deleted, it assumes that the database instance still exists and attempts to roll back to it, causing the update rollback to fail.

A stack in the UPDATE_ROLLBACK_FAILED state can't be updated, but it can be rolled back to a working state (UPDATE_ROLLBACK_COMPLETE). After returning the stack to its original settings, you can try to update it again.

In most cases, you must fix the error that causes the update rollback to fail before you can continue to roll back your stack. In other cases, you can continue to roll back the update without any changes, for example when a stack operation times out.

If you can't fix the error, you can force the rollback to complete. CloudFormation skips each resource that fails to roll back and returns the stack to UPDATE_ROLLBACK_COMPLETE. For more information, see Force a rollback to complete.

Force a rollback to complete

Use force rollback when you can't fix the error that causes the update rollback to fail. Instead of looking up and listing each resource in --resources-to-skip, you let CloudFormation skip the resources that fail to roll back.

When you force a rollback, CloudFormation continues rolling back the stack. CloudFormation skips each resource that is in the UPDATE_FAILED state because a rollback failed, including resources in nested stacks. CloudFormation sets the status of each skipped resource to UPDATE_COMPLETE with the status reason Resource skipped during UpdateRollback. When the rollback completes, the stack status reason reports that one or more resources were skipped.

CloudFormation skips only resources whose rollback failed. Resources that failed during the forward update aren't skipped.

The following continue-update-rollback example forces the rollback of the MyStack stack to complete.

aws cloudformation continue-update-rollback --stack-name MyStack \
    --force-rollback

You can't use --force-rollback and --resources-to-skip in the same request. To skip only a specific set of resources, use --resources-to-skip without --force-rollback.

Drift detection reports skipped resources as NOT_CHECKED. To find out whether a skipped resource differs from the template, run DetectStackResourceDrift on the resource.

Continue rolling back from failed nested stack updates

When you have multiple stacks nested within each other, you may need to skip resources across multiple nested levels to get the full stack hierarchy back to a working state.

For example, you have a root stack called WebInfra that contains two smaller stacks inside it: WebInfra-Compute and WebInfra-Storage. These two stacks also have their own nested stacks within them.

If something goes wrong during an update, and the update process fails, the entire stack hierarchy may end up in the UPDATE_ROLLBACK_FAILED state, as shown in the following diagram.

To get the root stack into an operable state using the continue-update-rollback command, you must skip the resources that failed to roll back. You can either use the --force-rollback option to skip them across all nested levels, or use the --resources-to-skip option to list them.

The following continue-update-rollback example forces the rollback of the whole hierarchy to complete. You don't need to find the nested stack names or logical IDs.

aws cloudformation continue-update-rollback --stack-name WebInfra \
    --force-rollback

The following continue-update-rollback example resumes a rollback operation from a previously failed stack update. In this example, the --resources-to-skip option includes the following items:

For the resources of the root stack, you only need to provide the logical ID, for example, myCustom. However, for the resources that are contained in nested stacks, you must provide both the nested stack name and its logical ID, separated by a period. For example, WebInfra-Compute-Asg.myAsg.

aws cloudformation continue-update-rollback --stack-name WebInfra \
    --resources-to-skip myCustom WebInfra-Compute-Asg.myAsg WebInfra-Compute-LB.myLoadBalancer WebInfra-Storage.DB

To find the stack name of a nested stack

You can locate it within the child stack's stack ID or Amazon Resource Name (ARN).

The following ARN example refers to a stack named WebInfra-Storage-Z2VKC706XKXT.

arn:aws:cloudformation:us-east-1:123456789012:stack/WebInfra-Storage-Z2VKC706XKXT/ea9e7f90-54f7-11e6-a032-028f3d2330bd

To find the logical ID of a nested stack

You can find a child stack's logical ID in the template definition of its parent. In the diagram, the LogicalId of the WebInfra-Storage-DB child stack is DB in its parent WebInfra-Storage.

In the CloudFormation console, you can also find the logical ID in the Logical ID column for the stack resource on the Resources tab or the Events tab. For more information, see View stack information from the CloudFormation console.