# Troubleshooting CodeDeploy
Use the topics in this section to help solve problems and errors you might encounter when using CodeDeploy.
**Note**
You can identify the causes of many deployment failures by reviewing the log files created during the deployment process. For simplicity, we recommend using Amazon CloudWatch Logs to centrally monitor log files instead of viewing them instance by instance. For information, see [Monitoring deployments with Amazon CloudWatch tools](monitoring-cloudwatch.md).
**Topics**
+ [General troubleshooting issues](troubleshooting-general.md)
+ [Troubleshoot EC2/On-Premises deployment issues](troubleshooting-deployments.md)
+ [Troubleshoot Amazon ECS deployment issues](troubleshooting-ecs.md)
+ [Troubleshoot AWS Lambda deployment issues](troubleshooting-deployments-lambda.md)
+ [Troubleshoot deployment group issues](troubleshooting-deployment-groups.md)
+ [Troubleshoot instance issues](troubleshooting-ec2-instances.md)
+ [Troubleshoot GitHub token issues](troubleshooting-github-token-issues.md)
+ [Troubleshoot Amazon EC2 Auto Scaling issues](troubleshooting-auto-scaling.md)
+ [Error codes for AWS CodeDeploy](error-codes.md)
# General troubleshooting issues
**Topics**
+ [General troubleshooting checklist](#troubleshooting-checklist)
+ [CodeDeploy deployment resources are supported in only in some AWS Regions](#troubleshooting-supported-regions)
+ [Procedures in this guide do not match the CodeDeploy console](#troubleshooting-old-console)
+ [Required IAM roles are not available](#troubleshooting-iam-cloudformation)
+ [Using some text editors to create AppSpec files and shell scripts can cause deployments to fail](#troubleshooting-text-editors)
+ [Using Finder in macOS to bundle an application revision can cause deployments to fail](#troubleshooting-bundle-with-finder)
## General troubleshooting checklist
You can use the following checklist to troubleshoot a failed deployment.
1. See [View CodeDeploy deployment details](deployments-view-details.md) and [View instance details with CodeDeploy](instances-view-details.md) to determine why the deployment failed. If you cannot determine the cause, review the items in this checklist.
1. Check that you have correctly configured the instances:
+ Was the instance launched with an EC2 key pair specified? For more information, see [EC2 Key Pairs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2-key-pairs.html) in *Amazon EC2 User Guide*.
+ Is the correct IAM instance profile attached to the instance? For more information, see [Configure an Amazon EC2 instance to work with CodeDeploy](instances-ec2-configure.md) and [Step 4: Create an IAM instance profile for your Amazon EC2 instances](getting-started-create-iam-instance-profile.md).
+ Was the instance tagged? For more information, see [Working with tags in the console](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#Using_Tags_Console) in *Amazon EC2 User Guide*.
+ Is the CodeDeploy agent installed, updated, and running on the instance? For more information, see [Managing CodeDeploy agent operations](codedeploy-agent-operations.md). To check which version of the agent is installed, see [Determine the version of the CodeDeploy agent](codedeploy-agent-operations-version.md).
1. Check the application and deployment group settings:
+ To check your application settings, see [View application details with CodeDeploy](applications-view-details.md).
+ To check your deployment group settings, see [View deployment group details with CodeDeploy](deployment-groups-view-details.md).
1. Confirm the application revision is correctly configured:
+ Check the format of your AppSpec file. For more information, see [Add an application specification file to a revision for CodeDeploy](application-revisions-appspec-file.md) and [CodeDeploy AppSpec file reference](reference-appspec-file.md).
+ Check your Amazon S3 bucket or GitHub repository to verify your application revision is in the expected location.
+ Review the details of your CodeDeploy application revision to ensure that it is registered correctly. For information, see [View application revision details with CodeDeploy](application-revisions-view-details.md).
+ If you're deploying from Amazon S3, check your Amazon S3 bucket to verify CodeDeploy has been granted permissions to download the application revision. For information about bucket policies, see [Deployment prerequisites](deployments-create-prerequisites.md).
+ If you're deploying from GitHub, check your GitHub repository to verify CodeDeploy has been granted permissions to download the application revision. For more information, see [Create a deployment with CodeDeploy](deployments-create.md) and [GitHub authentication with applications in CodeDeploy](integrations-partners-github.md#behaviors-authentication).
1. Check that the service role is correctly configured. For information, see [Step 2: Create a service role for CodeDeploy](getting-started-create-service-role.md).
1. Confirm you followed the steps in [Getting started with CodeDeploy](getting-started-codedeploy.md) to:
+ Provisioned a user with the appropriate permissions.
+ Install or upgrade and configure the AWS CLI.
+ Create an IAM instance profile and a service role.
For more information, see [Identity and access management for AWS CodeDeploy](security-iam.md).
1. Confirm you are using AWS CLI version 1.6.1 or later. To check the version you have installed, call **aws --version**.
If you are still unable to troubleshoot your failed deployment, review the other issues in this topic.
## CodeDeploy deployment resources are supported in only in some AWS Regions
If you do not see or cannot access applications, deployment groups, instances, or other deployment resources from the AWS CLI or the CodeDeploy console, make sure you're referencing one of the AWS Regions listed in [Region and endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html#codedeploy_region) in *AWS General Reference*.
EC2 instances and Amazon EC2 Auto Scaling groups that are used in CodeDeploy deployments must be launched and created in one of these AWS Regions.
If you're using the AWS CLI, run the **aws configure** command from the AWS CLI. Then you can view and set your default AWS Region.
If you're using the CodeDeploy console, on the navigation bar, from the region selector, choose one of the supported AWS Regions.
**Important**
To use services in the China (Beijing) Region or China (Ningxia) Region, you must have an account and credentials for those regions. Accounts and credentials for other AWS regions do not work for the Beijing and Ningxia Regions, and vice versa.
Information about some resources for the China Regions, such as CodeDeploy Resource Kit bucket names and CodeDeploy agent installation procedures, are not included in this edition of the *CodeDeploy User Guide*.
For more information:
[CodeDeploy](http://docs.amazonaws.cn/en_us/aws/latest/userguide/codedeploy.html) in *[Getting Started with AWS in the China (Beijing) Region](http://docs.amazonaws.cn/en_us/aws/latest/userguide/introduction.html) *
*CodeDeploy User Guide for the China Regions* ([English version](http://docs.amazonaws.cn/en_us/codedeploy/latest/userguide/welcome.html) \$1 [Chinese version](http://docs.amazonaws.cn/codedeploy/latest/userguide/welcome.html))
## Procedures in this guide do not match the CodeDeploy console
The procedures in this guide are written to reflect the new console design. If you are using the older version of the console, many of the concepts and basic procedures in this guide still apply. To access help in the new console, choose the information icon.
## Required IAM roles are not available
If you rely on an IAM instance profile or a service role that was created as part of an AWS CloudFormation stack, if you delete the stack, all IAM roles are deleted, too. This may be why the IAM role is no longer displayed in the IAM console and CodeDeploy no longer works as expected. To fix this problem, you must manually re-create the deleted IAM role.
## Using some text editors to create AppSpec files and shell scripts can cause deployments to fail
Some text editors introduce non-conforming, non-printing characters into files. If you use text editors to create or modify AppSpec files or shell script files to run on Amazon Linux, Ubuntu Server, or RHEL instances, then any deployments that rely on these files might fail. When CodeDeploy uses these files during a deployment, the presence of these characters can lead to hard-to-troubleshoot AppSpec file validation failures and script execution failures.
In the CodeDeploy console, on the event details page for the deployment, choose **View logs**. (Or you use the AWS CLI to call the [get-deployment-instance](https://docs.aws.amazon.com/cli/latest/reference/deploy/get-deployment-instance.html) command.) Look for errors like `invalid character`, `command not found`, or `file not found`.
To address this issue, we recommend the following:
+ Do not use text editors that introduce non-printing characters such as carriage returns (`^M` characters) into your AppSpec files and shell script files.
+ Use text editors that display non-printing characters such as carriage returns in your AppSpec files and shell script files, so you can find and remove any that might be introduced. For examples of these types of text editors, search the internet for text editors that show carriage returns.
+ Use text editors running on Amazon Linux, Ubuntu Server, or RHEL instances to create shell script files that run on Amazon Linux, Ubuntu Server, or RHEL instances. For examples of these types of text editors, search the internet for Linux shell script editors.
+ If you must use a text editor in Windows or macOS to create shell script files to run on Amazon Linux, Ubuntu Server, or RHEL instances, use a program or utility that converts text in Windows or macOS format to Unix format. For examples of these programs and utilities, search the internet for DOS to UNIX or Mac to UNIX. Be sure to test the converted shell script files on the target operating systems.
## Using Finder in macOS to bundle an application revision can cause deployments to fail
Deployments might fail if you use the Finder graphical user interface (GUI) application on a Mac to bundle (zip) an AppSpec file and related files and scripts into an application revision archive (.zip) file. This is because Finder creates an intermediate `__MACOSX` folder in the .zip file and places component files into it. CodeDeploy cannot find the component files, so the deployment fails.
To address this issue, we recommend you use the AWS CLI to call the [push](https://docs.aws.amazon.com/cli/latest/reference/deploy/push.html) command, which zips the component files into the expected structure. Alternatively, you can use Terminal instead of the GUI to zip the component files. Terminal does not create an intermediate `__MACOSX` folder.
# Troubleshoot EC2/On-Premises deployment issues
**Topics**
+ [CodeDeploy plugin CommandPoller missing credentials error](#troubleshooting-agent-commandpoller-error)
+ [Deployment fails with the message “Validation of PKCS7 signed message failed”](#troubleshooting-deployments-agent-SHA-256)
+ [Deployment or redeployment of the same files to the same instance locations fail with the error "The deployment failed because a specified file already exists at this location"](#troubleshooting-same-files-different-app-name)
+ [Long file paths cause "No such file or directory" errors](#troubleshooting-long-file-paths)
+ [Long-running processes can cause deployments to fail](#troubleshooting-long-running-processes)
+ [Troubleshooting a failed AllowTraffic lifecycle event with no error reported in the deployment logs](#troubleshooting-deployments-allowtraffic-no-logs)
+ [Troubleshooting a failed ApplicationStop, BeforeBlockTraffic, or AfterBlockTraffic deployment lifecycle event](#troubleshooting-deployments-lifecycle-event-failures)
+ [Troubleshooting a failed DownloadBundle deployment lifecycle event with UnknownError: not opened for reading](#troubleshooting-deployments-downloadbundle)
+ [Troubleshooting all lifecycle events skipped errors](#troubleshooting-skipped-lifecycle-events)
+ [Windows PowerShell scripts fail to use the 64-bit version of Windows PowerShell by default](#troubleshooting-deployments-powershell)
**Note**
The causes of many deployment failures can be identified by reviewing the log files created during the deployment process. For simplicity, we recommend using Amazon CloudWatch Logs to centrally monitor log files instead of viewing them instance by instance. For information, see [View CodeDeploy Logs in CloudWatch Logs Console](https://aws.amazon.com/blogs/devops/view-aws-codedeploy-logs-in-amazon-cloudwatch-console/).
**Tip**
For a runbook that automates many troubleshooting tasks related to EC2/On-Premises deployments, see [AWSSupport-TroubleshootCodeDeploy](https://docs.aws.amazon.com/systems-manager-automation-runbooks/latest/userguide/automation-awssupport-troubleshootcodedeploy.html) in the *AWS Systems Manager Automation runbook reference*.
## CodeDeploy plugin CommandPoller missing credentials error
If you receive an error similar to `InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Missing credentials - please check if this instance was started with an IAM instance profile`, it might be caused by one of the following:
+ The instance you are deploying to does not have an IAM instance profile associated with it.
+ Your IAM instance profile does not have the correct permissions configured.
An IAM instance profile grants the CodeDeploy agent permission to communicate with CodeDeploy and to download your revision from Amazon S3. For EC2 instances, see [Identity and access management for AWS CodeDeploy](security-iam.md). For on-premises instances, see [Working with on-premises instances for CodeDeploy](instances-on-premises.md).
## Deployment fails with the message “Validation of PKCS7 signed message failed”
This error message indicates the instance is running a version of the CodeDeploy agent that supports only the SHA-1 hash algorithm. Support for the SHA-2 hash algorithm was introduced in version 1.0.1.854 of the CodeDeploy agent, released in November 2015. Effective October 17, 2016, deployments fail if a version of the CodeDeploy agent earlier than 1.0.1.854 is installed. For more information, see [AWS to switch to SHA256 hash algorithm for SSL Certificates](https://aws.amazon.com/security/security-bulletins/aws-to-switch-to-sha256-hash-algorithm-for-ssl-certificates/), [NOTICE: Retiring CodeDeploy host agents older than version 1.0.1.85](https://forums.aws.amazon.com/thread.jspa?threadID=223319), and [Update the CodeDeploy agent](codedeploy-agent-operations-update.md).
## Deployment or redeployment of the same files to the same instance locations fail with the error "The deployment failed because a specified file already exists at this location"
When CodeDeploy tries to deploy a file to an instance but a file with the same name already exists in the specified target location, the deployment to that instance may fail. You may receive the error message "The deployment failed because a specified file already exists at this location: *location-name*." This is because, during each deployment, CodeDeploy first deletes all files from the previous deployment, which are listed in a cleanup log file. If there are files in the target installation folders that aren’t listed in this cleanup file, the CodeDeploy agent by default interprets this as an error and fails the deployment.
**Note**
On Amazon Linux, RHEL, and Ubuntu Server instances, the cleanup file is located in `/opt/codedeploy-agent/deployment-root/deployment-instructions/`. On Windows Server instances, the location is `C:\ProgramData\Amazon\CodeDeploy\deployment-instructions\`.
The easiest way to avoid this error is to specify an option other than the default behavior to fail the deployment. For each deployment, you can choose whether to fail the deployment, to overwrite the files not listed in the cleanup file, or to retain the files already on the instance.
The overwrite option is useful when, for example, you manually placed a file on an instance after the last deployment, but then added a file of the same name to the next application revision.
You might choose the retain option for files you place on the instance that you want to be part of the next deployment without having to add them to the application revision package. The retain option is also useful if your application files are already in your production environment and you want to deploy using CodeDeploy for the first time. For more information, see [Create an EC2/On-Premises Compute Platform deployment (console)](deployments-create-console.md) and [Rollback behavior with existing content](deployments-rollback-and-redeploy.md#deployments-rollback-and-redeploy-content-options).
### Troubleshooting `The deployment failed because a specified file already exists at this location` deployment errors
If you choose not to specify an option to overwrite or retain content that CodeDeploy detects in your target deployment locations (or if you do not specify any deployment option for handling existing content in a programmatic command), you can choose to troubleshoot the error.
The following information applies only if you choose not to retain or overwrite the content.
If you try to redeploy files with the same names and locations, the redeployment is more likely to succeed if you specify the application name and the deployment group with the same underlying deployment group ID you used before. CodeDeploy uses the underlying deployment group ID to identify files to remove before a redeployment.
Deploying new files or redeploying the same files to the same locations on instances can fail for these reasons:
+ You specified a different application name for a redeployment of the same revision to the same instances. The redeployment fails because even if the deployment group name is the same, the use of a different application name means a different underlying deployment group ID is being used.
+ You deleted and re-created a deployment group for an application and then tried to redeploy the same revision to the deployment group. The redeployment fails because even if the deployment group name is the same, CodeDeploy references a different underlying deployment group ID.
+ You deleted an application and deployment group in CodeDeploy, and then created a new application and deployment group with the same names as the ones you deleted. After that, you tried to redeploy a revision that had been deployed to the previous deployment group to the new one with the same name. The redeployment fails because even though the application and deployment group names are the same, CodeDeploy still references the ID of the deployment group you deleted.
+ You deployed a revision to a deployment group and then deployed the same revision to another deployment group to the same instances. The second deployment fails because CodeDeploy references a different underlying deployment group ID.
+ You deployed a revision to one deployment group and then deployed another revision to another deployment group to the same instances. There is at least one file with the same name and in the same location that the second deployment group tries to deploy. The second deployment fails because CodeDeploy does not remove the existing file before the second deployment starts. Both deployments >reference different deployment group IDs.
+ You deployed a revision in CodeDeploy, but there is at least one file with the same name and in the same location. The deployment fails because, by default, CodeDeploy does not remove the existing file before the deployment starts.
To address these situations, do one of the following:
+ Remove the files from the locations and instances to which they were previously deployed, and then try the deployment again.
+ In your revision's AppSpec file, in either the ApplicationStop or BeforeInstall deployment lifecycle events, specify a custom script to delete files in any locations that match the files your revision is about to install.
+ Deploy or redeploy the files to locations or instances that were not part of previous deployments.
+ Before you delete an application or a deployment group, deploy a revision that contains an AppSpec file that specifies no files to copy to the instances. For the deployment, specify the application name and deployment group name that use the same underlying application and deployment group IDs as those you are about to delete. (You can use the [get-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/get-deployment-group.html) command to retrieve the deployment group ID.) CodeDeploy uses the underlying deployment group ID and AppSpec file to remove all of the files it installed in the previous successful deployment.
## Long file paths cause "No such file or directory" errors
For deployments to Windows instances, if you have a file path greater than 260 characters in the files section of your appspec.yml file, you may see deployments fail with an error similar to the following:
`No such file or directory @ dir_s_mkdir - C:\your-long-file-path`
This error occurs because Windows by default does not allow file paths greater than 260 characters, as detailed in [Microsoft’s documentation](https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=powershell#enable-long-paths-in-windows-10-version-1607-and-later).
For CodeDeploy agent versions 1.4.0 or later, you can enable long file paths in two ways, depending on the agent installation process:
If the CodeDeploy agent has not yet been installed:
1. On the machine where you plan to install the CodeDeploy agent, enable the `LongPathsEnabled` Windows registry key using this command:
```
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem"
-Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force
```
1. Install the CodeDeploy agent. For more information, see [Install the CodeDeploy agent](codedeploy-agent-operations-install.md).
If the CodeDeploy agent has already been installed:
1. On the CodeDeploy agent machine, enable the `LongPathsEnabled` Windows registry key using this command:
```
New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem"
-Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force
```
1. Restart the CodeDeploy agent for the registry key change to take effect. To restart the agent, use this command:
```
powershell.exe -Command Restart-Service -Name codedeployagent
```
## Long-running processes can cause deployments to fail
For deployments to Amazon Linux, Ubuntu Server, and RHEL instances, if you have a deployment script that starts a long-running process, CodeDeploy might spend a long time waiting in the deployment lifecycle event and then fail the deployment. This is because if the process runs longer than the foreground and background processes in that event are expected to take, CodeDeploy stops and fails the deployment, even if the process is still running as expected.
For example, an application revision contains two files in its root, `after-install.sh` and `sleep.sh`. Its AppSpec file contains the following instructions:
```
version: 0.0
os: linux
files:
- source: ./sleep.sh
destination: /tmp
hooks:
AfterInstall:
- location: after-install.sh
timeout: 60
```
The `after-install.sh` file runs during the AfterInstall application lifecycle event. Here are its contents:
```
#!/bin/bash
/tmp/sleep.sh
```
The `sleep.sh` file contains the following, which suspends program execution for three minutes (180 seconds), simulating some long-running process:
```
#!/bin/bash
sleep 180
```
When `after-install.sh` calls `sleep.sh`, `sleep.sh` starts and runs for three minutes (180 seconds), which is two minutes (120 seconds) past the time CodeDeploy expects `sleep.sh` (and, by relation, `after-install.sh`) to stop running. After the timeout of one minute (60 seconds), CodeDeploy stops and fails the deployment at the AfterInstall application lifecycle event, even though `sleep.sh` continues to run as expected. The following error is displayed:
`Script at specified location: after-install.sh failed to complete in 60 seconds`.
You cannot simply add an ampersand (`&`) in `after-install.sh` to run `sleep.sh` in the background.
```
#!/bin/bash
# Do not do this.
/tmp/sleep.sh &
```
Doing so can leave the deployment in a pending state for up to the default one-hour deployment lifecycle event timeout period, after which CodeDeploy stops and fails the deployment at the AfterInstall application lifecycle event as before.
In `after-install.sh`, call `sleep.sh` as follows, which enables CodeDeploy to continue after the process starts running:
```
#!/bin/bash
/tmp/sleep.sh > /dev/null 2> /dev/null < /dev/null &
```
In the preceding call, `sleep.sh` is the name of the process you want to start running in the background, redirecting stdout, stderr, and stdin to `/dev/null`.
## Troubleshooting a failed AllowTraffic lifecycle event with no error reported in the deployment logs
In some cases, a blue/green deployment fails during the AllowTraffic lifecycle event, but the deployment logs do not indicate the cause for the failure.
This failure is typically due to incorrectly configured health checks in Elastic Load Balancing for the Classic Load Balancer, Application Load Balancer, or Network Load Balancer used to manage traffic for the deployment group.
To resolve the issue, review and correct any errors in the health check configuration for the load balancer.
For Classic Load Balancers, see [Configure Health Checks](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-healthchecks.html) in the *User Guide for Classic Load Balancers* and [ConfigureHealthCheck](https://docs.aws.amazon.com/elasticloadbalancing/2012-06-01/APIReference/API_ConfigureHealthCheck.html) in the *Elastic Load Balancing API Reference version 2012-06-01*.
For Application Load Balancers, see [Health Checks for Your Target Groups](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/target-group-health-checks.html) in the *User Guide for Application Load Balancers*.
For Network Load Balancers, see [Health Checks for Your Target Groups](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/target-group-health-checks.html) in the *Network Load Balancer User Guide*.
## Troubleshooting a failed ApplicationStop, BeforeBlockTraffic, or AfterBlockTraffic deployment lifecycle event
During a deployment, the CodeDeploy agent runs the scripts specified for ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic in the AppSpec file from the previous successful deployment. (All other scripts are run from the AppSpec file in the current deployment.) If one of these scripts contains an error and does not run successfully, the deployment can fail.
Possible reasons for these failures include:
+ The CodeDeploy agent finds the `deployment-group-id_last_successful_install` file in the correct location, but the location listed in the `deployment-group-id_last_successful_install` file does not exist.
**On Amazon Linux, Ubuntu Server, and RHEL instances**, this file must exist in `/opt/codedeploy-agent/deployment-root/deployment-instructions`.
**On Windows Server instances**, this file must be stored in the `C:\ProgramData\Amazon\CodeDeploy\deployment-instructions` folder.
+ In the location listed in the `deployment-group-id_last_successful_install` file, either the AppSpec file is invalid or the scripts do not run successfully.
+ The script contains an error that cannot be corrected, so it never runs successfully.
Use the CodeDeploy console to investigate why a deployment might have failed during any of these events. On the details page for the deployment, choose **View events**. On the details page for the instance, in the **ApplicationStop**, **BeforeBlockTraffic**, or **AfterBlockTraffic** row, choose **View logs**. Or use the AWS CLI to call the [get-deployment-instance](https://docs.aws.amazon.com/cli/latest/reference/deploy/get-deployment-instance.html) command.
If the cause of the failure is a script from the last successful deployment that never runs successfully, create a deployment and specify that the ApplicationStop, BeforeBlockTraffic, and AfterBlockTraffic failures should be ignored. There are two ways to do this:
+ Use the CodeDeploy console to create a deployment. On the **Create deployment** page, under **ApplicationStop lifecycle event failure**, choose **Don't fail the deployment to an instance if this lifecycle event on the instance fails**.
+ Use the AWS CLI to call the **[create-deployment](https://docs.aws.amazon.com/cli/latest/reference/deploy/create-deployment.html)** command and include the `--ignore-application-stop-failures` option.
When you deploy the application revision again, the deployment continues even if any of these three lifecycle events fail. If the new revision includes fixed scripts for those lifecycle events, future deployments can succeed without applying this fix.
## Troubleshooting a failed DownloadBundle deployment lifecycle event with UnknownError: not opened for reading
If you are trying to deploy an application revision from Amazon S3, and the deployment fails during the DownloadBundle deployment lifecycle event with the `UnknownError: not opened for reading` error:
+ There was internal Amazon S3 service error. Deploy the application revision again.
+ The IAM instance profile on your EC2 instance does not have permissions to access the application revision in Amazon S3. For information about Amazon S3 bucket policies, see [Push a revision for CodeDeploy to Amazon S3 (EC2/On-Premises deployments only)](application-revisions-push.md) and [Deployment prerequisites](deployments-create-prerequisites.md).
+ The instances you deploy to are associated with one AWS Region (for example, US West (Oregon)), but the Amazon S3 bucket that contains the application revision is associated with another AWS Region (for example, US East (N. Virginia)). Make sure the application revision is in an Amazon S3 bucket associated with the same AWS Region as the instances.
On the event details page for the deployment, in the **Download bundle** row, choose **View logs**. Or use the AWS CLI to call the [get-deployment-instance](https://docs.aws.amazon.com/cli/latest/reference/deploy/get-deployment-instance.html) command. If this error occurred, there should be an error in the output with the error code `UnknownError` and the error message `not opened for reading`.
To determine the reason for this error:
1. Enable wire logging on at least one of the instances, and then deploy the application revision again.
1. Examine the wire logging file to find the error. Common error messages for this issue include the phrase "access denied."
1. After you have examined the log files, we recommend that you disable wire logging to reduce log file size and the amount of sensitive information that might appear in the output in plain text on the instance in the future.
For information about how to find the wire logging file and enable and disable wire logging, see `:log_aws_wire:` in [CodeDeploy agent configuration reference](https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-agent-configuration.html).
## Troubleshooting all lifecycle events skipped errors
If all lifecycle events of an EC2 or on-premises deployment are skipped, you might receive an error similar to `The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems. (Error code: HEALTH_CONSTRAINTS)`. Here are some possible causes and solutions:
+ The CodeDeploy agent might not be installed or running on the instance. To determine if the CodeDeploy agent is running:
+ For Amazon Linux RHEL or Ubuntu server, run the following:
```
systemctl status codedeploy-agent
```
+ For Windows, run the following:
```
powershell.exe -Command Get-Service -Name CodeDeployagent
```
If the CodeDeploy agent is not installed or running, see [Verify the CodeDeploy agent is running](codedeploy-agent-operations-verify.md).
Your instance might not be able to reach the CodeDeploy or Amazon S3 public endpoint using port 443. Try one of the following:
+ Assign a public IP address to the instance and use its route table to allow internet access. Make sure the security group associated with the instance allows outbound access over port 443 (HTTPS). For more information, see [Communication protocol and port for the CodeDeploy agent](codedeploy-agent.md#codedeploy-agent-outbound-port).
+ If an instance is provisioned in a private subnet, use a NAT gateway instead of an internet gateway in the route table. For more information, see [NAT Gateways](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html).
+ The service role for CodeDeploy might not have required permissions. To configure a CodeDeploy service role, see [Step 2: Create a service role for CodeDeploy](getting-started-create-service-role.md).
+ If you use an HTTP proxy, make sure it is specified in the `:proxy_uri:` setting in the CodeDeploy agent configuration file. For more information, see [CodeDeploy agent configuration reference](reference-agent-configuration.md).
+ The date and time signature of your deployment instance might not match the date and time signature of your deployment request. Look for an error similar to `Cannot reach InstanceService: Aws::CodeDeployCommand::Errors::InvalidSignatureException - Signature expired` in your CodeDeploy agent log file. If you see this error, follow the steps in [Troubleshooting “InvalidSignatureException – Signature expired: [time] is now earlier than [time]” deployment errors](troubleshooting-ec2-instances.md#troubleshooting-instance-time-failures). For more information, see [View log data for CodeDeploy EC2/On-Premises deployments](deployments-view-logs.md).
+ The CodeDeploy agent might stop running because an instance is running low on memory or hard disk space. Try to lower the number of archived deployments on your instance by updating the `max_revisions` setting in the CodeDeploy agent configuration. If you do this for an EC2 instance and the issue persists, consider using a larger instance. For example, if your instance type is `t2.small`, try using a `t2.medium`. For more information, see [Files installed by the CodeDeploy agent](codedeploy-agent.md#codedeploy-agent-install-files), [CodeDeploy agent configuration reference](reference-agent-configuration.md), and [Instance types](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html).
+ The instance you're deploying to might not have an IAM instance profile attached, or it might have an IAM instance profile attached that does not have the required permissions.
+ If an IAM instance profile is not attached to your instance, create one with the required permissions and then attach it.
+ If an IAM instance profile is already attached to your instance, make sure it has the required permissions.
After you confirm your attached instance profile is configured with the required permissions, restart your instance. For more information, see [Step 4: Create an IAM instance profile for your Amazon EC2 instances](getting-started-create-iam-instance-profile.md) and [IAM Roles for Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-EC2.html) in the *Amazon EC2 User Guide*.
## Windows PowerShell scripts fail to use the 64-bit version of Windows PowerShell by default
If a Windows PowerShell script running as part of a deployment relies on 64-bit functionality (for example, because it consumes more memory than a 32-bit application allows or calls libraries that are offered only in a 64-bit version), the script might crash or not run as expected. This is because, by default, CodeDeploy uses the 32-bit version of Windows PowerShell to run Windows PowerShell scripts that are part of an application revision.
Add code like the following to the beginning of any script that must run with the 64-bit version of Windows PowerShell:
```
# Are you running in 32-bit mode?
# (\SysWOW64\ = 32-bit mode)
if ($PSHOME -like "*SysWOW64*")
{
Write-Warning "Restarting this script under 64-bit Windows PowerShell."
# Restart this script under 64-bit Windows PowerShell.
# (\SysNative\ redirects to \System32\ for 64-bit mode)
& (Join-Path ($PSHOME -replace "SysWOW64", "SysNative") powershell.exe) -File `
(Join-Path $PSScriptRoot $MyInvocation.MyCommand) @args
# Exit 32-bit script.
Exit $LastExitCode
}
# Was restart successful?
Write-Warning "Hello from $PSHOME"
Write-Warning " (\SysWOW64\ = 32-bit mode, \System32\ = 64-bit mode)"
Write-Warning "Original arguments (if any): $args"
# Your 64-bit script code follows here...
# ...
```
Although the file path information in this code might seem counterintuitive, 32-bit Windows PowerShell uses a path like:
`c:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe`
64-bit Windows PowerShell uses a path like:
`c:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe`
# Troubleshoot Amazon ECS deployment issues
**Topics**
+ [A timeout occurs while waiting for replacement task set](#troubleshooting-ecs-timeout)
+ [A timeout occurs while waiting for a notification to continue](#troubleshooting-ecs-timeout-notif)
+ [The IAM role does not have enough permissions](#troubleshooting-ecs-iam)
+ [The deployment timed out while waiting for a status callback](#troubleshooting-ecs-timeout-callback)
+ [The deployment failed because one or more of the lifecycle event validation functions failed](#troubleshooting-ecs-lifecycle)
+ [The ELB could not be updated due to the following error: Primary taskset target group must be behind listener](#troubleshooting-ecs-elb)
+ [My deployment sometimes fails when using Auto Scaling](#troubleshooting-ecs-auto-scaling)
+ [Only ALB supports gradual traffic routing, use AllAtOnce Traffic routing instead when you create/update Deployment group](#troubleshooting-ecs-lb)
+ [Even though my deployment succeeded, the replacement task set fails the Elastic Load Balancing health checks, and my application is down](#troubleshooting-ecs-task-set-stability)
+ [Can I attach multiple load balancers to a deployment group?](#troubleshooting-ecs-lb-multi)
+ [Can I perform CodeDeploy blue/green deployments without a load balancer?](#troubleshooting-ecs-lb-bg)
+ [How can I update my Amazon ECS service with new information during a deployment?](#troubleshooting-ecs-exec)
## A timeout occurs while waiting for replacement task set
**Problem**: You see the following error message while deploying your Amazon ECS application using CodeDeploy:
`The deployment timed out while waiting for the replacement task set to become healthy. This time out period is 60 minutes.`
**Possible cause**: This error might occur if there is a mistake in your task definition file or other deployment-related files. For example, if there is a typo in the `image` field in your task definition file, Amazon ECS will try to pull the wrong container image and continuously fail, causing this error.
**Possible fixes and next steps**:
+ Fix typographical errors and configuration problems in your task definition file and other files.
+ Check the related Amazon ECS service event and find out why replacement tasks are not becoming healthy. For more information on Amazon ECS events, see [Amazon ECS events](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_cwe_events.html) in the *Amazon Elastic Container Service Developer Guide*.
+ Check the [Amazon ECS troubleshooting](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/troubleshooting.html) section in the *Amazon Elastic Container Service Developer Guide* for errors related to the messages in the event.
## A timeout occurs while waiting for a notification to continue
**Problem**: You see the following error message while deploying your Amazon ECS application using CodeDeploy:
`The deployment timed out while waiting for a notification to continue. This time out period is n minutes.`
**Possible cause**: This error might occur if you specified a wait time in the **Specify when to reroute traffic** field when you created your deployment group, but the deployment couldn't finish before the wait time expired.
**Possible fixes and next steps**:
+ In your deployment group, set the **Specify when to reroute traffic** to a larger amount of time and redeploy. For more information, see [Create a deployment group for an Amazon ECS deployment (console)](deployment-groups-create-ecs.md).
+ In your deployment group, change **Specify when to reroute traffic** to **Reroute traffic immediately** and redeploy. For more information, see [Create a deployment group for an Amazon ECS deployment (console)](deployment-groups-create-ecs.md).
+ Redeploy and then run the [https://docs.aws.amazon.com/cli/latest/reference/deploy/continue-deployment.html](https://docs.aws.amazon.com/cli/latest/reference/deploy/continue-deployment.html) AWS CLI command with the `--deployment-wait-type` option set to `READY_WAIT`. Make sure to run this command *before* the time specified in **Specify when to reroute traffic** expires.
## The IAM role does not have enough permissions
**Problem**: You see the following error message while deploying your Amazon ECS application using CodeDeploy:
`The IAM role role-arn does not give you permission to perform operations in the following AWS service: AWSLambda.`
**Possible cause**: This error might occur if you specified a Lambda function in the [AppSpec file's `Hooks` section](reference-appspec-file-structure-hooks.md#appspec-hooks-ecs), but you did not give CodeDeploy permission to the Lambda service.
**Possible fix**: Add the `lambda:InvokeFunction` permission to the CodeDeploy service role. To add this permission, add one of the following AWS-managed policies to the role: **AWSCodeDeployRoleForECS** or **AWSCodeDeployRoleForECSLimited**. For information about these policies and how to add them to the CodeDeploy service role, see [Step 2: Create a service role for CodeDeploy](getting-started-create-service-role.md).
## The deployment timed out while waiting for a status callback
**Problem**: You see the following error message while deploying your Amazon ECS application using CodeDeploy:
`The deployment timed out while waiting for a status callback. CodeDeploy expects a status callback within one hour after a deployment hook is invoked.`
**Possible cause**: This error might occur if you specified a Lambda function in the [AppSpec file's `Hooks` section](reference-appspec-file-structure-hooks.md#appspec-hooks-ecs), but Lambda function could not call the necessary `PutLifecycleEventHookExecutionStatus` API to return a `Succeeded` or `Failed` status to CodeDeploy.
**Possible fixes and next steps**:
+ Add the `codedeploy:putlifecycleEventHookExecutionStatus` permission to the Lambda execution role used by the Lambda function that you specified in the AppSpec file. This permission grants the Lambda function the ability to return a status of `Succeeded` or `Failed` to CodeDeploy. For more information about the Lambda execution role, see [Lambda execution role](https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html) in the *AWS Lambda User Guide*.
+ Check your Lambda function code and execution logs to make sure your Lambda function is calling CodeDeploy's `PutLifecycleEventHookExecutionStatus` API to inform CodeDeploy about whether the lifecycle validation test `Succeeded` or `Failed`. For information about the `putlifecycleEventHookExecutionStatus` API, see [PutLifecycleEventHookExecutionStatus](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_PutLifecycleEventHookExecutionStatus.html) in the *AWS CodeDeploy API Reference*. For information about Lambda execution logs, see [Accessing Amazon CloudWatch logs for AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html).
## The deployment failed because one or more of the lifecycle event validation functions failed
**Problem**: You see the following error message while deploying your Amazon ECS application using CodeDeploy:
`The deployment failed because one or more of the lifecycle event validation functions failed.`
**Possible cause**: This error might occur if you specified a Lambda function in the [AppSpec file's `Hooks` section](reference-appspec-file-structure-hooks.md#appspec-hooks-ecs), but the Lambda function returned `Failed` to CodeDeploy when it called `PutLifecycleEventHookExecutionStatus`. This failure indicates to CodeDeploy that the lifecycle validation test failed.
**Possible next step**: Check your Lambda execution logs to see why the validation test code is failing. For information about Lambda execution logs, see [Accessing Amazon CloudWatch logs for AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-cloudwatchlogs.html).
## The ELB could not be updated due to the following error: Primary taskset target group must be behind listener
**Problem**: You see the following error message while deploying your Amazon ECS application using CodeDeploy:
`The ELB could not be updated due to the following error: Primary taskset target group must be behind listener`
**Possible cause**: This error might occur if you have configured an optional test listener, and it is configured with wrong target group. For more information about the test listener in CodeDeploy, see [Before you begin an Amazon ECS deployment](deployment-steps-ecs.md#deployment-steps-prerequisites-ecs) and [What happens during an Amazon ECS deployment](deployment-steps-ecs.md#deployment-steps-what-happens). For more information about task sets, see [TaskSet](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_TaskSet.html) in the *Amazon Elastic Container Service API Reference* and [describe-task-set](https://docs.aws.amazon.com/cli/latest/reference/ecs/describe-task-set.html) in the Amazon ECS section of the *AWS CLI Command Reference*.
**Possible fix**: Make sure that the Elastic Load Balancing's production listener and test listener are both pointing to the target group that's currently serving your workloads. There are three places to check:
+ In Amazon EC2, in your load balancer's **Listeners and rules** settings. For more information, see [Listeners for your Application Load Balancers](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html) in the *User Guide for Application Load Balancers*, or [Listeners for your Network Load Balancers](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-listeners.html) in the *User Guide for Network Load Balancers*.
+ In Amazon ECS, in your cluster, under your service's **Networking** configuration. For more information, see [Application Load Balancer and Network Load Balancer considerations](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/load-balancer-types.html#alb-considerations) in the *Amazon Elastic Container Service Developer Guide*.
+ In CodeDeploy, in your deployment group settings. For more information, see [Create a deployment group for an Amazon ECS deployment (console)](deployment-groups-create-ecs.md).
## My deployment sometimes fails when using Auto Scaling
**Problem**: You are using Auto Scaling with CodeDeploy and you notice that your deployments occasionally fail. For more information about the symptoms of this problem, see the topic that reads [For services configured to use service auto scaling and the blue/green deployment type, auto scaling is not blocked during a deployment but the deployment may fail under some circumstances](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deployment-type-bluegreen.html#deployment-type-bluegreen-considerations) in the *Amazon Elastic Container Service Developer Guide*.
**Possible cause**: This problem might occur if CodeDeploy and Auto Scaling processes conflict.
**Possible fix**: Suspend and resume Auto Scaling processes during the CodeDeploy deployment using the `RegisterScalableTarget` API (or the corresponding `register-scalable-target` AWS CLI command). For more information, see [Suspend and resume scaling for Application Auto Scaling](https://docs.aws.amazon.com/autoscaling/application/userguide/application-auto-scaling-suspend-resume-scaling.html) in the *Application Auto Scaling User Guide*.
**Note**
CodeDeploy can't call `RegisterScaleableTarget` directly. To use this API, you must configure CodeDeploy to send a notification or event to Amazon Simple Notification Service (or Amazon CloudWatch). You must then configure Amazon SNS (or CloudWatch) to call a Lambda function, and configure the Lambda function to call the `RegisterScalableTarget` API. The `RegisterScalableTarget` API must be called with the `SuspendedState` parameter set to `true` to suspend Auto Scaling operations, and `false` to resume them.
The notification or event that CodeDeploy sends out must occur when a deployment starts (to trigger Auto Scaling suspend operations), or when a deployment succeeds, fails, or stops (to trigger Auto Scaling resume operations).
For information about how to configure CodeDeploy to generate Amazon SNS notifications or CloudWatch events, see [Monitoring deployments with Amazon CloudWatch Events](monitoring-cloudwatch-events.md). and [Monitoring deployments with Amazon SNS event notifications](monitoring-sns-event-notifications.md).
## Only ALB supports gradual traffic routing, use AllAtOnce Traffic routing instead when you create/update Deployment group
**Problem**: You see the following error message while creating or updating a deployment group in CodeDeploy:
`Only ALB supports gradual traffic routing, use AllAtOnce Traffic routing instead when you create/update Deployment group.`
**Possible cause**: This error might occur if you're using a Network Load Balancer and tried to use a predefined deployment configuration other than `CodeDeployDefault.ECSAllAtOnce`.
**Possible fixes**:
+ Change your predefined deployment configuration to `CodeDeployDefault.ECSAllAtOnce`. This is the only predefined deployment configuration supported by Network Load Balancers.
For more information about predefined deployment configurations, see [Predefined deployment configurations for an Amazon ECS compute platform](deployment-configurations.md#deployment-configurations-predefined-ecs).
+ Change your load balancer to an Application Load Balancer. Application Load Balancer's support all the predefined deployment configurations. For more information about creating a Application Load Balancer, see [Set up a load balancer, target groups, and listeners for CodeDeploy Amazon ECS deployments](deployment-groups-create-load-balancer-for-ecs.md).
## Even though my deployment succeeded, the replacement task set fails the Elastic Load Balancing health checks, and my application is down
**Problem**: Even though CodeDeploy indicates that my deployment succeeded, the replacement task set fails the health checks from Elastic Load Balancing, and my application is down.
**Possible cause**: This issue might occur if you performed a CodeDeploy all-at-once deployment, and your replacement (green) task set contains bad code that is causing the Elastic Load Balancing health checks to fail. With the all-at-once deployment configuration, the load balancer’s health checks start running on the replacement task set *after* traffic has been shifted to it (that is, *after* CodeDeploy’s `AllowTraffic` lifecycle event occurs). That’s why you will see health checks failing on the replacement task set after traffic has shifted, but not before. For information about the lifecycle events that CodeDeploy generates, see [What happens during an Amazon ECS deployment](deployment-steps-ecs.md#deployment-steps-what-happens).
**Possible fixes**:
+ Change your deployment configuration from all-at-once to canary or linear. In a canary or linear configuration, the load balancer’s health checks start running on the replacement task set while CodeDeploy installs your application in the replacement environment, and *before* traffic is shifted (that is, during the `Install` lifecycle event, and *before* the `AllowTraffic` event). By allowing the checks to run during the application installation but before traffic is shifted, bad application code will be detected and cause deployment failures before the application becomes publicly available.
For information about how to configure canary or linear deployments, see [Change deployment group settings with CodeDeploy](deployment-groups-edit.md).
For information about CodeDeploy lifecycle events that run during an Amazon ECS deployment, see [What happens during an Amazon ECS deployment](deployment-steps-ecs.md#deployment-steps-what-happens).
**Note**
Canary and linear deployment configurations are only supported with Application Load Balancers.
+ If you want to keep your all-at-once deployment configuration, set up a test listener and check the health status of the replacement task set with the `BeforeAllowTraffic` lifecycle hook. For more information, see [List of lifecycle event hooks for an Amazon ECS deployment](reference-appspec-file-structure-hooks.md#reference-appspec-file-structure-hooks-list-ecs).
## Can I attach multiple load balancers to a deployment group?
No. If you want to use multiple Application Load Balancers or Network Load Balancers, use Amazon ECS rolling updates instead of CodeDeploy blue/green deployments. For more information about rolling updates, see [Rolling update](https://docs.aws.amazon.com/AmazonECS/latest/userguide/deployment-type-ecs.html) in the *Amazon Elastic Container Service Developer Guide*. For more information about using multiple load balancers with Amazon ECS, see [Registering multiple target groups with a service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/register-multiple-targetgroups.html) in the *Amazon Elastic Container Service Developer Guide*.
## Can I perform CodeDeploy blue/green deployments without a load balancer?
No, you cannot perform CodeDeploy blue/green deployments without a load balancer. If you are unable to use a load balancer, use Amazon ECS's rolling updates feature instead. For more information about Amazon ECS rolling updates, see [Rolling update](https://docs.aws.amazon.com/AmazonECS/latest/userguide/deployment-type-ecs.html) in the *Amazon Elastic Container Service Developer Guide*.
## How can I update my Amazon ECS service with new information during a deployment?
To have CodeDeploy update your Amazon ECS service with a new parameter while it conducts a deployment, specify the parameter in the `resources` section of the AppSpec file. Only a few Amazon ECS parameters are supported by CodeDeploy, such as the task definition file and container name parameters. For a full list of Amazon ECS parameters that CodeDeploy can update, see [AppSpec 'resources' section for Amazon ECS deployments](reference-appspec-file-structure-resources.md#reference-appspec-file-structure-resources-ecs).
**Note**
If you need to update your Amazon ECS service with a parameter that is not supported by CodeDeploy, complete these tasks:
Call Amazon ECS's `UpdateService` API with the parameter you want to update. For a full list of parameters that can be updated, see [UpdateService](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_UpdateService.html) in the *Amazon Elastic Container Service API Reference*.
To apply the change to the tasks, create a new Amazon ECS blue/green deployment. For more information, see [Create an Amazon ECS Compute Platform deployment (console)](deployments-create-console-ecs.md).
# Troubleshoot AWS Lambda deployment issues
**Topics**
+ [AWS Lambda deployments fail after manually stopping a Lambda deployment that does not have configured rollbacks](#troubleshooting-manually-stopped-lambda-deployment)
## AWS Lambda deployments fail after manually stopping a Lambda deployment that does not have configured rollbacks
In some cases, the alias of a Lambda function specified in a deployment might reference two different versions of the function. The result is that subsequent attempts to deploy the Lambda function fail. A Lambda deployment can get in this state when it does not have rollbacks configured and is manually stopped. To proceed, use the AWS Lambda console to make sure the function is not configured to shift traffic between two versions:
1. Sign in to the AWS Management Console and open the AWS Lambda console at [https://console.aws.amazon.com/lambda/](https://console.aws.amazon.com/lambda/).
1. From the left pane, choose **Functions**.
1. Select the name of the Lambda function that is in your CodeDeploy deployment.
1. From **Aliases**, choose the alias used in your CodeDeploy deployment, and then choose **Edit**.
1. From **Weighted alias**, choose **none**. This ensures the alias is not configured to shift a percentage, or weight, of traffic to more than one version. Make a note of the version selected in **Version**.
1. Choose **Save**.
1. Open the CodeDeploy console and attempt a deployment of the version displayed in the drop-down menu in step 5.
# Troubleshoot deployment group issues
## Tagging an instance as part of a deployment group does not automatically deploy your application to the new instance
CodeDeploy does not automatically deploy your application to a newly tagged instance. You must create a new deployment in the deployment group.
You can use CodeDeploy to enable automatic deployments to new EC2 instances in Amazon EC2 Auto Scaling groups. For more information, see [Integrating CodeDeploy with Amazon EC2 Auto Scaling](integrations-aws-auto-scaling.md).
# Troubleshoot instance issues
**Topics**
+ [Tags must be set correctly](#troubleshooting-EC2-tags)
+ [AWS CodeDeploy agent must be installed and running on instances](#troubleshooting-sds-agent)
+ [Deployments do not fail for up to an hour when an instance is terminated during a deployment](#troubleshooting-one-hour-timeout)
+ [Analyzing log files to investigate deployment failures on instances](#troubleshooting-deploy-failures)
+ [Create a new CodeDeploy log file if it was accidentally deleted](#troubleshooting-create-new-log-file)
+ [Troubleshooting “InvalidSignatureException – Signature expired: [time] is now earlier than [time]” deployment errors](#troubleshooting-instance-time-failures)
## Tags must be set correctly
Use the [list-deployment-instances](https://docs.aws.amazon.com/cli/latest/reference/deploy/list-deployment-instances.html) command to confirm the instances used for a deployment are tagged correctly. If an EC2 instance is missing in the output, use the EC2 console to confirm the tags have been set on the instance. For more information, see [Working with tags in the console](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html#Using_Tags_Console) in the *Amazon EC2 User Guide*.
**Note**
If you tag an instance and immediately use CodeDeploy to deploy an application to it, the instance might not be included in the deployment. This is because it can take several minutes before CodeDeploy can read the tags. We recommend that you wait at least five minutes between the time you tag an instance and attempt to deploy to it.
## AWS CodeDeploy agent must be installed and running on instances
To verify the CodeDeploy agent is installed and running on an instance, see [Verify the CodeDeploy agent is running](codedeploy-agent-operations-verify.md).
To install, uninstall, or reinstall the CodeDeploy agent, see [Install the CodeDeploy agent](codedeploy-agent-operations-install.md).
## Deployments do not fail for up to an hour when an instance is terminated during a deployment
CodeDeploy provides a one-hour window for each deployment lifecycle event to run to completion. This provides ample time for long-running scripts.
If the scripts don't run to completion while a lifecycle event is in progress (for example, if an instance is terminated or the CodeDeploy agent is shut down), it might take up to an hour for the status of the deployment to be displayed as Failed. This is true even if the timeout period specified in the script is shorter than an hour. This is because when the instance is terminated, the CodeDeploy agent shuts down and cannot process more scripts.
If an instance is terminated between lifecycle events or before the first lifecycle event step starts, the timeout occurs after just five minutes.
## Analyzing log files to investigate deployment failures on instances
If the status of an instance in the deployment is anything other than `Succeeded`, you can review the deployment log file data to help identify the problem. For information about accessing deployment log data, see [View log data for CodeDeploy EC2/On-Premises deployments](deployments-view-logs.md).
## Create a new CodeDeploy log file if it was accidentally deleted
If you accidentally delete the deployment log file on an instance, CodeDeploy does not create a replacement log file. To create a new log file, sign in to the instance, and then run these commands:
**For an Amazon Linux, Ubuntu Server, or RHEL instance**, run these commands in this order, one at a time:
```
systemctl stop codedeploy-agent
```
```
systemctl start codedeploy-agent
```
**For a Windows Server instance**:
```
powershell.exe -Command Restart-Service -Name codedeployagent
```
## Troubleshooting “InvalidSignatureException – Signature expired: [time] is now earlier than [time]” deployment errors
CodeDeploy requires accurate time references to perform its operations. If the date and time on your instance are not set correctly, they might not match the signature date of your deployment request, which CodeDeploy rejects.
To avoid deployment failures related to incorrect time settings, see the following topics:
+ [Setting the Time for Your Linux Instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html)
+ [Setting the Time for a Windows Instance](https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/windows-set-time.html)
# Troubleshoot GitHub token issues
## Invalid GitHub OAuth token
CodeDeploy applications created after June 2017 use GitHub OAuth tokens for each AWS Region. The use of tokens tied to specific AWS Regions gives you more control over which CodeDeploy applications have access to a GitHub repository.
If you receive a GitHub token error, you might have an older token that is now invalid.
**To fix an invalid GitHub OAuth token**
1. Remove the old token using one of the following methods:
+ To remove the old token using the API, use [ DeleteGitHubAccountToken](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_DeleteGitHubAccountToken.html).
+ To remove the old token using the AWS Command Line Interface:
1. Go to the computer where the token resides.
1. Make sure the AWS CLI is installed on this computer. For installation instructions, see [Installing, updating, and uninstalling the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) in the *AWS Command Line Interface User Guide*
1. Enter the following command on the computer where the token resides:
**aws delete-git-hub-account-token**
For details on the command syntax, see [delete-git-hub-account-token](https://docs.aws.amazon.com/cli/latest/reference/deploy/delete-git-hub-account-token.html).
1. Add a new OAuth token. For more information, see [Integrating CodeDeploy with GitHub](integrations-partners-github.md).
## Maximum number of GitHub OAuth tokens exceeded
When you create a CodeDeploy deployment, the maximum number of allowed GitHub tokens is 10. If you receive an error about GitHub OAuth tokens, make sure you have 10 or fewer tokens. If you have more than 10 tokens, the first tokens that were created are invalid. For example, if you have 11 tokens, the first token you created is invalid. If you have 12 tokens, the first two tokens you created are invalid. For information about using the CodeDeploy API to remove old tokens, see [ DeleteGitHubAccountToken](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_DeleteGitHubAccountToken.html).
# Troubleshoot Amazon EC2 Auto Scaling issues
**Topics**
+ [General Amazon EC2 Auto Scaling troubleshooting](#troubleshooting-auto-scaling-general)
+ ["CodeDeployRole does not give you permission to perform operations in the following AWS service: AmazonAutoScaling" error](#troubleshooting-auto-scaling-permissions-error)
+ [Instances in an Amazon EC2 Auto Scaling group are continuously provisioned and terminated before a revision can be deployed](#troubleshooting-auto-scaling-provision-termination-loop)
+ [Terminating or rebooting an Amazon EC2 Auto Scaling instance might cause deployments to fail](#troubleshooting-auto-scaling-reboot)
+ [Avoid associating multiple deployment groups with a single Amazon EC2 Auto Scaling group](#troubleshooting-multiple-depgroups)
+ [EC2 instances in an Amazon EC2 Auto Scaling group fail to launch and receive the error "Heartbeat Timeout"](#troubleshooting-auto-scaling-heartbeat)
+ [Mismatched Amazon EC2 Auto Scaling lifecycle hooks might cause automatic deployments to Amazon EC2 Auto Scaling groups to stop or fail](#troubleshooting-auto-scaling-hooks)
+ ["The deployment failed because no instances were found for your deployment group" error](#troubleshooting-deployment-failed-because-no-instances-found)
## General Amazon EC2 Auto Scaling troubleshooting
Deployments to EC2 instances in an Amazon EC2 Auto Scaling group can fail for the following reasons:
+ **Amazon EC2 Auto Scaling continuously launches and terminates EC2 instances.** If CodeDeploy cannot automatically deploy your application revision, Amazon EC2 Auto Scaling continuously launches and terminates EC2 instances.
Disassociate the Amazon EC2 Auto Scaling group from the CodeDeploy deployment group or change the configuration of your Amazon EC2 Auto Scaling group so that the desired number of instances matches the current number of instances (thus preventing Amazon EC2 Auto Scaling from launching any more EC2 instances). For more information, see [Change deployment group settings with CodeDeploy](deployment-groups-edit.md) or [Manual Scaling for Amazon EC2 Auto Scaling](https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-manual-scaling.html).
+ **The CodeDeploy agent is unresponsive.** The CodeDeploy agent might not be installed if initialization scripts (for example, cloud-init scripts) that run immediately after an EC2 instance is launched or started take more than one hour to run. CodeDeploy has a one-hour timeout for the CodeDeploy agent to respond to pending deployments. To address this issue, move your initialization scripts into your CodeDeploy application revision.
+ **An EC2 instance in an Amazon EC2 Auto Scaling group reboots during a deployment.** Your deployment can fail if an EC2 instance is rebooted during a deployment or the CodeDeploy agent is shut down while processing a deployment command. For more information, see [Terminating or rebooting an Amazon EC2 Auto Scaling instance might cause deployments to fail](#troubleshooting-auto-scaling-reboot).
+ **Multiple application revisions are deployed simultaneously to the same EC2 instance in an Amazon EC2 Auto Scaling group.** Deploying multiple application revisions to the same EC2 instance in an Amazon EC2 Auto Scaling group at the same time can fail if one of the deployments has scripts that run for more than a few minutes. Do not deploy multiple application revisions to the same EC2 instances in an Amazon EC2 Auto Scaling group.
+ **A deployment fails for new EC2 instances that are launched as part of an Amazon EC2 Auto Scaling group.** In this scenario, running the scripts in a deployment can prevent the launching of EC2 instances in the Amazon EC2 Auto Scaling group. (Other EC2 instances in the Amazon EC2 Auto Scaling group might appear to be running normally.) To address this issue, make sure that all other scripts are complete first:
+ **CodeDeploy agent is not included in your AMI **: If you use the **cfn-init** command to install the CodeDeploy agent while launching a new instance, place the agent installation script at the end of the `cfn-init` section of your CloudFormation template.
+ **CodeDeploy agent is included in your AMI **: Configure the AMI so that the agent is in a `Stopped` state when the instance is created, and then include a script for starting the agent as the final step in your `cfn-init` script library.
## "CodeDeployRole does not give you permission to perform operations in the following AWS service: AmazonAutoScaling" error
Deployments that use an Auto Scaling group created with a launch template require the following permissions. These are in addition to the permissions granted by the `AWSCodeDeployRole` AWS managed policy.
+ `EC2:RunInstances`
+ `EC2:CreateTags`
+ `iam:PassRole`
You might received this error if you are missing these permissions. For more information, see [Tutorial: Use CodeDeploy to deploy an application to an Auto Scaling group](tutorials-auto-scaling-group.md), [Creating a launch template for an Auto Scaling group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/create-launch-template.html), and [Permissions](https://docs.aws.amazon.com/autoscaling/ec2/userguide/launch-templates.html#launch-templates-permissions) in the *Amazon EC2 Auto Scaling User Guide*.
## Instances in an Amazon EC2 Auto Scaling group are continuously provisioned and terminated before a revision can be deployed
In some cases, an error can prevent a successful deployment to newly provisioned instances in an Amazon EC2 Auto Scaling group. The results are no healthy instances and no successful deployments. Because the deployment can't run or be completed successfully, the instances are terminated soon after being created. The Amazon EC2 Auto Scaling group configuration then causes another batch of instances to be provisioned to try to meet the minimum healthy hosts requirement. This batch is also terminated, and the cycle continues.
Possible causes include:
+ Failed Amazon EC2 Auto Scaling group health checks.
+ An error in the application revision.
To work around this issue, follow these steps:
1. Manually create an EC2 instance that is not part of the Amazon EC2 Auto Scaling group. Tag the instance with a unique EC2 instance tag.
1. Add the new instance to the affected deployment group.
1. Deploy a new, error-free application revision to the deployment group.
This prompts the Amazon EC2 Auto Scaling group to deploy the application revision to future instances in the Amazon EC2 Auto Scaling group.
**Note**
After you confirm that deployments are successful, delete the instance you created to avoid ongoing charges to your AWS account.
## Terminating or rebooting an Amazon EC2 Auto Scaling instance might cause deployments to fail
If an EC2 instance is launched through Amazon EC2 Auto Scaling, and the instance is then terminated or rebooted, deployments to that instance might fail for the following reasons:
+ During an in-progress deployment, a scale-in event or any other termination event causes the instance to detach from the Amazon EC2 Auto Scaling group and then terminate. Because the deployment cannot be completed, it fails.
+ The instance is rebooted, but it takes more than five minutes for the instance to start. CodeDeploy treats this as a timeout. The service fails all current and future deployments to the instance.
To address this issue:
+ In general, make sure that all deployments are complete before the instance is terminated or rebooted. Make sure that all deployments start after the instance has started or been rebooted.
+ Deployments can fail if you specify a Windows Server base Amazon Machine Image (AMI) for an Amazon EC2 Auto Scaling configuration and use the EC2Config service to set the computer name of the instance. To fix this issue, in the Windows Server base AMI, on the **General** tab of **EC2 Service Properties**, clear **Set Computer Name**. After you clear this check box, this behavior is disabled for all new Windows Server Amazon EC2 Auto Scaling instances launched with that Windows Server base AMI. For Windows Server Amazon EC2 Auto Scaling instances on which this behavior enabled, you do not need to clear this check box. Simply redeploy failed deployments to those instances after they have been rebooted.
## Avoid associating multiple deployment groups with a single Amazon EC2 Auto Scaling group
As a best practice, you should associate only one deployment group with each Amazon EC2 Auto Scaling group.
This is because if Amazon EC2 Auto Scaling scales up an instance that has hooks associated with multiple deployment groups, it sends notifications for all of the hooks at once. This causes multiple deployments to each instance to start at the same time. When multiple deployments send commands to the CodeDeploy agent at the same time, the five-minute timeout between a lifecycle event and either the start of the deployment or the end of the previous lifecycle event might be reached. If this happens, the deployment fails, even if a deployment process is otherwise running as expected.
**Note**
The default timeout for a script in a lifecycle event is 30 minutes. You can change the timeout to a different value in the AppSpec file. For more information, see [Add an AppSpec file for an EC2/On-Premises deployment](application-revisions-appspec-file.md#add-appspec-file-server).
It's not possible to control the order in which deployments occur if more than one deployment attempts to run at the same time.
Finally, if deployment to any instance fails, Amazon EC2 Auto Scaling immediately terminates the instance. When that first instance shuts down, the other running deployments start to fail. Because CodeDeploy has a one-hour timeout for the CodeDeploy agent to respond to pending deployments, it can take up to 60 minutes for each instance to time out.
For more information about Amazon EC2 Auto Scaling, see [Under the hood: CodeDeploy and Auto Scaling integration](https://aws.amazon.com/blogs/devops/under-the-hood-aws-codedeploy-and-auto-scaling-integration/).
## EC2 instances in an Amazon EC2 Auto Scaling group fail to launch and receive the error "Heartbeat Timeout"
An Amazon EC2 Auto Scaling group might fail to launch new EC2 instances, generating a message similar to the following:
`Launching a new EC2 instance . Status Reason: Instance failed to complete user's Lifecycle Action: Lifecycle Action with token was abandoned: Heartbeat Timeout`.
This message usually indicates one of the following:
+ The maximum number of concurrent deployments associated with an AWS account was reached. For more information about deployment limits, see [CodeDeploy quotas](limits.md).
+ The Auto Scaling group tried to launch too many EC2 instances too quickly. The API calls to [RecordLifecycleActionHeartbeat](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_RecordLifecycleActionHeartbeat.html) or [CompleteLifecycleAction](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_CompleteLifecycleAction.html) for each new instance were throttled.
+ An application in CodeDeploy was deleted before its associated deployment groups were updated or deleted.
When you delete an application or deployment group, CodeDeploy attempts to clean up any Amazon EC2 Auto Scaling hooks associated with it, but some hooks might remain. If you run a command to delete a deployment group, the leftover hooks are returned in the output. However, if you run a command to delete an application, the leftover hooks do not appear in the output.
Therefore, as a best practice, you should delete all deployment groups associated with an application before you delete the application. You can use the command output to identify the lifecycle hooks that must be deleted manually.
If you receive a “Heartbeat Timeout” error message, you can determine if leftover lifecycle hooks are the cause and resolve the problem by doing the following:
1. Do one of the following:
+ Call the [delete-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/delete-deployment-group.html) command to delete the deployment group associated with the Auto Scaling group that is causing the heartbeat timeout.
+ Call the [update-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/update-deployment-group.html) command with a non-null empty list of Auto Scaling group names to detach all CodeDeploy-managed Auto Scaling lifecycle hooks.
For example, enter the following AWS CLI command:
`aws deploy update-deployment-group --application-name my-example-app --current-deployment-group-name my-deployment-group --auto-scaling-groups`
As another example, if you are using the CodeDeploy API with Java, call `UpdateDeploymentGroup` and set `autoScalingGroups` to `new ArrayList()`. This sets `autoScalingGroups` to an empty list and removes the existing list. Do not use `null`, which is the default, because this leaves `autoScalingGroups` as-is, which is not what you want.
Examine the output of the call. If the output contains a `hooksNotCleanedUp` structure with a list of Amazon EC2 Auto Scaling lifecycle hooks, there are leftover lifecycle hooks.
1. Call the [describe-lifecycle-hooks](https://docs.aws.amazon.com/cli/latest/reference/autoscaling/describe-lifecycle-hooks.html) command, specifying the name of the Amazon EC2 Auto Scaling group associated with the EC2 instances that failed to launch. In the output, look for any of the following:
+ Amazon EC2 Auto Scaling lifecycle hook names that correspond to the `hooksNotCleanedUp` structure you identified in step 1.
+ Amazon EC2 Auto Scaling lifecycle hook names that contain the name of the deployment group associated with the Auto Scaling group that's failing.
+ Amazon EC2 Auto Scaling lifecycle hook names that may have caused the heartbeat timeout for the CodeDeploy deployment.
1. If a hook falls into one of the categories listed in step 2, call the [delete-lifecycle-hook](https://docs.aws.amazon.com/cli/latest/reference/autoscaling/delete-lifecycle-hook.html) command to delete it. Specify the Amazon EC2 Auto Scaling group and lifecycle hook in the call.
**Important**
Only delete hooks that are causing problems, as outlined in step 2. If you delete viable hooks, your deployments might fail, or CodeDeploy might not be able to deploy your application revisions to scaled out EC2 instances.
1. Call either the [update-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/update-deployment-group.html) or [create-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/create-deployment-group.html) command with the desired Auto Scaling group names. CodeDeploy re-installs the Auto Scaling hooks with new UUIDs.
**Note**
If you detach an Auto Scaling group from a CodeDeploy deployment group, any in-progress deployments to the Auto Scaling group may fail, and new EC2 instances that are scaled out by the Auto Scaling group will not receive your application revisions from CodeDeploy. To get Auto Scaling working again with CodeDeploy, you'll need to re-attach the Auto Scaling group to the deployment group and call a new `CreateDeployment` to start a fleet-wide deployment.
## Mismatched Amazon EC2 Auto Scaling lifecycle hooks might cause automatic deployments to Amazon EC2 Auto Scaling groups to stop or fail
Amazon EC2 Auto Scaling and CodeDeploy use lifecycle hooks to determine which application revisions should be deployed to which EC2 instances after they are launched in Amazon EC2 Auto Scaling groups. Automatic deployments can stop or fail if lifecycle hooks and information about these hooks do not match exactly in Amazon EC2 Auto Scaling and CodeDeploy.
If deployments to an Amazon EC2 Auto Scaling group are failing, see if the lifecycle hook names in Amazon EC2 Auto Scaling and CodeDeploy match. If not, use these AWS CLI command calls.
First, get the list of lifecycle hook names for both the Amazon EC2 Auto Scaling group and the deployment group:
1. Call the [describe-lifecycle-hooks](https://docs.aws.amazon.com/cli/latest/reference/autoscaling/describe-lifecycle-hooks.html) command, specifying the name of the Amazon EC2 Auto Scaling group associated with the deployment group in CodeDeploy. In the output, in the `LifecycleHooks` list, make a note of each `LifecycleHookName` value.
1. Call the [get-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/get-deployment-group.html) command, specifying the name of the deployment group associated with the Amazon EC2 Auto Scaling group. In the output, in the `autoScalingGroups` list, find each item whose name value matches the Amazon EC2 Auto Scaling group name, and then make a note of the corresponding `hook` value.
Now compare the two sets of lifecycle hook names. If they match exactly, character for character, then this is not the issue. You may want to try other Amazon EC2 Auto Scaling troubleshooting steps described elsewhere in this section.
However, if the two sets of lifecycle hook names do not match exactly, character for character, do the following:
1. If there are lifecycle hook names in the **describe-lifecycle-hooks** command output that are not also in the **get-deployment-group** command output, then do the following:
1. For each lifecycle hook name in the **describe-lifecycle-hooks** command output, call the [delete-lifecycle-hook](https://docs.aws.amazon.com/cli/latest/reference/autoscaling/delete-lifecycle-hook.html) command.
1. Call the [update-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/update-deployment-group.html) command, specifying the name of the original Amazon EC2 Auto Scaling group. CodeDeploy creates new, replacement lifecycle hooks in the Amazon EC2 Auto Scaling group and associates the lifecycle hooks with the deployment group. Automatic deployments should now resume as new instances are added to the Amazon EC2 Auto Scaling group.
1. If there are lifecycle hook names in the **get-deployment-group** command output that are not also in the **describe-lifecycle-hooks** command output, do the following:
1. Call the [update-deployment-group](https://docs.aws.amazon.com/cli/latest/reference/deploy/update-deployment-group.html) command, but do not specify the name of the original Amazon EC2 Auto Scaling group.
1. Call the **update-deployment-group** command again, but this time specify the name of the original Amazon EC2 Auto Scaling group. CodeDeploy re-creates the missing lifecycle hooks in the Amazon EC2 Auto Scaling group. Automatic deployments should now resume as new instances are added to the Amazon EC2 Auto Scaling group.
After you get the two sets of lifecycle hook names to match exactly, character for character, application revisions should be deployed again, but only to new instances as they are added to the Amazon EC2 Auto Scaling group. Deployments do not occur automatically to instances that are already in the Amazon EC2 Auto Scaling group.
## "The deployment failed because no instances were found for your deployment group" error
Read this section if you see the following CodeDeploy error:
`The deployment failed because no instances were found for your deployment group. Check your deployment group settings to make sure the tags for your EC2 instances or Auto Scaling groups correctly identify the instances you want to deploy to, and then try again.`
Possible causes for this error are:
1. Your deployment group settings include tags for EC2 instances, on-premises instances, or Auto Scaling groups that are not correct. To fix this problem, check that your tags are correct, and then redeploy your application.
1. Your fleet scaled out after the deployment started. In this scenario, you see healthy instances in the `InService` state in your fleet, but you also see the error above. To fix this problem, redeploy your application.
1. Your Auto Scaling group does not include any instances that are in the `InService` state. In this scenario, when you try to do a fleet-wide deployment, the deployment fails with the error message above because CodeDeploy needs at least one instance to be in the `InService` state. There are many reasons why you might have no instances in the `InService` state. A few of them include:
+ You scheduled (or manually configured) the Auto Scaling group size to be `0`.
+ Auto Scaling detected bad EC2 instances (for example, the EC2 instances had hardware failures), so canceled them all, leaving none in the `InService` state.
+ During a scale out event from `0` to `1`, CodeDeploy deployed a previously successful revision (called a *last successful revision*) that had become unhealthy since the last deployment. This caused the deployment on the scaled-out instance to fail, which in turn caused Auto Scaling to cancel the instance, leaving no instances in the `InService` state.
If you find that you have no instances in the `InService` state, troubleshoot the problem as described in the following procedure, [To troubleshoot the error if there are no instances in the InService state](#ToTroubleshootIfNoInServiceInstances).
**To troubleshoot the error if there are no instances in the InService state**
1. In the Amazon EC2 console, verify the **Desired Capacity** setting. If it is zero, set it to a positive number. Wait for the instance to be `InService`, which means the deployment succeeded. You have fixed the problem and can skip the remaining steps of this troubleshooting procedure. For information on setting the **Desired Capacity** setting, see [Setting capacity limits on your Auto Scaling group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-capacity-limits.html) in the *Amazon EC2 Auto Scaling User Guide*.
1. If Auto Scaling keeps attempting to launch new EC2 instances to meet the desired capacity but can never fulfill the scale out, it is usually due to a failing Auto Scaling lifecycle hook. Troubleshoot this problem as follows:
1. To check which Auto Scaling lifecycle hook event is failing, see [Verifying a scaling activity for an Auto Scaling group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-verify-scaling-activity.html) in the Amazon EC2 Auto Scaling User Guide.
1. If the failing hook's name is `CodeDeploy-managed-automatic-launch-deployment-hook-DEPLOYMENT_GROUP_NAME`, go to CodeDeploy, find the deployment group, and find the failed deployment that was started by Auto Scaling. Then investigate why the deployment failed.
1. If you understand why the deployment failed (for example, CloudWatch alarms were occurring) and you can fix the problem without changing the revision, then do so now.
1. If, after investigation, you determine that CodeDeploy’s *last successful revision* is no longer healthy, and there are zero healthy instances in your Auto Scaling group, you are in a deployment deadlock scenario. To solve this issue, you must fix the bad CodeDeploy revision by temporarily removing CodeDeploy’s lifecycle hook from the Auto Scaling group, and then reinstalling the hook and redeploying a new (good) revision. For instructions, see:
+ [To fix the deployment deadlock issue (CLI)](#ToFixDeployDeadlockCLI)
+ [To fix the deployment deadlock issue (console)](#ToFixDeployDeadlockConsole)
**To fix the deployment deadlock issue (CLI)**
1. (Optional) Block your CI/CD pipelines that are causing the CodeDeploy error so that unexpected deployments do not occur while you’re fixing this problem.
1. Take note of your current Auto Scaling **DesiredCapacity** setting:
`aws autoscaling describe-auto-scaling-groups --auto-scaling-group-name ASG_NAME`
You may need to scale back to this number at the end of this procedure.
1. Set your Auto Scaling **DesiredCapacity** setting to `1`. This is optional if your desired capacity was greater than `1` to begin with. By decreasing it to `1`, the instance will take less time to provision and deploy later, which speeds up troubleshooting. If your Auto Scaling desired capacity was originally set to `0`, you must increase it to `1`. This is mandatory.
aws autoscaling set-desired-capacity --auto-scaling-group-name *ASG\$1NAME* --desired-capacity 1
**Note**
The remaining steps of this procedure assume you have set your **DesiredCapacity** to `1`.
At this point, Auto Scaling attempts to scale to one instance. Then, because the hook that CodeDeploy added is still present, CodeDeploy tries to deploy; the deployment fails; Auto Scaling cancels the instance; and Auto Scaling tries to re-launch an instance to reach desired capacity of one, which again fails. You are in a cancel-relaunch loop.
1. De-register the Auto Scaling group from the deployment group:
**Warning**
The following command will launch a new EC2 instance with no software on it. Before running the command, make sure that an Auto Scaling `InService` instance running no software is acceptable. For example, make sure the load balancer associated with the instance is not sending traffic to this host without software.
**Important**
Use the CodeDeploy command shown below to remove the hook. Do not remove the hook through the Auto Scaling service, because the removal will not be recognized by CodeDeploy.
`aws deploy update-deployment-group --application-name APPLICATION_NAME --current-deployment-group-name DEPLOYMENT_GROUP_NAME --auto-scaling-groups`
After running this command, the following occurs:
1. CodeDeploy de-registers the Auto Scaling group from the deployment group.
1. CodeDeploy removes the Auto Scaling lifecycle hook from the Auto Scaling group.
1. Since the hook that was causing a failed deployment is no longer present, Auto Scaling cancels the existing EC2 instance and immediately launches a new one to scale to the desired capacity. The new instance should soon move into `InService` state. The new instance does not include software.
1. Wait for the EC2 instance to enter the `InService` state. To verify its state, use the following command:
`aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names ASG_NAME --query AutoScalingGroups[0].Instances[*].LifecycleState`
1. Add the hook back to the EC2 instance:
**Important**
Use the CodeDeploy command shown below to add the hook. Do not use the Auto Scaling service to add the hook, because the addition will not be recognized by CodeDeploy.
`aws deploy update-deployment-group --application-name APPLICATION_NAME --current-deployment-group-name DEPLOYMENT_GROUP_NAME --auto-scaling-groups ASG_NAME`
After running this command, the following occurs:
1. CodeDeploy re-installs the Auto Scaling lifecycle hook to the EC2 instance
1. CodeDeploy reregisters the Auto Scaling group with the deployment group.
1. Create a fleet-wide deployment with the Amazon S3 or GitHub revision that you know is healthy and want to use.
For example, if the revision is a .zip file in an Amazon S3 bucket called `my-revision-bucket` with an object key of `httpd_app.zip`, enter the following command:
`aws deploy create-deployment --application-name APPLICATION_NAME --deployment-group-name DEPLOYMENT_GROUP_NAME --revision "revisionType=S3,s3Location={bucket=my-revision-bucket,bundleType=zip,key=httpd_app.zip}"`
Since there is now one `InService` instance in the Auto Scaling group, this deployment should work, and you should no longer see the error *The deployment failed because no instances were found for your deployment group*.
1. After the deployment succeeds, scale your Auto Scaling group back out to the original capacity, if you previously scaled it in:
`aws autoscaling set-desired-capacity --auto-scaling-group-name ASG_NAME --desired-capacity ORIGINAL_CAPACITY`
**To fix the deployment deadlock issue (console)**
1. (Optional) Block your CI/CD pipelines that are causing the CodeDeploy error so that unexpected deployments do not occur while you’re fixing this problem.
1. Go to the Amazon EC2 console, and take note of your Auto Scaling **Desired capacity** setting. You may need to scale back to this number at the end of this procedure. For information on finding this setting, see [Setting capacity limits on your Auto Scaling group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-capacity-limits.html).
1. Set the desired number of EC2 instances to `1`:
This is optional if your desired capacity was greater than `1` to begin with. By decreasing it to `1`, the instance will take less time to provision and deploy later, which speeds up troubleshooting. If your Auto Scaling **Desired capacity** was originally set to `0`, you must increase it to `1`. This is mandatory.
**Note**
The remaining steps of this procedure assume you have set your **Desired capacity** to `1`.
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Choose the appropriate Region.
1. Go to the problematic Auto Scaling group.
1. In **Group details**, choose **Edit**.
1. Set **Desired capacity** to **1**.
1. Choose **Update**.
1. De-register the Auto Scaling group from the deployment group:
**Warning**
The following sub-steps will launch a new EC2 instance with no software on it. Before running the command, make sure that an Auto Scaling `InService` instance running no software is acceptable. For example, make sure the load balancer associated with the instance is not sending traffic to this host without software.
1. Open the CodeDeploy console at [https://console.aws.amazon.com/codedeploy/](https://console.aws.amazon.com/codedeploy/).
1. Choose the appropriate Region.
1. In the navigation pane, choose **Applications**.
1. Choose the name of your CodeDeploy application.
1. Choose the name of your CodeDeploy deployment group.
1. Choose **Edit**.
1. In **Environment configuration**, deselect **Amazon EC2 Auto Scaling groups**.
**Note**
The console does not allow you to save the configuration if there is no environment configuration defined. To bypass the check, temporarily add a tag of `EC2` or `On-premises` that you know won't resolve to any hosts. To add a tag, select **Amazon EC2 instances** or **On-premises instance**, and add a tag **Key** of **EC2** or **On-premises**. You can leave the tag **Value** empty.
1. Choose **Save changes**.
After completing these sub-steps, the following occurs:
1. CodeDeploy de-registers the Auto Scaling group from the deployment group.
1. CodeDeploy removes the Auto Scaling lifecycle hook from the Auto Scaling group.
1. Since the hook that was causing a failed deployment is no longer present, Auto Scaling cancels the existing EC2 instance and immediately launches a new one to scale to the desired capacity. The new instance should soon move into `InService` state. The new instance does not include software.
1. Wait for the EC2 instance to enter the `InService` state. To verify its state:
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/).
1. In the navigation pane, choose **Auto Scaling Groups**.
1. Choose your Auto Scaling group.
1. In the content pane, choose the **Instance Management** tab.
1. Under **Instances**, make sure that the **Lifecycle** column indicates **InService** next to the instance.
1. Re-register the Auto Scaling group with the CodeDeploy deployment group using the same method you used to remove it:
1. Open the CodeDeploy console at [https://console.aws.amazon.com/codedeploy/](https://console.aws.amazon.com/codedeploy/).
1. Choose the appropriate Region.
1. In the navigation pane, choose **Applications**.
1. Choose the name of your CodeDeploy application.
1. Choose the name of your CodeDeploy deployment group.
1. Choose **Edit**.
1. In **Environment configuration**, select **Amazon EC2 Auto Scaling groups** and select your Auto Scaling group from the list.
1. Under **Amazon EC2 instances** or **On-premises instances**, find the tag you added and remove it.
1. Deselect the check box next to **Amazon EC2 instances** or **On-premises instances**.
1. Choose **Save changes**.
This configuration re-installs the lifecycle hook into the Auto Scaling group.
1. Create a fleet-wide deployment with the Amazon S3 or GitHub revision that you know is healthy and want to use.
For example, if the revision is a .zip file in an Amazon S3 bucket called `my-revision-bucket` with an object key of `httpd_app.zip`, do the following:
1. In the CodeDeploy console, in the **Deployment Group** page, choose **Create deployment**.
1. For **Revision type**, choose **My application is stored in Amazon S3**.
1. For **Revision location**, choose `s3://my-revision-bucket/httpd_app.zip`.
1. For **Revision file type**, choose `.zip`.
1. Choose **Create deployment**.
Since there is now one `InService` instance in the Auto Scaling group, this deployment should work, and you should no longer see the error *The deployment failed because no instances were found for your deployment group*.
1. After the deployment succeeds, scale your Auto Scaling group back out to the original capacity, if you previously scaled it in:
1. Open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/), and choose **Auto Scaling Groups** from the navigation pane.
1. Choose the appropriate Region.
1. Go to your Auto Scaling group.
1. In **Group details**, choose **Edit**.
1. Set **Desired capacity** back to its original value.
1. Choose **Update**.
# Error codes for AWS CodeDeploy
This topic provides reference information about CodeDeploy errors.
****
| Error Code | Description |
| --- | --- |
| `AGENT_ISSUE` | The deployment failed because of a problem with the CodeDeploy agent. Make sure the agent is installed and running on all instances in this deployment group. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `AUTO_SCALING_IAM_ROLE_PERMISSIONS` | The service role associated with your deployment group does not have the permission required to perform operations in the following AWS service. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `HEALTH_CONSTRAINTS` | The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `HEALTH_CONSTRAINTS_INVALID` | The deployment can’t start because the minimum number of healthy instances, as defined by your deployment configuration, are not available. You can reduce the required number of healthy instances by updating your deployment configuration or increase the number of instances in this deployment group. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `IAM_ROLE_MISSING` | The deployment failed because no service role exists with the service role name specified for the deployment group. Make sure you are using the correct service role name. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `IAM_ROLE_PERMISSIONS` | CodeDeploy does not have the permissions required to assume a role, or the IAM role you're using does't give you permission to perform operations in an AWS service. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `NO_INSTANCES` | This might be caused by one of the following. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `OVER_MAX_INSTANCES` | The deployment failed because more instances are targeted for deployment than are allowed for your account. To reduce the number of instances targeted for this deployment, update the tag settings for this deployment group or delete some of the targeted instances. Alternatively, you can contact AWS Support to request a limit increase. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `THROTTLED` | The deployment failed because more requests were made than are permitted for AWS CodeDeploy by an IAM role. Try reducing the number of requests. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
| `UNABLE_TO_SEND_ASG` | The deployment failed because the deployment group isn’t configured correctly with its Amazon EC2 Auto Scaling group. In the CodeDeploy console, delete the Amazon EC2 Auto Scaling group from the deployment group, and then add it again. Learn more: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/codedeploy/latest/userguide/error-codes.html) |
## Related topics
[Troubleshooting CodeDeploy](troubleshooting.md)