Application status checks
Application status checks help you monitor the performance and health of your applications running on Amazon EC2. With application status checks, you can detect and respond to application health impairments by monitoring your applications through configurable paths and ports. For example, you can use application status checks to confirm that your web server is listening on its expected port and accepting new connections.
Application status checks monitor the HTTP and HTTPS responses of your applications at configurable paths and ports. They run every 60 seconds and integrate with Amazon EC2 Auto Scaling, so you can automate replacement of instances whose applications are impaired.
Contents
How application status checks work
Application status checks send HTTP or HTTPS requests to an endpoint listening at a network port on your instance every 60 seconds. AWS compares the response code against the status code matcher you configured. The check is marked impaired after a number of consecutive failed requests, and healthy again after a number of consecutive successful requests. Both counts default to 2 and are configurable. For more information, see Evaluation thresholds.
Note
Application status checks send the health check request over HTTP/2.
The HTTPS protocol check does not validate the server certificate.
During a reboot, application status checks report a failure until the instance becomes available again because the application cannot respond to health check requests while the operating system is restarting.
Network architecture
Application status checks originate from the Amazon EC2 application status checks service. To reach your instances, AWS creates a managed elastic network interface (ENI) in your VPC. AWS creates one ENI per combination of source subnet and security group that has associated instances. AWS creates the managed ENI when an application status check first requires that combination, and removes it when no remaining application status check requires it. The managed ENI does not count against your instance ENI limit, but does count against your global limit for ENIs per VPC.
Application status checks reach your instances from a private vantage point within your VPC. The scope describes where the check originates, not a property of your instance's IP address. AWS creates the managed ENI in a subnet within your VPC and reaches the instance over the private network path.
Health check traffic originates from AWS-managed Amazon EC2 instances in the
same Availability Zone as the target instance (or the parent Availability
Zone for Local Zone targets), travels over the AWS internal network, and
does not traverse the public internet. For more information, see Amazon VPC FAQs
AWS managed and customer-managed network paths
Application status checks support two onboarding modes that determine who selects the source subnets and security groups for the health check ENI and the destination subnets and security groups for the target instances.
- AWS managed network paths
-
AWS selects the source subnets and security groups for the health check ENI and the destination subnets and security groups for the target instances.
- Customer-managed network paths
-
You specify the source subnets and security groups for the health check ENI and the destination subnets and security groups for the target instances.
Use customer-managed network paths when you need to control which subnets and security groups health check traffic originates from, such as when your VPC has strict network segmentation, firewall rules, or compliance requirements that restrict which sources can reach your application endpoints.
You choose the mode by including or omitting the --health-check-paths parameter in the create command. If you
omit the --health-check-paths parameter, AWS selects source
and destination subnets and security groups (AWS managed network paths).
If you include the --health-check-paths parameter, you manage
them (customer-managed network paths).
IP version
Each application status check is associated with a single IP version (IPv4 or IPv6). To monitor an instance over both IPv4 and IPv6, create two separate application status checks and associate both with the instance.
Checks reach your instance from within your VPC for both IPv4 and IPv6.
Check status values
Each individual check reports one of the following statuses:
-
passed: the check completed successfully -
failed: the check failed. The response includes the HTTP status code returned by your application. For interpretation and remediation guidance, see Troubleshooting. -
initializing: the check has not yet completed its first evaluation -
insufficient-data: the check did not receive enough data to determine a result -
not-applicable: the check is not associated with the instance
The overall application status reported for the instance aggregates all individual check results. The overall status is one of the following:
-
ok: all checks passed -
impaired: one or more checks failed -
initializing: one or more checks have not yet completed their first evaluation -
insufficient-data: one or more checks report insufficient data -
not-applicable: all associated application status checks are excluded from aggregation -
suppressed: application status check evaluation is suppressed for the instance
Aggregation
You can mark each application status check as included in or excluded from the overall status for the instance. By default, a check is
included.
included-
The check contributes to the overall status for the instance and Amazon EC2 Auto Scaling uses it.
excluded-
The check reports its individual status but does not contribute to the overall status for the instance and Amazon EC2 Auto Scaling does not use it. Use this setting to validate a new check in production without affecting the overall status or triggering Amazon EC2 Auto Scaling replacements. This is the recommended workflow when adding a check to an existing production workload; see Testing a new application status check.
Get started with application status checks
Prerequisites
Before you create an application status check, make sure you have the following:
-
A VPC with the instances you want to monitor.
-
An application endpoint on each instance that can respond to HTTP or HTTPS requests on the port and HTTP path you will configure.
-
A security group on each destination instance that allows inbound traffic on the check port from the source security group used by the application status check. See Security and permissions.
Step 1: Configure your application
Configure your application endpoint to respond to HTTP or HTTPS requests on the port and HTTP path you will specify when you create the check. Return a response code included in your status code matcher to indicate the application is healthy.
Make sure the destination instance's security group allows inbound traffic on the check port from the source security group used by the application status check. For managed network paths, AWS provides the source security group at check creation. For customer-managed network paths, you specify the source security group when you create the check.
Step 2: Create a check definition
Use the AWS CLI to create an application status check.
Step 3: Associate the check with instances
Associate the check with the instances you want to monitor, either by instance ID or by tag.
Associate and disassociate operations return per-instance success and failure results. If some instances cannot be associated (for example, because the check is already associated), those instances appear in the unsuccessful results with a reason.
Step 4: View results
View the per-instance application health status.
Configuration options
Application status checks accept several configuration parameters. This section explains the parameters where the behavior isn't self-evident from the parameter name. For the complete list of parameters and validation rules, see CreateApplicationStatusCheck and AssociateApplicationStatusCheck in the Amazon EC2 API Reference.
Evaluation thresholds
FailureThreshold-
The number of consecutive failed requests before the check is marked impaired. Default: 2.
SuccessThreshold-
The number of consecutive successful requests before the check is marked healthy again. Default: 2.
Timeout-
The number of seconds to wait for a response before the request is recorded as failed. Enforced as a forced timeout; if your application does not respond within this window, the request is recorded as a failure regardless of eventual response. Default: 6. Valid range: 1-30.
Startup grace period
InitializationGracePeriodSeconds-
The number of seconds to wait after an instance launches before AWS starts evaluating the check. Use this parameter to give applications time to start listening before checks begin. If the grace period is too short, Amazon EC2 Auto Scaling might replace new instances before their application is ready. Default: 300. Valid range: 1 to 600.
IP scope
IpScope-
Application status checks use
privatescope; the check runs from within your VPC. For IPv4, this corresponds to the instance's private IP address. For IPv6, AWS does not classify the address as public or private; the check accepts any IPv6 address and evaluates it from within your VPC.
Device index
DeviceIndex-
The index of the network device on your instance that AWS evaluates for the health check. Change this when your instance's primary network device is not the one you want checked. Default: 0.
Aggregation, IP version, and health check paths (source and destination subnets and security groups) are covered in their own sections earlier on this page.
Default settings
With AWS managed network paths, application status checks use the following defaults.
| Setting | Default |
|---|---|
Check interval |
60 seconds (fixed; not configurable) |
Failure threshold |
2 consecutive failures |
Success threshold |
2 consecutive successes |
Timeout |
6 seconds |
Status code matcher |
200 |
HTTP path |
/ |
IP version |
ipv4 |
IP scope |
private |
Device index |
0 |
Initialization grace period |
300 seconds |
Aggregation |
included |
Source subnets and security groups |
Managed by AWS |
Amazon EC2 Auto Scaling integration
Amazon EC2 Auto Scaling automatically terminates and replaces instances whose overall
application status reports impaired, as long as the check is
included in aggregation. No Auto Scaling group configuration is required beyond
associating the application status check with the instances in the
group.
Amazon EC2 Auto Scaling uses the overall status for the instance, not individual check
status. Checks marked excluded do not drive Amazon EC2 Auto Scaling actions.
Checks in the suppressed state do not drive Amazon EC2 Auto Scaling
actions.
Use the InitializationGracePeriodSeconds parameter on the check
to allow new instances time to start up before application status checks
begin. If the grace period is too short, new instances might be terminated and
replaced by Amazon EC2 Auto Scaling before their application is ready to serve traffic.
For more information about how Amazon EC2 Auto Scaling uses health checks, see Health checks for instances in an Auto Scaling group and Use application status checks with an Auto Scaling group in the Amazon EC2 Auto Scaling User Guide.
Handling deployment, in-place patching, and replacements
Deployments, in-place patching, and other maintenance operations can temporarily stop or restart your application. During that time, application status checks report a failure because the application cannot respond to health check requests. If your instances are in an Auto Scaling group with application status checks included in aggregation, Amazon EC2 Auto Scaling might terminate and replace these instances even though the disruption is expected.
Option A: Suppress the check
Use suppression for bounded maintenance windows where you know the duration. Suppression is enforced at the instance level. You specify a duration, or omit it to suppress the check until you disable the suppression.
While suppressed, the overall application status for the instance
reports suppressed. Amazon EC2 Auto Scaling does not act on
suppressed instances.
Option B: Exclude the check from aggregation
If you want the check to keep evaluating and reporting its individual
status but not affect the overall status or trigger Amazon EC2 Auto Scaling actions, set
the check's aggregation setting to excluded. This is useful
for longer-lived scenarios such as rolling out a new check version or
validating a change without risking replacement, and for cases where
you want telemetry to continue without operational impact.
For more information, see Aggregation.
Option C: Disassociate the check
Use disassociation for longer-lived or indefinite removal.
aws ec2 disassociate-application-status-check \ --application-status-check-id asc-1234567890abcdef0 \ --instance-ids i-0123456789abcdef0
If you associated by tag, remove the tag from the instance to
disassociate. After disassociation, the overall application status for
the instance reports not-applicable.
Deployment guidance
Deployments are the most common maintenance scenario that requires suppression. Use suppression when your deployment tool has a pre-deployment hook and a post-deployment hook so that you can suppress the check before the deployment starts and disable suppression after the deployment completes.
The general pattern is:
-
In the pre-deployment hook, call enable-application-status-check-suppression for the instance, with a duration that covers the expected deployment window.
-
Perform the deployment.
-
In the post-deployment hook, call disable-application-status-check-suppression for the instance.
If your deployment tool does not have hooks, drive suppression from the CI/CD pipeline that invokes the deployment.
Testing a new application status check
You can validate a new application status check in production before it
starts contributing to your instance-level monitoring. Set the aggregation
setting to excluded when you create the check, then confirm it
reports the expected status and HTTP response codes. When you're ready,
change the setting to included so the check contributes to the
overall status for the instance and integrates with Amazon EC2 Auto Scaling.
-
Create the check with the aggregation setting set to
excluded.aws ec2 create-application-status-check \ --protocol https \ --port 443 \ --path "/health" \ --status-code-matcher "200" \ --aggregation excluded -
Associate the check with a test instance or a subset of your production fleet.
-
Wait for at least two check intervals (approximately two minutes) to allow the check to complete an initial evaluation.
-
Use describe-application-status to verify the check is reporting the expected status and HTTP response code.
aws ec2 describe-application-status \ --instance-ids i-0123456789abcdef0 -
If the check reports as expected, update the aggregation setting to
includedto make the check contribute to the overall status for the instance and drive Amazon EC2 Auto Scaling actions.aws ec2 modify-application-status-check \ --application-status-check-id asc-1234567890abcdef0 \ --aggregation included
Advanced networking
Application status checks originate from a managed ENI in the source subnet and security group you specify (or that AWS selects for you). For workloads that require higher availability than a single-source configuration provides, or for workloads that run in Local Zones or Outposts, consider the following patterns.
Local Zones
For instances running in AWS Local Zones, the managed elastic network interface (ENI) resides in the parent AWS Region, not in the Local Zone. Health check traffic between the parent Region and your Local Zone instances traverses the Local Zone service link, which may incur additional data transfer charges.
Troubleshooting
When an application status check reports impaired but you expect your application to be healthy, verify each of the following:
-
Instance reachability. Confirm that the instance's Instance and System status checks are
ok. -
Security group inbound rule. The destination instance's security group must allow inbound traffic on the check port from the source security group used by the application status check. For AWS managed network paths, AWS provides the source security group; for customer-managed network paths, use the security group you specified as the source.
-
Host firewall. Any host-level firewall (iptables, Windows Firewall, third-party host firewall) on the instance must allow inbound traffic on the check port.
-
Application endpoint. The application must be listening on the port and path you configured. Confirm with a local request from the instance (
curl http://localhost:PORT/PATH). -
Protocol mismatch. If the check is configured as HTTPS but the endpoint serves HTTP only (or vice versa), all calls will fail.
-
Status code matcher. Confirm that your application's actual response code is included in the status code matcher you configured.
-
Network path. If you configured customer-managed network paths, confirm the source subnet and security group have connectivity to the destination subnet. Use VPC Reachability Analyzer to trace the network path.
-
Available ENI quota. AWS creates a managed elastic network interface (ENI) in your account for each source subnet and security group combination. Confirm your account has an available ENI in its quota for the VPC. If your account has reached its ENIs per VPC quota, AWS cannot create the managed ENI and the check cannot run. For more information, see Amazon VPC quotas.
Reason codes
The describe-application-status response includes a reason for
each check. The reason contains the HTTP status code returned by your
application (as a number), along with the protocol used for the check.
A check is marked passed if the returned status code is
included in your status code matcher, and failed
otherwise.
The reason also includes a reason code and, for HTTP-level results, the protocol and the returned HTTP status code. The reason contains the following fields:
Code-
The reason code for the application status check result. One of the following values:
-
ResponseCodeMatched: the HTTP status code returned by the health check matched the configuredStatusCodeMatcher. -
ResponseCodeMismatch: the HTTP status code returned by the health check did not match the configuredStatusCodeMatcher. -
ConnectionTimeout: the connection to the target timed out. -
ResponseTimeout: the health check timed out while waiting for a response from the target. -
ConnectionRefused: the target refused the health check connection. -
ConnectionReset: the health check connection was reset before a response was received.
For
ResponseCodeMatchedandResponseCodeMismatch, theStatusCodefield contains the returned HTTP status code and theProtocolfield contains the protocol used for the health check. For connection errors, such asConnectionTimeout,ResponseTimeout,ConnectionRefused, andConnectionReset, theStatusCodeandProtocolfields are not present. -
Protocol-
The protocol used for the health check. One of
HTTPorHTTPS. StatusCode-
The HTTP status code returned by the health check.
Use the returned HTTP status code to identify why a check failed. Some common examples:
| HTTP status code | Typical meaning | Common remediation |
|---|---|---|
|
Application returned a successful response. |
None. This is typically a healthy status. |
|
Application returned a redirect. Health check calls do not follow redirects. |
Point the health check path at the destination of the redirect, or add the redirect code to your status code matcher if you consider it healthy. |
|
The application requires authentication or denied access to the health check path. |
Configure the health check path to be unauthenticated, or serve health checks on a path that does not require credentials. |
|
The configured health check path was not found on the application. |
Confirm the path matches a route your application serves. |
|
Application returned an internal server error. |
Investigate application logs on the instance. |
|
Application is reachable but reports upstream or capacity issues. |
Investigate application health, dependencies,
and capacity. If your application returns these
codes during startup, increase
|
For the complete ApplicationStatusReason structure, see
ApplicationStatusReason in the
Amazon EC2 API Reference.
Common mistakes
-
The security group does not allow inbound traffic from the health check source on the check port.
-
The application is bound to
127.0.0.1and not listening on the network interface. -
The health check path returns a redirect (301, 302) rather than a success response, and the status code matcher does not include the redirect code.
-
The check is configured for HTTPS but the application only serves HTTP, or vice versa.
-
The application takes longer to start than the
InitializationGracePeriodSecondsvalue, and Amazon EC2 Auto Scaling replaces the instance before it is ready.
Monitor application status checks
You can monitor application status checks in three ways:
-
Amazon CloudWatch. The
StatusCheckFailed_Applicationmetric reflects the overall application status for the instance and can drive alarms. The metric is aggregated per instance across associated checks whose aggregation setting isincluded. CloudWatch also publishes a per-check metric for each associated check, namedStatusCheckFailed_Application_{application-status-check-id}_{application-status-check-name}. -
describe-instance-status. Returns the overall application status alongside your instance's other status information.
-
describe-application-status. Returns detailed per-instance results, including each associated check's individual status and the HTTP status code returned by your application.
Use the CloudWatch metric for alarm-driven automation. Use
describe-instance-status when you already query it for
instance status. Use describe-application-status for detailed
per-check visibility.
Security and permissions
AWS creates and manages the network interfaces used for application status
checks through a service-linked role. No IAM setup is required for the
service to create these ENIs. The service-linked role uses the EC2ApplicationStatusChecksServiceRolePolicy AWS
managed policy.
To create, associate, describe, delete, and suppress application status checks yourself, your IAM user or role needs the corresponding Amazon EC2 permissions. See the Amazon EC2 API Reference for the full list of actions.
Your instance's security group must allow inbound traffic from the health check source security group on the port you configured. With AWS managed network paths, AWS provides the source security group; with customer-managed network paths, use the security group you specified as the source.
Pricing
Application status checks are billed on the following components:
-
An hourly charge of $0.01 for each managed elastic network interface (ENI), per Availability Zone.
-
Standard Amazon CloudWatch pricing applies to application status check metrics.
Quotas
Application status checks are subject to AWS service quotas. For the quota names, default values, and descriptions, see Amazon EC2 endpoints and quotas in the AWS General Reference.