Set the default instance warmup for an Auto Scaling group
CloudWatch collects and aggregates usage data, such as CPU and network I/O, across your Auto Scaling instances. You use these metrics to create scaling policies that adjust the number of instances in your Auto Scaling group as the selected metric's value increases and decreases.
You can specify how long after an instance reaches the InService state it
waits before contributing usage data to the aggregated metrics. This specified time is
called the default instance warmup. This keeps
dynamic scaling from being affected by metrics for individual instances that aren't yet
handling application traffic and that might be experiencing temporarily high usage of
compute resources.
To optimize the performance of your target tracking and step scaling policies, we strongly recommend that you enable and configure the default instance warmup. It is not enabled or configured by default.
When you enable the default instance warmup, keep in mind that if your Auto Scaling group is set to use an instance maintenance policy, or you use an instance refresh to replace instances, you can prevent instances from being counted toward the minimum healthy percentage before they have finished initializing.
Contents
Scaling performance considerations
It's useful for most applications to have one default instance warmup time that applies to all features, rather than different warmup times for different features. For example, if you don't set a default instance warmup, the instance refresh feature uses the health check grace period as the default warmup time. If you have any target tracking and step scaling policies, they use the value set for the default cooldown as the default warmup time. If you have any predictive scaling policies, they have no default warmup time.
While instances are warming up, your dynamic scaling policies scale out only if the metric value from instances that are not warming up is greater than the policy's alarm high threshold (or the target utilization of a target tracking scaling policy). If demand decreases, dynamic scaling becomes more conservative to protect your application's availability. This blocks the scale in activities for dynamic scaling until the new instances finish warming up.
While scaling out, Amazon EC2 Auto Scaling considers instances that are warming up as part of the capacity of the group when deciding how many instances to add to the group. Therefore, multiple alarm breaches that require a similar amount of capacity to be added result in a single scaling activity. The intention is to continuously scale out, without doing so excessively.
If default instance warmup is not enabled, the amount of time an instance waits before sending metrics to CloudWatch and counting it towards the current capacity will vary from instance to instance. So, there is the potential for your scaling policies to perform unpredictably compared to the actual workload that is occurring.
For example, consider an application with a recurring on-and-off workload pattern. A predictive scaling policy is used to make recurring decisions about whether to increase the number of instances. Because there is no default warmup time for predictive scaling policies, the instances start contributing to the aggregated metrics immediately. If these instances have higher resource usage on startup, then adding instances could cause the aggregated metrics to spike. Depending on how long it takes for usage to stabilize, this could impact any dynamic scaling policies that use these metrics. If a dynamic scaling policy's alarm high threshold is breached, then the group increases in size again. While the new instances are warming up, scale in activities will be blocked.
Choose the default instance warmup time
The key to setting the default instance warmup is determining how long your
instances need to finish initializing and for resource consumption to stabilize
after they reach the InService state. When choosing the instance warmup
time, try to keep an optimal balance between collecting usage data for legitimate
traffic, and minimizing data collection associated with temporary usage spikes on
startup.
Suppose you have an Auto Scaling group attached to an Elastic Load Balancing load balancer. When new
instances finish launching, they're registered to the load balancer before they
enter the InService state. After the instances enter the
InService state, resource consumption can still experience
temporary spikes and need time to stabilize. For example, resource consumption for
an application server that must download and cache large assets takes longer to
stabilize than a lightweight web server with no large assets to download. The
instance warmup provides the time delay necessary for resource consumption to
stabilize.
Important
If you're not sure how much time you need for the warmup time, you could start
with 300 seconds. Then gradually decrease or increase it until you get the best
scaling performance for your application. You might need to do this a few times
to get it right. Alternatively, if you have any scaling policies that have their
own warmup time (EstimatedInstanceWarmup), you could use this value
to start. For more information, see Find scaling
policies with a previously set instance warmup time.
Consider using lifecycle hooks for use cases where you have configuration tasks or scripts to run on startup. Lifecycle hooks can delay new instances from being put in service until they have finished initializing. They are particularly useful if you have bootstrapping scripts that take a while to complete. If you add a lifecycle hook, you can reduce the value of the default instance warmup. For more information about using lifecycle hooks, see Amazon EC2 Auto Scaling lifecycle hooks.
