Configuring Amazon MWAA web server automatic scaling

For environments running Apache Airflow Apache Airflow v2.2.2 and above, Amazon MWAA dynamically scales your web servers to handle fluctuating workloads, which in turn prevents performance issues during peak loads. By automatically scaling the number of web servers based on CPU utilization and active connection count, Amazon MWAA ensures that your Apache Airflow environment can seamlessly accommodate increased demand, whether from REST API requests, CLI usage, or more concurrent Apache Airflow user interface users.

How web server scaling works

Amazon MWAA uses the container metric, CPUUtilization, and the load balancer metric, ActiveConnectionCount, to determine if scaling the web servers is required based on the amount of traffic. If CPUUtilization is higher than 70 or ActiveConnectionCount is higher than 15, Amazon MWAA will add additional Fargate web server containers up to the maximum value specified by MaxWebservers.

As traffic decreases and the CPUUtilization and ActiveConnectionCount values reduce, Amazon MWAA requests Fargate to scale down the web server containers for the environment to the minimum value set by MinimumWebservers.

Using the Amazon MWAA console

You can choose the number of web servers that can run on your environment concurrently on the Amazon MWAA console. By default, the minimum number of web servers is two, and the maximum number of web servers is five.