View a markdown version of this page

Use Amazon MSK storage capacity alerts - Amazon Managed Streaming for Apache Kafka
Monitor storage capacity alertsDisk-full protection for Standard brokers

Use Amazon MSK storage capacity alerts

On Amazon MSK provisioned clusters, you choose the cluster's primary storage capacity. If you exhaust the storage capacity on a broker in your provisioned cluster, it can affect its ability to produce and consume data, leading to costly downtime. Amazon MSK offers CloudWatch metrics to help you monitor your cluster's storage capacity. However, to make it easier for you to detect and resolve storage capacity issues, Amazon MSK automatically sends you dynamic cluster storage capacity alerts. The storage capacity alerts include recommendations for short-term and long-term steps to manage your cluster's storage capacity. From the Amazon MSK console, you can use quick links within the alerts to take recommended actions immediately.

There are two types of MSK storage capacity alerts: proactive and remedial.

  • Proactive ("Action required") storage capacity alerts warn you about potential storage issues with your cluster. When a broker in an MSK cluster has used over 60% or 80% of its disk storage capacity, you'll receive proactive alerts for the affected broker.

  • Remedial ("Critical action required") storage capacity alerts require you to take remedial action to fix a critical cluster issue when one of the brokers in your MSK cluster has run out of disk storage capacity.

Amazon MSK automatically sends these alerts to the Amazon MSK console, AWS Health Dashboard, Amazon EventBridge, and email contacts for your AWS account. You can also configure Amazon EventBridge to deliver these alerts to Slack or to tools such as New Relic, and Datadog.

Storage capacity alerts are enabled by default for all MSK provisioned clusters and can't be turned off. This feature is supported in all regions where MSK is available.

Monitor storage capacity alerts

You can check for storage capacity alerts in several ways:

Disk-full protection for Standard brokers

To keep a broker from running out of storage, Amazon MSK protects Standard brokers when their data disk usage reaches 90%. When a broker reaches this threshold, Amazon MSK does the following:

  • Stops accepting new produce requests and replication traffic on the broker.

  • Moves partition leadership from the broker to other in-sync replicas, so that producers can continue writing to those partitions through the new leaders. Consumers can continue reading.

  • Resumes normal operation on the broker automatically after its data disk usage stays below 90% for 5 minutes.

If all in-sync replicas of a partition are on protected brokers, writes to that partition fail until the broker's disk usage drops below 90%. To avoid this, act on the 60% and 80% storage capacity alerts. You can do any of the following:

If a broker is already protected, use the same options to bring its disk usage below 90%. The broker resumes normal operation automatically after its disk usage stays below 90% for 5 minutes.

Disk-full protection is on by default, and you can't turn it off. You get disk-full protection on Standard brokers that run one of the following Apache Kafka versions:

Apache Kafka versionMetadata mode
3.6.0ZooKeeper
3.8.xZooKeeper
3.9.xZooKeeper
3.9.x.kraftKRaft
4.0.x.kraftKRaft
4.1.x.kraftKRaft
4.2.x.kraftKRaft

Express brokers don't need disk-full protection because they don't use provisioned storage.