Amazon MSK quota
Your AWS account has default quotas for Amazon MSK. Unless otherwise stated, each per-account quota is region-specific within your AWS account.
Amazon MSK Standard broker quota
| Dimension | Quota | Notes |
|---|---|---|
| Brokers per account | 90 | To request higher quota, go to the Service Quotas console |
| Brokers per cluster | 30 for ZooKeeper-based clusters 60 for KRaft-based clusters | To request higher quota, go to the Service Quotas console |
| Minimum storage per broker | 1 GiB | |
| Maximum storage per broker | 16384 GiB | |
| Maximum TCP connections per broker (IAM Access control) | 3000 | To increase this limit, you can adjust the listener.name.client_iam.max.connections or the listener.name.client_iam_public.max.connections configuration property using the Kafka AlterConfig API or the kafka-configs.sh tool. It's important to note that increasing either property to a high value can result in unavailability. |
| Maximum TCP connections rate per broker (IAM) | 100 per second (M5 and M7g instance sizes) 4 per second (t3 instance size) | To handle retries on failed connections, you can set the
reconnect.backoff.ms configuration parameter on the
client side. For example, if you want a client to retry connections
after 1 second, set reconnect.backoff.ms to
1000. For more information, see reconnect.backoff.ms |
| Maximum TCP connections per broker (non-IAM) | N/A | MSK does not enforce connection limits for non-IAM auth. You should monitor other metrics such as CPU and memory usage to ensure you do not overload your cluster because of excessive connections. |
| Configurations per account | 100 | To request higher quota, go to the Service Quotas console To update the configuration or the Apache Kafka version of an MSK cluster, first ensure the number of partitions per broker is under the limits described in Right-size your cluster: Number of partitions per Standard broker. |
| Configuration revisions per account | 50 |
Amazon MSK Express broker quota
| Dimension | Quota | Notes |
|---|---|---|
| Brokers per account | 90 | To request higher quota, go to the Service Quotas console |
| Brokers per cluster | 30 | To request higher quota, go to the Service Quotas console |
| Maximum storage | Unlimited | |
| Maximum TCP connections per broker (IAM Access control) | 3000 | |
| Maximum TCP connections rate per broker (IAM) | 100 per second | To handle retries on failed connections, you can set the
reconnect.backoff.ms configuration parameter on the client side. For
example, if you want a client to retry connections after 1 second,
set reconnect.backoff.ms to 1000. For more information, see reconnect.backoff.ms |
| Maximum TCP connections per broker (non-IAM) | N/A | MSK does not enforce connection limits for non-IAM auth. You should however monitor other metrics such as CPU and memory usage to ensure you do not overload your cluster because of excessive connections. |
| Configurations per account | 100 | To request higher quota, go to the Service Quotas console |
| Configuration revisions per account | 50 | |
| Max Ingress per broker | Recommended: 15.6 - 500.0 MBps | Based on instance size. |
| Max Egress per broker | Recommended: 31.2 - 1000.0 MBps | Based on instance size. |
Express broker throughput throttle limits by broker size
| Instance size | Ingress throttle limits per broker (MBps) | Egress throttle limits per broker (MBps) |
|---|---|---|
| express.m7g.large | 23.4 | 58.5 |
| express.m7g.4xlarge | 187.5 | 468.7 |
| express.m7g.16xlarge | 750.0 | 1875.0 |
MSK Replicator quotas
A maximum of 15 MSK Replicators per account.
-
MSK Replicator only replicates up to 750 topics in sorted order. If you need to replicate more topics, we recommend that you create a separate Replicator. Go to the Service Quotas console
, if you need support for more than 750 topics per Replicator. You can monitor the number of topics being replicated using the "TopicCount" metric. -
A maximum ingress throughput of 1GB per second per MSK Replicator. Request a higher quota by going through the Service Quotas console
. -
MSK Replicator Record Size - A maximum of 10MB record size (message.max.bytes). Request a higher quota by going through the Service Quotas console
.
MSK Serverless quota
Note
If you experience any issue with service quota limits, create a support case with your use case and requested limit.
| Dimension | Quota | Quota violation result |
|---|---|---|
|
Maximum ingress throughput |
200 MBps |
Slowdown with throttle duration in response |
| Maximum egress throughput | 400 MBps |
Slowdown with throttle duration in response |
| Maximum retention duration | Unlimited | N/A |
|
Maximum number of client connections |
3000 | Connection close |
| Maximum connection attempts | 100 per second | Connection close |
|
Maximum message size |
8 MB | Request fails with ErrorCode: INVALID_REQUEST |
| Maximum request rate | 15,000 per second | Slowdown with throttle duration in response |
|
Maximum rate of topic management APIs requests rate |
2 per second | Slowdown with throttle duration in response |
|
Maximum fetch bytes per request |
55 MB |
Request fails with ErrorCode: INVALID_REQUEST |
|
Maximum number of consumer groups |
500 | JoinGroup request fails |
| Maximum number of partitions (leaders) | 2400 for non-compacted topics. 120 for compacted topics. To request a service quota adjustment, create a support case with your use case and requested limit. | Request fails with ErrorCode: INVALID_REQUEST |
| Maximum rate of partition creation and deletion | 250 in 5 minutes | Request fails with ErrorCode: THROUGHPUT_QUOTA_EXCEEDED |
|
Maximum ingress throughput per partition |
5 MBps | Slowdown with throttle duration in response |
|
Maximum egress throughput per partition |
10 MBps | Slowdown with throttle duration in response |
|
Maximum partition size (for compacted topics) |
250 GB | Request fails with ErrorCode: THROUGHPUT_QUOTA_EXCEEDED |
| Maximum number of client VPCs per serverless cluster | 5 | |
| Maximum number of serverless clusters per account | 10. To request a service quota adjustment, create a support case with your use case and requested limit. |
MSK Connect quota
Up to 100 custom plugins.
-
Up to 100 worker configurations.
-
Up to 60 connect workers. If a connector is set up to have auto scaled capacity, then the maximum number of workers that the connector is set up to have is the number MSK Connect uses to calculate the quota for the account.
-
Up to 10 workers per connector.
To request higher quota for MSK Connect, go to the Service Quotas console
