# Manage storage throughput for Standard brokers in a Amazon MSK cluster
For information on how to provision throughput using the Amazon MSK console, CLI, and API, see [Provision storage throughput for Standard brokers in a Amazon MSK cluster](msk-provision-throughput.md).
**Topics**
+ [Amazon MSK broker throughput bottlenecks and maximum throughput settings](#throughput-bottlenecks)
+ [Measure storage throughput of a Amazon MSK cluster](#throughput-metrics)
+ [Configuration update values for provisioned storage in a Amazon MSK cluster](#provisioned-throughput-config)
+ [Provision storage throughput for Standard brokers in a Amazon MSK cluster](msk-provision-throughput.md)
## Amazon MSK broker throughput bottlenecks and maximum throughput settings
There are multiple causes of bottlenecks in broker throughput: volume throughput, Amazon EC2 to Amazon EBS network throughput, and Amazon EC2 egress throughput. You can enable provisioned storage throughput to adjust volume throughput. However, broker throughput limitations can be caused by Amazon EC2 to Amazon EBS network throughput and Amazon EC2 egress throughput.
Amazon EC2 egress throughput is impacted by the number of consumer groups and consumers per consumer groups. Also, both Amazon EC2 to Amazon EBS network throughput and Amazon EC2 egress throughput are higher for larger broker sizes.
For volume sizes of 10 GiB or larger, you can provision storage throughput of 250 MiB per second or greater. 250 MiB per second is the default. To provision storage throughput, you must choose broker size kafka.m5.4xlarge or larger (or kafka.m7g.2xlarge or larger), and you can specify maximum throughput as shown in the following table.
****
| broker size | Maximum storage throughput (MiB/second) |
| --- | --- |
| kafka.m5.4xlarge | 593 |
| kafka.m5.8xlarge | 850 |
| kafka.m5.12xlarge | 1000 |
| kafka.m5.16xlarge | 1000 |
| kafka.m5.24xlarge | 1000 |
| kafka.m7g.2xlarge | 312.5 |
| kafka.m7g.4xlarge | 625 |
| kafka.m7g.8xlarge | 1000 |
| kafka.m7g.12xlarge | 1000 |
| kafka.m7g.16xlarge | 1000 |
## Measure storage throughput of a Amazon MSK cluster
You can use the `VolumeReadBytes` and `VolumeWriteBytes` metrics to measure the average storage throughput of a cluster. The sum of these two metrics gives the average storage throughput in bytes. To get the average storage throughput for a cluster, set these two metrics to SUM and the period to 1 minute, then use the following formula.
```
Average storage throughput in MiB/s = (Sum(VolumeReadBytes) + Sum(VolumeWriteBytes)) / (60 * 1024 * 1024)
```
For information about the `VolumeReadBytes` and `VolumeWriteBytes` metrics, see [`PER_BROKER` Level monitoring](metrics-details.md#broker-metrics).
## Configuration update values for provisioned storage in a Amazon MSK cluster
You can update your Amazon MSK configuration either before or after you turn on provisioned throughput. However, you won't see the desired throughput until you perform both actions: update the `num.replica.fetchers` configuration parameter and turn on provisioned throughput.
In the default Amazon MSK configuration, `num.replica.fetchers` has a value of 2. To update your `num.replica.fetchers`, you can use the suggested values from the following table. These values are for guidance purposes. We recommend that you adjust these values based on your use case.
****
| broker size | num.replica.fetchers |
| --- | --- |
| kafka.m5.4xlarge | 4 |
| kafka.m5.8xlarge | 8 |
| kafka.m5.12xlarge | 14 |
| kafka.m5.16xlarge | 16 |
| kafka.m5.24xlarge | 16 |
Your updated configuration may not take effect for up to 24 hours, and may take longer when a source volume is not fully utilized. However, transitional volume performance at least equals the performance of source storage volumes during the migration period. A fully-utilized 1 TiB volume typically takes about six hours to migrate to an updated configuration.
# Provision storage throughput for Standard brokers in a Amazon MSK cluster
Amazon MSK brokers persist data on storage volumes. Storage I/O is consumed when producers write to the cluster, when data is replicated between brokers, and when consumers read data that isn't in memory. The volume storage throughput is the rate at which data can be written into and read from a storage volume. Provisioned storage throughput is the ability to specify that rate for the brokers in your cluster.
You can specify the provisioned throughput rate in MiB per second for clusters whose brokers are of size `kafka.m5.4xlarge` or larger and if the storage volume is 10 GiB or greater. It is possible to specify provisioned throughput during cluster creation. You can also enable or disable provisioned throughput for a cluster that is in the `ACTIVE` state.
For information about managing throughput, see [Manage storage throughput for Standard brokers in a Amazon MSK cluster](msk-provision-throughput-management.md).
**Topics**
+ [Provision Amazon MSK cluster storage throughput using the AWS Management Console](#provisioned-throughput-console)
+ [Provision Amazon MSK cluster storage throughput using the AWS CLI](#provisioned-throughput-cli)
+ [Provision storage throughput when creating a Amazon MSK cluster using the API](#provisioned-throughput-api)
## Provision Amazon MSK cluster storage throughput using the AWS Management Console
This process shows an example of how you can use the AWS Management Console to create a Amazon MSK cluster with provisioned throughput enabled.
1. Sign in to the AWS Management Console, and open the Amazon MSK console at [https://console.aws.amazon.com/msk/home?region=us-east-1\$1/home/](https://console.aws.amazon.com/msk/home?region=us-east-1#/home/).
1. Choose **Create cluster**.
1. Choose **Custom create**.
1. Specify a name for the cluster.
1. In the **Storage** section, choose **Enable**.
1. Choose a value for storage throughput per broker.
1. Choose a VPC, zones and subnets, and a security group.
1. Choose **Next**.
1. At the bottom of the **Security** step, choose **Next**.
1. At the bottom of the **Monitoring and tags** step, choose **Next**.
1. Review the cluster settings, then choose **Create cluster**.
## Provision Amazon MSK cluster storage throughput using the AWS CLI
This process shows an example of how you can use the AWS CLI to create a cluster with provisioned throughput enabled.
1. Copy the following JSON and paste it into a file. Replace the subnet IDs and security group ID placeholders with values from your account. Name the file `cluster-creation.json` and save it.
```
{
"Provisioned": {
"BrokerNodeGroupInfo":{
"InstanceType":"kafka.m5.4xlarge",
"ClientSubnets":[
"Subnet-1-ID",
"Subnet-2-ID"
],
"SecurityGroups":[
"Security-Group-ID"
],
"StorageInfo": {
"EbsStorageInfo": {
"VolumeSize": 10,
"ProvisionedThroughput": {
"Enabled": true,
"VolumeThroughput": 250
}
}
}
},
"EncryptionInfo": {
"EncryptionInTransit": {
"InCluster": false,
"ClientBroker": "PLAINTEXT"
}
},
"KafkaVersion":"2.8.1",
"NumberOfBrokerNodes": 2
},
"ClusterName": "provisioned-throughput-example"
}
```
1. Run the following AWS CLI command from the directory where you saved the JSON file in the previous step.
```
aws kafka create-cluster-v2 --cli-input-json file://cluster-creation.json
```
## Provision storage throughput when creating a Amazon MSK cluster using the API
To configure provisioned storage throughput while creating a cluster, use [CreateClusterV2](https://docs.aws.amazon.com/MSK/2.0/APIReference/v2-clusters.html#CreateClusterV2).