View a markdown version of this page

Supported configurations - Amazon Managed Streaming for Apache Kafka
Supported cluster types and versionsAuthentication requirementsMSK Serverless cluster configuration

Supported configurations

The following are requirements for cluster types, Kafka versions, instance types, and network configurations when using MSK Replicator.

Supported cluster types and versions

  • MSK Replicator supports both MSK Provisioned clusters and MSK Serverless clusters in any combination as source and target clusters.

  • MSK Replicator also supports self-managed Apache Kafka clusters (Kafka version 2.8.1 or later) with SASL/SCRAM authentication as source clusters when replicating to Amazon MSK Provisioned clusters with Express brokers. For more information, see Migrate from non-MSK Apache Kafka clusters to Amazon MSK Express brokers.

  • MSK Replicator is supported only on MSK clusters running Apache Kafka 2.7.0 or higher.

    • Identical topic name replication requires an MSK cluster running Kafka version 2.8.1 or higher.

    • Self-managed Apache Kafka clusters require Kafka version 2.8.1 or later.

  • MSK Replicator supports clusters using instance types of m5.large or larger. t3.small clusters are not supported.

  • If you are using MSK Replicator with an MSK Provisioned cluster, you need a minimum of three brokers in both source and target clusters. You can replicate data across clusters in two Availability Zones, but you need a minimum of four brokers in those clusters.

  • Both your source and target MSK clusters must be in the same AWS account. Replication across clusters in different accounts is not supported.

Authentication requirements

  • If you are using MSK Replicator with an MSK cluster, the cluster must have IAM access control turned on.

  • If you are using MSK Replicator with a self-managed Apache Kafka cluster as the source, the self-managed cluster must have SASL/SCRAM authentication enabled.

  • For cross-region replication, the source cluster must have multi-VPC private connectivity turned on for its IAM access control method.

  • For same-region replication, multi-VPC private connectivity is not required. The source cluster can still be accessed by other clients using the unauthenticated auth type.

  • MSK Serverless clusters require IAM access control and do not support Apache Kafka ACL replication.

MSK Serverless cluster configuration

  • MSK Serverless supports replication of these topic configurations for target clusters during topic creation: cleanup.policy, compression.type, max.message.bytes, retention.bytes, retention.ms.

  • MSK Serverless supports only these topic configurations during topic configuration sync: compression.type, max.message.bytes, retention.bytes, retention.ms.

  • Replicator uses 83 compacted partitions on target MSK Serverless clusters. Make sure that target MSK Serverless clusters have a sufficient number of compacted partitions. See MSK Serverless quota.