Select your cookie preferences

We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.

If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”

Preferences
Contact Us
Feedback
AWS Documentation
Create an AWS Account

Migrate from one Amazon MSK cluster to another using MSK Replicator

PDF
RSS
Focus mode
Migrate from one Amazon MSK cluster to another using MSK Replicator - Amazon Managed Streaming for Apache Kafka

You can use Identical topic name replication for cluster migration, but your consumers must be able to handle duplicate messages without downstream impact. This is because MSK Replicator provides at-least-once replication, which can lead to duplicate messages in rare scenarios. If your consumers meet this requirement, follow these steps.

  1. Create a Replicator that replicates data from your old cluster to the new cluster with Replicator's starting position set to Earliest and using Identical topic name replication (Keep the same topics name in console).

  2. Configure cluster-level settings and permissions on the new cluster. You do not need to configure topic-level settings and “literal” read ACLs, as MSK Replicator automatically copies them.

  3. Monitor the MessageLag metric in Amazon CloudWatch until it reaches 0 which indicates all data has been replicated.

  4. After all data has been replicated, stop producers from writing data to the old cluster.

  5. Reconfigure those producers to connect to the new cluster and start them.

  6. Monitor MaxOffsetLag metric for your consumers reading data from the old cluster until it becomes 0, which indicates all existing data has been processed.

  7. Stop consumers that are connecting to the old cluster.

  8. Reconfigure consumers to connect to the new cluster and start them.

PrivacySite termsCookie preferences
© 2025, Amazon Web Services, Inc. or its affiliates. All rights reserved.