Migration playbook: MongoDB to Amazon DocumentDB

This migration playbook provides you with resources and steps to help you migrate from a MongoDB database to Amazon DocumentDB.

Migration process

Listed below are the high-level steps typically involved in migrating your data from a MongoDB database to Amazon DocumentDB.

Step 1: Compatibility and functional differences

Amazon DocumentDB interacts with the Apache 2.0 open-source MongoDB 3.6, 4.0, and 5.0 APIs. As a result, you can use the same MongoDB drivers, applications, and tools with Amazon DocumentDB with little or no changes.

The first step is to check for compatibility between the operators and indexes your application uses on your MongoDB database and their availability in Amazon DocumentDB, as well as to understand the functional differences between them.

Operators compatibility

Use the Amazon DocumentDB compatibility tool* to easily discover if your application uses any unsupported operators in its queries. This tool can scan your MongoDB database server log files or your application source code to provide a report of unsupported operators. If you find the usage of unsupported operators, you need to modify your application to work around unsupported operators.

To check compatibility between the MongoDB operators used in your setup and the supported Amazon DocumentDB operators, execute the following:

git clone https://github.com/awslabs/amazon-documentdb-tools.git
cd amazon-documentdb-tools/compat-tool/
python3 compat.py --version <Amazon DocumentDB version> --directory <mongodb logfiles/source code>

For more information, see Supported MongoDB APIs, operations, and data types in Amazon DocumentDB.

^{* Not officially supported by AWS.}

Indexes compatibility

You can use the Amazon DocumentDB index tool* to find out if you are using any index types unsupported in Amazon DocumentDB. This tool needs a connection to your source database to read index definitions.

For this, you first need to dump index definitions into a directory using the --dump-indexes option. Then run the tool with the --show-issues option, providing the directory to locate incompatible indexes.

Export indexes:

git clone https://github.com/awslabs/amazon-documentdb-tools.git
sudo pip install -r amazon-documentdb-tools/index-tool/requirements.txt
mkdir <directory to dump index definitions>
python3 migrationtools/documentdb_index_tool.py --dump-indexes --dir <directory> --uri <source-mongodb-uri>

Check for incompatible indexes:

python3 migrationtools/documentdb_index_tool.py --show-issues --dir <dumped-index-definitions-directory>

If you find the usage of any unsupported index types, you must modify your application or data model to work around or continue without the incompatible indexes.

For more information on supported index types and properties in Amazon DocumentDB, see Indexes and index properties and How to index on Amazon DocumentDB.

^{* Not officially supported by AWS.}

Functional differences

Review Functional differences with MongoDB to get familiar with the differences.

Step 2: Proof of concept

Perform a proof of concept by running your application or your regular test suite on Amazon DocumentDB to test for functionality and performance. You may need to populate your Amazon DocumentDB cluster with data to perform the tests. For example, you can use the mongodump and mongorestore tools to copy data from your source MongoDB.

Functional testing

Create an Amazon DocumentDB cluster (see Creating an Amazon DocumentDB cluster) and run your application or your functional test suite to validate if all the application workflows continue working seamlessly on Amazon DocumentDB.

Performance testing

Run performance tests on your application or performance test suite running on Amazon DocumentDB with a workload similar to your production workload to see if the setup meets your latency requirements. Fine-tune your workload for performance or scale your Amazon DocumentDB cluster as applicable. For more information, see Performance and resource utilization and Scaling Amazon DocumentDB clusters.

It is important to size your Amazon DocumentDB cluster with the right instance types for optimal performance. For more information, see best practices for Instance sizing.

You can use the Amazon DocumentDB sizing calculator* to assist you in estimating the size of your Amazon DocumentDB cluster.

^{* Not officially supported by AWS.}

Failover testing

You may want to observe how your application responds to an Amazon DocumentDB primary node reboot, a primary node failover, or a primary node deletion in a multi-node cluster, as well as when replica nodes are rebooted or removed. This will help you confirm that your application is resilient to these events. For more information, see Testing failover.

To understand the exceptions that an application should tolerate and how to handle them efficiently, see Building resilient applications with Amazon DocumentDB.

Step 3: Migrate the data

After a successful proof of concept, migrate your data to Amazon DocumentDB. Most of our customers use online or offline migration approaches to migrate their data.

Online migration

Using the online migration method, you can migrate data from your source database, ranging from a few gigabytes to multiple terabytes, to Amazon DocumentDB with near-zero downtime. For more information, see AWS Database Migration Service (AWS DMS).

If you are migrating from a MongoDB database you can use AWS DMS to do a full load and replicate ongoing changes.

For a step-by-step process, see Migrating to Amazon DocumentDB with the online method.

Additional information can be found in the Using Amazon DocumentDB as a target for AWS Database Migration Service section in the AWS Database Migration Service User Guide.

Points to note with AWS DMS:

Offline migration

Offline migration is the most straightforward approach to move databases to Amazon DocumentDB. This approach is mainly used for POCs and for workloads that can take write downtime during the migration.

For a step-by-step process, see Migrate from MongoDB to Amazon DocumentDB using the offline method.

Step 4: Data validation

Once the data is migrated successfully, validate the data for correctness to gain confidence. On the AWS DMS migration task console, you can find migrated data metrics. For more information, see verify migrated data.

You can also use the Amazon DocumentDB DataDiffer Tool* to validate the data consistency between the the source and target collections.

^{* Not officially supported by AWS.}

Step 5: Application cutover

This involves changing your application’s database connection string to use your Amazon DocumentDB cluster.

For more information on connecting to Amazon DocumentDB, see Connecting to Amazon DocumentDB as a replica set.

Online Migration

After the full data load is done, AWS DMS continues to replicate ongoing changes from your source to Amazon DocumentDB. After the changes are caught up and your data validation checks are complete, you can perform a cutover to Amazon DocumentDB.

Offline Migration

Once done with the full data load and data validation checks, you can perform the cutover to Amazon DocumentDB.

Additional resources

Here are some additional resources that could help your migration:

^{* Not officially supported by AWS.}