

# Sources for data migration
<a name="CHAP_Source"></a>

AWS Database Migration Service (AWS DMS) can use many of the most popular data engines as a source for data replication. The database source can be a self-managed engine running on an Amazon EC2 instance or an on-premises database. Or it can be a data source on an AWS service such as Amazon RDS or Amazon S3.

For a comprehensive list of valid sources, see [Sources for AWS DMS](CHAP_Introduction.Sources.md#CHAP_Introduction.Sources.title).

**Topics**
+ [

# Using an Oracle database as a source for AWS DMS
](CHAP_Source.Oracle.md)
+ [

# Using a Microsoft SQL Server database as a source for AWS DMS
](CHAP_Source.SQLServer.md)
+ [

# Using Microsoft Azure SQL database as a source for AWS DMS
](CHAP_Source.AzureSQL.md)
+ [

# Using Microsoft Azure SQL Managed Instance as a source for AWS DMS
](CHAP_Source.AzureMgd.md)
+ [

# Using Microsoft Azure Database for PostgreSQL flexible server as a source for AWS DMS
](CHAP_Source.AzureDBPostgreSQL.md)
+ [

# Using Microsoft Azure Database for MySQL flexible server as a source for AWS DMS
](CHAP_Source.AzureDBMySQL.md)
+ [

# Using OCI MySQL Heatwave as a source for AWS DMS
](CHAP_Source.heatwave.md)
+ [

# Using Google Cloud for MySQL as a source for AWS DMS
](CHAP_Source.GC.md)
+ [

# Using Google Cloud for PostgreSQL as a source for AWS DMS
](CHAP_Source.GCPostgres.md)
+ [

# Using a PostgreSQL database as an AWS DMS source
](CHAP_Source.PostgreSQL.md)
+ [

# Using a MySQL-compatible database as a source for AWS DMS
](CHAP_Source.MySQL.md)
+ [

# Using an SAP ASE database as a source for AWS DMS
](CHAP_Source.SAP.md)
+ [

# Using MongoDB as a source for AWS DMS
](CHAP_Source.MongoDB.md)
+ [

# Using Amazon DocumentDB (with MongoDB compatibility) as a source for AWS DMS
](CHAP_Source.DocumentDB.md)
+ [

# Using Amazon S3 as a source for AWS DMS
](CHAP_Source.S3.md)
+ [

# Using IBM Db2 for Linux, Unix, Windows, and Amazon RDS database (Db2 LUW) as a source for AWS DMS
](CHAP_Source.DB2.md)
+ [

# Using IBM Db2 for z/OS databases as a source for AWS DMS
](CHAP_Source.DB2zOS.md)

# Using an Oracle database as a source for AWS DMS
<a name="CHAP_Source.Oracle"></a>

You can migrate data from one or many Oracle databases using AWS DMS. With an Oracle database as a source, you can migrate data to any of the targets supported by AWS DMS.

AWS DMS supports the following Oracle database editions:
+ Oracle Enterprise Edition
+ Oracle Standard Edition
+ Oracle Express Edition
+ Oracle Personal Edition

For information about versions of Oracle databases that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

You can use Secure Sockets Layer (SSL) to encrypt connections between your Oracle endpoint and your replication instance. For more information on using SSL with an Oracle endpoint, see [SSL support for an Oracle endpoint](#CHAP_Security.SSL.Oracle).

AWS DMS supports the use of Oracle transparent data encryption (TDE) to encrypt data at rest in the source database. For more information on using Oracle TDE with an Oracle source endpoint, see [Supported encryption methods for using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.Encryption).

AWS supports the use of TLS version 1.2 and later with Oracle endpoints (and all other endpoint types), and recommends using TLS version 1.3 or later.

Follow these steps to configure an Oracle database as an AWS DMS source endpoint:

1. Create an Oracle user with the appropriate permissions for AWS DMS to access your Oracle source database.

1. Create an Oracle source endpoint that conforms with your chosen Oracle database configuration. To create a full-load-only task, no further configuration is needed.

1. To create a task that handles change data capture (a CDC-only or full-load and CDC task), choose Oracle LogMiner or AWS DMS Binary Reader to capture data changes. Choosing LogMiner or Binary Reader determines some of the later permissions and configuration options. For a comparison of LogMiner and Binary Reader, see the following section.

**Note**  
For more information on full-load tasks, CDC-only tasks, and full-load and CDC tasks, see [Creating a task](CHAP_Tasks.Creating.md)

For additional details on working with Oracle source databases and AWS DMS, see the following sections. 

**Topics**
+ [

## Using Oracle LogMiner or AWS DMS Binary Reader for CDC
](#CHAP_Source.Oracle.CDC)
+ [

## Workflows for configuring a self-managed or AWS-managed Oracle source database for AWS DMS


## Configuring an Oracle source database
](#CHAP_Source.Oracle.Workflows)
+ [

## Working with a self-managed Oracle database as a source for AWS DMS
](#CHAP_Source.Oracle.Self-Managed)
+ [

## Working with an AWS-managed Oracle database as a source for AWS DMS
](#CHAP_Source.Oracle.Amazon-Managed)
+ [

## Limitations on using Oracle as a source for AWS DMS
](#CHAP_Source.Oracle.Limitations)
+ [

## SSL support for an Oracle endpoint
](#CHAP_Security.SSL.Oracle)
+ [

## Supported encryption methods for using Oracle as a source for AWS DMS
](#CHAP_Source.Oracle.Encryption)
+ [

## Supported compression methods for using Oracle as a source for AWS DMS
](#CHAP_Source.Oracle.Compression)
+ [

## Replicating nested tables using Oracle as a source for AWS DMS
](#CHAP_Source.Oracle.NestedTables)
+ [

## Storing REDO on Oracle ASM when using Oracle as a source for AWS DMS
](#CHAP_Source.Oracle.REDOonASM)
+ [

## Endpoint settings when using Oracle as a source for AWS DMS
](#CHAP_Source.Oracle.ConnectionAttrib)
+ [

## Source data types for Oracle
](#CHAP_Source.Oracle.DataTypes)

## Using Oracle LogMiner or AWS DMS Binary Reader for CDC
<a name="CHAP_Source.Oracle.CDC"></a>

In AWS DMS, there are two methods for reading the redo logs when doing change data capture (CDC) for Oracle as a source: Oracle LogMiner and AWS DMS Binary Reader. LogMiner is an Oracle API to read the online redo logs and archived redo log files. Binary Reader is an AWS DMS method that reads and parses the raw redo log files directly. These methods have the following features.


| Feature | LogMiner | Binary Reader | 
| --- | --- | --- | 
| Easy to configure | Yes | No | 
| Lower impact on source system I/O and CPU | No | Yes | 
| Better CDC performance | No | Yes | 
| Supports Oracle table clusters | Yes | No | 
| Supports all types of Oracle Hybrid Columnar Compression (HCC) | Yes |  Partially Binary Reader does not support QUERY LOW for tasks with CDC. All other HCC types are fully supported.  | 
| LOB column support in Oracle 12c only | No (LOB Support is not available with LogMiner in Oracle 12c.) | Yes | 
| Supports UPDATE statements that affect only LOB columns | No | Yes | 
| Supports Oracle transparent data encryption (TDE) |  Partially When using Oracle LogMiner, AWS DMS does not support TDE encryption on column level for Amazon RDS for Oracle.  |  Partially Binary Reader supports TDE only for self-managed Oracle databases.  | 
| Supports all Oracle compression methods | Yes | No | 
| Supports XA transactions | No | Yes | 
| RAC |  Yes Not recommended, due to performance reasons, and some internal DMS limitations.  |  Yes Highly recommended  | 

**Note**  
By default, AWS DMS uses Oracle LogMiner for (CDC).   
AWS DMS supports transparent data encryption (TDE) methods when working with an Oracle source database. If the TDE credentials you specify are incorrect, the AWS DMS migration task does not fail, which can impact ongoing replication of encrypted tables. For more information about specifying TDE credentials, see [Supported encryption methods for using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.Encryption).

The main advantages of using LogMiner with AWS DMS include the following:
+ LogMiner supports most Oracle options, such as encryption options and compression options. Binary Reader does not support all Oracle options, particularly compression and most options for encryption.
+ LogMiner offers a simpler configuration, especially compared to Binary Reader direct-access setup or when the redo logs are managed using Oracle Automatic Storage Management (ASM).
+ LogMiner supports table clusters for use by AWS DMS. Binary Reader does not.

The main advantages of using Binary Reader with AWS DMS include the following:
+ For migrations with a high volume of changes, LogMiner might have some I/O or CPU impact on the computer hosting the Oracle source database. Binary Reader has less chance of having I/O or CPU impact because logs are mined directly rather than making multiple database queries.
+ For migrations with a high volume of changes, CDC performance is usually much better when using Binary Reader compared with using Oracle LogMiner.
+ Binary Reader supports CDC for LOBs in Oracle version 12c. LogMiner does not.

In general, use Oracle LogMiner for migrating your Oracle database unless you have one of the following situations:
+ You need to run several migration tasks on the source Oracle database.
+ The volume of changes or the redo log volume on the source Oracle database is high, or you have changes and are also using Oracle ASM.

**Note**  
If you change between using Oracle LogMiner and AWS DMS Binary Reader, make sure to restart the CDC task. 

### Configuration for CDC on an Oracle source database
<a name="CHAP_Source.Oracle.CDC.Configuration"></a>

For an Oracle source endpoint to connect to the database for a change data capture (CDC) task, you might need to specify extra connection attributes. This can be true for either a full-load and CDC task or for a CDC-only task. The extra connection attributes that you specify depend on the method you use to access the redo logs: Oracle LogMiner or AWS DMS Binary Reader. 

You specify extra connection attributes when you create a source endpoint. If you have multiple connection attribute settings, separate them from each other by semicolons with no additional white space (for example, `oneSetting;thenAnother`).

AWS DMS uses LogMiner by default. You don't have to specify additional extra connection attributes to use it. 

To use Binary Reader to access the redo logs, add the following extra connection attributes.

```
useLogMinerReader=N;useBfile=Y;
```

Use the following format for the extra connection attributes to access a server that uses ASM with Binary Reader.

```
useLogMinerReader=N;useBfile=Y;asm_user=asm_username;asm_server=RAC_server_ip_address:port_number/+ASM;
```

Set the source endpoint `Password` request parameter to both the Oracle user password and the ASM password, separated by a comma as follows.

```
oracle_user_password,asm_user_password
```

Where the Oracle source uses ASM, you can work with high-performance options in Binary Reader for transaction processing at scale. These options include extra connection attributes to specify the number of parallel threads (`parallelASMReadThreads`) and the number of read-ahead buffers (`readAheadBlocks`). Setting these attributes together can significantly improve the performance of the CDC task. The following settings provide good results for most ASM configurations.

```
useLogMinerReader=N;useBfile=Y;asm_user=asm_username;asm_server=RAC_server_ip_address:port_number/+ASM;
    parallelASMReadThreads=6;readAheadBlocks=150000;
```

For more information on values that extra connection attributes support, see [Endpoint settings when using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.ConnectionAttrib).

In addition, the performance of a CDC task with an Oracle source that uses ASM depends on other settings that you choose. These settings include your AWS DMS extra connection attributes and the SQL settings to configure the Oracle source. For more information on extra connection attributes for an Oracle source using ASM, see [Endpoint settings when using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.ConnectionAttrib)

You also need to choose an appropriate CDC start point. Typically when you do this, you want to identify the point of transaction processing that captures the earliest open transaction to begin CDC from. Otherwise, the CDC task can miss earlier open transactions. For an Oracle source database, you can choose a CDC native start point based on the Oracle system change number (SCN) to identify this earliest open transaction. For more information, see [Performing replication starting from a CDC start point](CHAP_Task.CDC.md#CHAP_Task.CDC.StartPoint).

For more information on configuring CDC for a self-managed Oracle database as a source, see [Account privileges required when using Oracle LogMiner to access the redo logs](#CHAP_Source.Oracle.Self-Managed.LogMinerPrivileges), [Account privileges required when using AWS DMS Binary Reader to access the redo logs](#CHAP_Source.Oracle.Self-Managed.BinaryReaderPrivileges), and [Additional account privileges required when using Binary Reader with Oracle ASM](#CHAP_Source.Oracle.Self-Managed.ASMBinaryPrivileges).

For more information on configuring CDC for an AWS-managed Oracle database as a source, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC) and [Using an Amazon RDS Oracle Standby (read replica) as a source with Binary Reader for CDC in AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.StandBy).

## Workflows for configuring a self-managed or AWS-managed Oracle source database for AWS DMS


## Configuring an Oracle source database
<a name="CHAP_Source.Oracle.Workflows"></a>

To configure a self-managed source database instance, use the following workflow steps , depending on how you perform CDC. 


| For this workflow step | If you perform CDC using LogMiner, do this | If you perform CDC using Binary Reader, do this | 
| --- | --- | --- | 
| Grant Oracle account privileges. | See [User account privileges required on a self-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Self-Managed.Privileges). | See [User account privileges required on a self-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Self-Managed.Privileges). | 
| Prepare the source database for replication using CDC. | See [Preparing an Oracle self-managed source database for CDC using AWS DMS](#CHAP_Source.Oracle.Self-Managed.Configuration). | See [Preparing an Oracle self-managed source database for CDC using AWS DMS](#CHAP_Source.Oracle.Self-Managed.Configuration). | 
| Grant additional Oracle user privileges required for CDC. | See [Account privileges required when using Oracle LogMiner to access the redo logs](#CHAP_Source.Oracle.Self-Managed.LogMinerPrivileges). | See [Account privileges required when using AWS DMS Binary Reader to access the redo logs](#CHAP_Source.Oracle.Self-Managed.BinaryReaderPrivileges). | 
| For an Oracle instance with ASM, grant additional user account privileges required to access ASM for CDC. | No additional action. AWS DMS supports Oracle ASM without additional account privileges. | See [Additional account privileges required when using Binary Reader with Oracle ASM](#CHAP_Source.Oracle.Self-Managed.ASMBinaryPrivileges). | 
| If you haven't already done so, configure the task to use LogMiner or Binary Reader for CDC. | See [Using Oracle LogMiner or AWS DMS Binary Reader for CDC](#CHAP_Source.Oracle.CDC). | See [Using Oracle LogMiner or AWS DMS Binary Reader for CDC](#CHAP_Source.Oracle.CDC). | 
| Configure Oracle Standby as a source for CDC. | AWS DMS does not support Oracle Standby as a source. | See [Using a self-managed Oracle Standby as a source with Binary Reader for CDC in AWS DMS](#CHAP_Source.Oracle.Self-Managed.BinaryStandby). | 

Use the following workflow steps to configure an AWS-managed Oracle source database instance.


| For this workflow step | If you perform CDC using LogMiner, do this | If you perform CDC using Binary Reader, do this | 
| --- | --- | --- | 
| Grant Oracle account privileges. | For more information, see [User account privileges required on an AWS-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.Privileges). | For more information, see [User account privileges required on an AWS-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.Privileges). | 
| Prepare the source database for replication using CDC. | For more information, see [Configuring an AWS-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.Configuration). | For more information, see [Configuring an AWS-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.Configuration). | 
| Grant additional Oracle user privileges required for CDC. | No additional account privileges are required. | For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC). | 
| If you haven't already done so, configure the task to use LogMiner or Binary Reader for CDC. | For more information, see [Using Oracle LogMiner or AWS DMS Binary Reader for CDC](#CHAP_Source.Oracle.CDC). | For more information, see [Using Oracle LogMiner or AWS DMS Binary Reader for CDC](#CHAP_Source.Oracle.CDC). | 
| Configure Oracle Standby as a source for CDC. | AWS DMS does not support Oracle Standby as a source. | For more information, see [Using an Amazon RDS Oracle Standby (read replica) as a source with Binary Reader for CDC in AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.StandBy). | 

## Working with a self-managed Oracle database as a source for AWS DMS
<a name="CHAP_Source.Oracle.Self-Managed"></a>

A *self-managed database* is a database that you configure and control, either a local on-premises database instance or a database on Amazon EC2. Following, you can find out about the privileges and configurations you need when using a self-managed Oracle database with AWS DMS.

### User account privileges required on a self-managed Oracle source for AWS DMS
<a name="CHAP_Source.Oracle.Self-Managed.Privileges"></a>

To use an Oracle database as a source in AWS DMS, grant the following privileges to the Oracle user specified in the Oracle endpoint connection settings.

**Note**  
When granting privileges, use the actual name of objects, not the synonym for each object. For example, use `V_$OBJECT` including the underscore, not `V$OBJECT` without the underscore.

```
GRANT CREATE SESSION TO dms_user;
GRANT SELECT ANY TRANSACTION TO dms_user;
GRANT SELECT ON V_$ARCHIVED_LOG TO dms_user;
GRANT SELECT ON V_$LOG TO dms_user;
GRANT SELECT ON V_$LOGFILE TO dms_user;
GRANT SELECT ON V_$LOGMNR_LOGS TO dms_user;
GRANT SELECT ON V_$LOGMNR_CONTENTS TO dms_user;
GRANT SELECT ON V_$DATABASE TO dms_user;
GRANT SELECT ON V_$THREAD TO dms_user;
GRANT SELECT ON V_$PARAMETER TO dms_user;
GRANT SELECT ON V_$NLS_PARAMETERS TO dms_user;
GRANT SELECT ON V_$TIMEZONE_NAMES TO dms_user;
GRANT SELECT ON V_$TRANSACTION TO dms_user;
GRANT SELECT ON V_$CONTAINERS TO dms_user;                   
GRANT SELECT ON ALL_INDEXES TO dms_user;
GRANT SELECT ON ALL_OBJECTS TO dms_user;
GRANT SELECT ON ALL_TABLES TO dms_user;
GRANT SELECT ON ALL_USERS TO dms_user;
GRANT SELECT ON ALL_CATALOG TO dms_user;
GRANT SELECT ON ALL_CONSTRAINTS TO dms_user;
GRANT SELECT ON ALL_CONS_COLUMNS TO dms_user;
GRANT SELECT ON ALL_TAB_COLS TO dms_user;
GRANT SELECT ON ALL_IND_COLUMNS TO dms_user;
GRANT SELECT ON ALL_ENCRYPTED_COLUMNS TO dms_user;
GRANT SELECT ON ALL_LOG_GROUPS TO dms_user;
GRANT SELECT ON ALL_TAB_PARTITIONS TO dms_user;
GRANT SELECT ON SYS.DBA_REGISTRY TO dms_user;
GRANT SELECT ON SYS.OBJ$ TO dms_user;
GRANT SELECT ON DBA_TABLESPACES TO dms_user;
GRANT SELECT ON DBA_OBJECTS TO dms_user; -– Required if the Oracle version is earlier than 11.2.0.3.
GRANT SELECT ON SYS.ENC$ TO dms_user; -– Required if transparent data encryption (TDE) is enabled. For more information on using Oracle TDE with AWS DMS, see Supported encryption methods for using Oracle as a source for AWS DMS.
GRANT SELECT ON GV_$TRANSACTION TO dms_user; -– Required if the source database is Oracle RAC in AWS DMS versions 3.4.6 and higher.
GRANT SELECT ON V_$DATAGUARD_STATS TO dms_user; -- Required if the source database is Oracle Data Guard and Oracle Standby is used in the latest release of DMS version 3.4.6, version 3.4.7, and higher.
GRANT SELECT ON V_$DATABASE_INCARNATION TO dms_user;
```

Grant the additional following privilege for each replicated table when you are using a specific table list.

```
GRANT SELECT on any-replicated-table to dms_user;
```

Grant the additional following privilege to use validation feature.

```
GRANT EXECUTE ON SYS.DBMS_CRYPTO TO dms_user;
```

Grant the additional following privilege if you use binary reader instead of LogMiner.

```
GRANT SELECT ON SYS.DBA_DIRECTORIES TO dms_user;
```

Grant the additional following privilege to expose views.

```
GRANT SELECT on ALL_VIEWS to dms_user;
```

To expose views, you must also add the `exposeViews=true` extra connection attribute to your source endpoint.

Grant the additional following privilege when using serverless replications.

```
GRANT SELECT on dba_segments to dms_user;
GRANT SELECT on v_$tablespace to dms_user;
GRANT SELECT on dba_tab_subpartitions to dms_user;
GRANT SELECT on dba_extents to dms_user;
```

For information about serverless replications, see [Working with AWS DMS Serverless](CHAP_Serverless.md).

Grant the additional following privileges when using Oracle-specific premigration assessments.

```
GRANT SELECT on gv_$parameter  to dms_user;
GRANT SELECT on v_$instance to dms_user;
GRANT SELECT on v_$version to dms_user;
GRANT SELECT on gv_$ASM_DISKGROUP to dms_user;
GRANT SELECT on gv_$database to dms_user;
GRANT SELECT on dba_db_links to dms_user;
GRANT SELECT on gv_$log_History to dms_user;
GRANT SELECT on gv_$log to dms_user;
GRANT SELECT ON DBA_TYPES TO dms_user;
GRANT SELECT ON DBA_USERS to dms_user;
GRANT SELECT ON DBA_DIRECTORIES to dms_user;
GRANT EXECUTE ON SYS.DBMS_XMLGEN TO dms_user;
```

For information about Oracle-specific premigration assessments, see [Oracle assessments](CHAP_Tasks.AssessmentReport.Oracle.md).

#### Prerequisites for handling open transactions for Oracle Standby
<a name="CHAP_Source.Oracle.Self-Managed.Privileges.Standby"></a>

When using AWS DMS versions 3.4.6 and higher, perform the following steps to handle open transactions for Oracle Standby. 

1. Create a database link named, `AWSDMS_DBLINK` on the primary database. `DMS_USER` will use the database link to connect to the primary database. Note that the database link is executed from the standby instance to query the open transactions running on the primary database. See the following example. 

   ```
   CREATE PUBLIC DATABASE LINK AWSDMS_DBLINK 
      CONNECT TO DMS_USER IDENTIFIED BY DMS_USER_PASSWORD
      USING '(DESCRIPTION=
               (ADDRESS=(PROTOCOL=TCP)(HOST=PRIMARY_HOST_NAME_OR_IP)(PORT=PORT))
               (CONNECT_DATA=(SERVICE_NAME=SID))
             )';
   ```

1. Verify the connection to the database link using `DMS_USER` is established, as shown in the following example.

   ```
   select 1 from dual@AWSDMS_DBLINK
   ```

### Preparing an Oracle self-managed source database for CDC using AWS DMS
<a name="CHAP_Source.Oracle.Self-Managed.Configuration"></a>

Prepare your self-managed Oracle database as a source to run a CDC task by doing the following: 
+ [Verifying that AWS DMS supports the source database version](#CHAP_Source.Oracle.Self-Managed.Configuration.DbVersion).
+ [Making sure that ARCHIVELOG mode is on](#CHAP_Source.Oracle.Self-Managed.Configuration.ArchiveLogMode).
+ [Setting up supplemental logging](#CHAP_Source.Oracle.Self-Managed.Configuration.SupplementalLogging).

#### Verifying that AWS DMS supports the source database version
<a name="CHAP_Source.Oracle.Self-Managed.Configuration.DbVersion"></a>

Run a query like the following to verify that the current version of the Oracle source database is supported by AWS DMS.

```
SELECT name, value, description FROM v$parameter WHERE name = 'compatible';
```

Here, `name`, `value`, and `description` are columns somewhere in the database that are being queried based on the value of `name`. If this query runs without error, AWS DMS supports the current version of the database and you can continue with the migration. If the query raises an error, AWS DMS does not support the current version of the database. To proceed with migration, first convert the Oracle database to an version supported by AWS DMS.

#### Making sure that ARCHIVELOG mode is on
<a name="CHAP_Source.Oracle.Self-Managed.Configuration.ArchiveLogMode"></a>

You can run Oracle in two different modes: the `ARCHIVELOG` mode and the `NOARCHIVELOG` mode. To run a CDC task, run the database in `ARCHIVELOG` mode. To know if the database is in `ARCHIVELOG` mode, execute the following query.

```
SQL> SELECT log_mode FROM v$database;
```

If `NOARCHIVELOG` mode is returned, set the database to `ARCHIVELOG` per Oracle instructions. 

#### Setting up supplemental logging
<a name="CHAP_Source.Oracle.Self-Managed.Configuration.SupplementalLogging"></a>

To capture ongoing changes, AWS DMS requires that you enable minimal supplemental logging on your Oracle source database. In addition, you need to enable supplemental logging on each replicated table in the database.

By default, AWS DMS adds `PRIMARY KEY` supplemental logging on all replicated tables. To allow AWS DMS to add `PRIMARY KEY` supplemental logging, grant the following privilege for each replicated table.

```
ALTER on any-replicated-table;
```

You can disable the default `PRIMARY KEY` supplemental logging added by AWS DMS using the extra connection attribute `addSupplementalLogging`. For more information, see [Endpoint settings when using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.ConnectionAttrib).

Make sure to turn on supplemental logging if your replication task updates a table using a `WHERE` clause that does not reference a primary key column.

**To manually set up supplemental logging**

1. Run the following query to verify if supplemental logging is already enabled for the database.

   ```
   SELECT supplemental_log_data_min FROM v$database;
   ```

   If the result returned is `YES` or `IMPLICIT`, supplemental logging is enabled for the database.

   If not, enable supplemental logging for the database by running the following command.

   ```
   ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
   ```

1. Make sure that the required supplemental logging is added for each replicated table.

   Consider the following:
   + If `ALL COLUMNS` supplemental logging is added to the table, you don't need to add more logging.
   + If a primary key exists, add supplemental logging for the primary key. You can do this either by using the format to add supplemental logging on the primary key itself, or by adding supplemental logging on the primary key columns on the database.

     ```
     ALTER TABLE Tablename ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
     ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
     ```
   + If no primary key exists and the table has a single unique index, add all of the unique index's columns to the supplemental log.

     ```
     ALTER TABLE TableName ADD SUPPLEMENTAL LOG GROUP LogGroupName (UniqueIndexColumn1[, UniqueIndexColumn2] ...) ALWAYS;
     ```

     Using `SUPPLEMENTAL LOG DATA (UNIQUE INDEX) COLUMNS` does not add the unique index columns to the log.
   + If no primary key exists and the table has multiple unique indexes, AWS DMS selects the first unique index in an alphabetically ordered ascending list. You need to add supplemental logging on the selected index’s columns as in the previous item.

     Using `SUPPLEMENTAL LOG DATA (UNIQUE INDEX) COLUMNS` does not add the unique index columns to the log.
   + If no primary key exists and there is no unique index, add supplemental logging on all columns.

     ```
     ALTER TABLE TableName ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
     ```

     In some cases, the target table primary key or unique index is different than the source table primary key or unique index. In such cases, add supplemental logging manually on the source table columns that make up the target table primary key or unique index.

     Also, if you change the target table primary key, add supplemental logging on the target unique index's columns instead of the columns of the source primary key or unique index.

If a filter or transformation is defined for a table, you might need to enable additional logging.

Consider the following:
+ If `ALL COLUMNS` supplemental logging is added to the table, you don't need to add more logging.
+ If the table has a unique index or a primary key, add supplemental logging on each column that is involved in a filter or transformation. However, do so only if those columns are different from the primary key or unique index columns.
+ If a transformation includes only one column, don't add this column to a supplemental logging group. For example, for a transformation `A+B`, add supplemental logging on both columns `A` and `B`. However, for a transformation `substring(A,10)` don't add supplemental logging on column `A`.
+ To set up supplemental logging on primary key or unique index columns and other columns that are filtered or transformed, you can set up `USER_LOG_GROUP` supplemental logging. Add this logging on both the primary key or unique index columns and any other specific columns that are filtered or transformed.

  For example, to replicate a table named `TEST.LOGGING` with primary key `ID` and a filter by the column `NAME`, you can run a command similar to the following to create the log group supplemental logging.

  ```
  ALTER TABLE TEST.LOGGING ADD SUPPLEMENTAL LOG GROUP TEST_LOG_GROUP (ID, NAME) ALWAYS;
  ```

### Account privileges required when using Oracle LogMiner to access the redo logs
<a name="CHAP_Source.Oracle.Self-Managed.LogMinerPrivileges"></a>

To access the redo logs using the Oracle LogMiner, grant the following privileges to the Oracle user specified in the Oracle endpoint connection settings.

```
GRANT EXECUTE on DBMS_LOGMNR to dms_user;
GRANT SELECT on V_$LOGMNR_LOGS to dms_user;
GRANT SELECT on V_$LOGMNR_CONTENTS to dms_user;
GRANT LOGMINING to dms_user; -– Required only if the Oracle version is 12c or higher.
```

### Account privileges required when using AWS DMS Binary Reader to access the redo logs
<a name="CHAP_Source.Oracle.Self-Managed.BinaryReaderPrivileges"></a>

To access the redo logs using the AWS DMS Binary Reader, grant the following privileges to the Oracle user specified in the Oracle endpoint connection settings.

```
GRANT SELECT on v_$transportable_platform to dms_user;   -– Grant this privilege if the redo logs are stored in Oracle Automatic Storage Management (ASM) and AWS DMS accesses them from ASM.
GRANT CREATE ANY DIRECTORY to dms_user;                  -– Grant this privilege to allow AWS DMS to use Oracle BFILE read file access in certain cases. This access is required when the replication instance does not have file-level access to the redo logs and the redo logs are on non-ASM storage.
GRANT EXECUTE on DBMS_FILE_TRANSFER to dms_user;         -– Grant this privilege to copy the redo log files to a temporary folder using the CopyToTempFolder method.
GRANT EXECUTE on DBMS_FILE_GROUP to dms_user;
```

Binary Reader works with Oracle file features that include Oracle directories. Each Oracle directory object includes the name of the folder containing the redo log files to process. These Oracle directories are not represented at the file system level. Instead, they are logical directories that are created at the Oracle database level. You can view them in the Oracle `ALL_DIRECTORIES` view.

If you want AWS DMS to create these Oracle directories, grant the `CREATE ANY DIRECTORY` privilege specified preceding. AWS DMS creates the directory names with the `DMS_` prefix. If you don't grant the `CREATE ANY DIRECTORY` privilege, create the corresponding directories manually. In some cases when you create the Oracle directories manually, the Oracle user specified in the Oracle source endpoint isn't the user that created these directories. In these cases, also grant the `READ on DIRECTORY` privilege.

**Note**  
AWS DMS CDC does not support Active Dataguard Standby that is not configured to use automatic redo transport service.

In some cases, you might use Oracle Managed Files (OMF) for storing the logs. Or your source endpoint is in ADG and thus you can't grant the CREATE ANY DIRECTORY privilege. In these cases, manually create the directories with all the possible log locations before starting the AWS DMS replication task. If AWS DMS does not find a precreated directory that it expects, the task stops. Also, AWS DMS does not delete the entries it has created in the `ALL_DIRECTORIES` view, so manually delete them.

### Additional account privileges required when using Binary Reader with Oracle ASM
<a name="CHAP_Source.Oracle.Self-Managed.ASMBinaryPrivileges"></a>

To access the redo logs in Automatic Storage Management (ASM) using Binary Reader, grant the following privileges to the Oracle user specified in the Oracle endpoint connection settings.

```
SELECT ON v_$transportable_platform
SYSASM -– To access the ASM account with Oracle 11g Release 2 (version 11.2.0.2) and higher, grant the Oracle endpoint user the SYSASM privilege. For older supported Oracle versions, it's typically sufficient to grant the Oracle endpoint user the SYSDBA privilege.
```

You can validate ASM account access by opening a command prompt and invoking one of the following statements, depending on your Oracle version as specified preceding.

If you need the `SYSDBA` privilege, use the following.

```
sqlplus asmuser/asmpassword@+asmserver as sysdba
```

If you need the `SYSASM` privilege, use the following. 

```
sqlplus asmuser/asmpassword@+asmserver as sysasm
```

### Using a self-managed Oracle Standby as a source with Binary Reader for CDC in AWS DMS
<a name="CHAP_Source.Oracle.Self-Managed.BinaryStandby"></a>

To configure an Oracle Standby instance as a source when using Binary Reader for CDC, start with the following prerequisites:
+ AWS DMS currently supports only Oracle Active Data Guard Standby.
+ Make sure that the Oracle Data Guard configuration uses:
  + Redo transport services for automated transfers of redo data.
  + Apply services to automatically apply redo to the standby database.

To confirm those requirements are met, execute the following query.

```
SQL> select open_mode, database_role from v$database;
```

From the output of that query, confirm that the standby database is opened in READ ONLY mode and redo is being applied automatically. For example:

```
OPEN_MODE             DATABASE_ROLE
--------------------  ----------------
READ ONLY WITH APPLY  PHYSICAL STANDBY
```

**To configure an Oracle Standby instance as a source when using Binary Reader for CDC**

1. Grant additional privileges required to access standby log files.

   ```
   GRANT SELECT ON v_$standby_log TO dms_user;
   ```

1. Create a source endpoint for the Oracle Standby by using the AWS Management Console or AWS CLI. When creating the endpoint, specify the following extra connection attributes.

   ```
   useLogminerReader=N;useBfile=Y;
   ```
**Note**  
In AWS DMS, you can use extra connection attributes to specify if you want to migrate from the archive logs instead of the redo logs. For more information, see [Endpoint settings when using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.ConnectionAttrib).

1. Configure archived log destination.

   DMS binary reader for Oracle source without ASM uses Oracle Directories to access archived redo logs. If your database is configured to use Fast Recovery Area (FRA) as an archive log destination, the location of archive redo files isn't constant. Each day that archived redo logs are generated results in a new directory being created in the FRA, using the directory name format YYYY\$1MM\$1DD. For example: 

   ```
   DB_RECOVERY_FILE_DEST/SID/archivelog/YYYY_MM_DD
   ```

   When DMS needs access to archived redo files in the newly created FRA directory and the primary read-write database is being used as a source, DMS creates a new or replaces an existing Oracle directory, as follows. 

   ```
   CREATE OR REPLACE DIRECTORY dmsrep_taskid AS ‘DB_RECOVERY_FILE_DEST/SID/archivelog/YYYY_MM_DD’;
   ```

   When the standby database is being used as a source, DMS is unable to create or replace the Oracle directory because the database is in read-only mode. But, you can choose to perform one of these additional steps: 

   1. Modify `log_archive_dest_id_1` to use an actual path instead of FRA in such a configuration that Oracle won't create daily subdirectories:

      ```
      ALTER SYSTEM SET log_archive_dest_1=’LOCATION=full directory path’
      ```

      Then, create an Oracle directory object to be used by DMS:

      ```
      CREATE OR REPLACE DIRECTORY dms_archived_logs AS ‘full directory path’;
      ```

   1. Create an additional archive log destination and an Oracle directory object pointing to that destination. For example:

      ```
      ALTER SYSTEM SET log_archive_dest_3=’LOCATION=full directory path’; 
      CREATE DIRECTORY dms_archived_log AS ‘full directory path’;
      ```

      Then add an extra connection attribute to the task source endpoint:

      ```
      archivedLogDestId=3
      ```

   1. Manually pre-create Oracle directory objects to be used by DMS.

      ```
      CREATE DIRECTORY dms_archived_log_20210301 AS ‘DB_RECOVERY_FILE_DEST/SID/archivelog/2021_03_01’;
      CREATE DIRECTORY dms_archived_log_20210302 AS ‘DB_RECOVERY_FILE_DEST>/SID>/archivelog/2021_03_02’; 
      ...
      ```

   1. Create an Oracle scheduler job that runs daily and creates the required directory.

1. Configure online log destination. 

   Create Oracle directory that points to OS directory with standby redo logs:

   ```
   CREATE OR REPLACE DIRECTORY STANDBY_REDO_DIR AS '<full directory path>';
   GRANT READ ON DIRECTORY STANDBY_REDO_DIR TO <dms_user>;
   ```

### Using a user-managed database on Oracle Cloud Infrastructure (OCI) as a source for CDC in AWS DMS
<a name="CHAP_Source.Oracle.Self-Managed.OCI"></a>

A user-managed database is a database that you configure and control, such as an Oracle database created on a virtual machine (VM), bare metal, or Exadata server. Or, databases that you configure and control that run on dedicated infrastructure, like Oracle Cloud Infrastructure (OCI). The following information describes the privileges and configurations you need when using an Oracle user-managed database on OCI as a source for change data capture (CDC) in AWS DMS.

**To configure an OCI hosted user-managed Oracle database as a source for change data capture**

1. Grant required user account privileges for a user-managed Oracle source database on OCI. For more information, see [Account privileges for a self-managed Oracle source endpoint](#CHAP_Source.Oracle.Self-Managed.Privileges).

1. Grant account privileges required when using Binary Reader to access the redo logs. For more information, see [Account privileges required when using Binary Reader](#CHAP_Source.Oracle.Self-Managed.BinaryReaderPrivileges).

1. Add account privileges that are required when using Binary Reader with Oracle Automatic Storage Management (ASM). For more information, see [Additional account privileges required when using Binary Reader with Oracle ASM](#CHAP_Source.Oracle.Self-Managed.ASMBinaryPrivileges).

1. Set-up supplemental logging. For more information, see [Setting up supplemental logging](#CHAP_Source.Oracle.Self-Managed.Configuration.SupplementalLogging).

1. Set-up TDE encryption. For more information, see [Encryption methods when using an Oracle database as a source endpoint](#CHAP_Source.Oracle.Encryption).

The following limitations apply when replicating data from an Oracle source database on Oracle Cloud Infrastructure (OCI).

**Limitations**
+ DMS does not support using Oracle LogMiner to access the redo logs.
+ DMS does not support Autonomous DB.

## Working with an AWS-managed Oracle database as a source for AWS DMS
<a name="CHAP_Source.Oracle.Amazon-Managed"></a>

An AWS-managed database is a database that is on an Amazon service such as Amazon RDS, Amazon Aurora, or Amazon S3. Following, you can find the privileges and configurations that you need to set up when using an AWS-managed Oracle database with AWS DMS.

### User account privileges required on an AWS-managed Oracle source for AWS DMS
<a name="CHAP_Source.Oracle.Amazon-Managed.Privileges"></a>

Grant the following privileges to the Oracle user account specified in the Oracle source endpoint definition.

**Important**  
For all parameter values such as `dms_user` and `any-replicated-table`, Oracle assumes the value is all uppercase unless you specify the value with a case-sensitive identifier. For example, suppose that you create a `dms_user` value without using quotation marks, as in `CREATE USER myuser` or `CREATE USER MYUSER`. In this case, Oracle identifies and stores the value as all uppercase (`MYUSER`). If you use quotation marks, as in `CREATE USER "MyUser"` or `CREATE USER 'MyUser'`, Oracle identifies and stores the case-sensitive value that you specify (`MyUser`).

```
GRANT CREATE SESSION to dms_user;
GRANT SELECT ANY TRANSACTION to dms_user;
GRANT SELECT on DBA_TABLESPACES to dms_user;
GRANT SELECT ON any-replicated-table to dms_user;
GRANT EXECUTE on rdsadmin.rdsadmin_util to dms_user;
 -- For Oracle 12c or higher:
GRANT LOGMINING to dms_user; – Required only if the Oracle version is 12c or higher.
```

In addition, grant `SELECT` and `EXECUTE` permissions on `SYS` objects using the Amazon RDS procedure `rdsadmin.rdsadmin_util.grant_sys_object` as shown. For more information, see [Granting SELECT or EXECUTE privileges to SYS objects](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.Oracle.CommonDBATasks.html#Appendix.Oracle.CommonDBATasks.TransferPrivileges).

```
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_VIEWS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_TAB_PARTITIONS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_INDEXES', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_OBJECTS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_TABLES', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_USERS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_CATALOG', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_CONSTRAINTS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_CONS_COLUMNS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_TAB_COLS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_IND_COLUMNS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_LOG_GROUPS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$ARCHIVED_LOG', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$LOG', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$LOGFILE', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$DATABASE', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$THREAD', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$PARAMETER', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$NLS_PARAMETERS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$TIMEZONE_NAMES', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$TRANSACTION', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$CONTAINERS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('DBA_REGISTRY', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('OBJ$', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('ALL_ENCRYPTED_COLUMNS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$LOGMNR_LOGS', 'dms_user', 'SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$LOGMNR_CONTENTS','dms_user','SELECT');
exec rdsadmin.rdsadmin_util.grant_sys_object('DBMS_LOGMNR', 'dms_user', 'EXECUTE');

-- (as of Oracle versions 12.1 and higher)
exec rdsadmin.rdsadmin_util.grant_sys_object('REGISTRY$SQLPATCH', 'dms_user', 'SELECT');

-- (for Amazon RDS Active Dataguard Standby (ADG))
exec rdsadmin.rdsadmin_util.grant_sys_object('V_$STANDBY_LOG', 'dms_user', 'SELECT'); 

-- (for transparent data encryption (TDE))

exec rdsadmin.rdsadmin_util.grant_sys_object('ENC$', 'dms_user', 'SELECT'); 
               
-- (for validation with LOB columns)
exec rdsadmin.rdsadmin_util.grant_sys_object('DBMS_CRYPTO', 'dms_user', 'EXECUTE');
                    
-- (for binary reader)
exec rdsadmin.rdsadmin_util.grant_sys_object('DBA_DIRECTORIES','dms_user','SELECT'); 
                    
-- Required when the source database is Oracle Data guard, and Oracle Standby is used in the latest release of DMS version 3.4.6, version 3.4.7, and higher.

exec rdsadmin.rdsadmin_util.grant_sys_object('V_$DATAGUARD_STATS', 'dms_user', 'SELECT');
```

For more information on using Amazon RDS Active Dataguard Standby (ADG) with AWS DMS see [Using an Amazon RDS Oracle Standby (read replica) as a source with Binary Reader for CDC in AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.StandBy).

For more information on using Oracle TDE with AWS DMS, see [Supported encryption methods for using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.Encryption).

#### Prerequisites for handling open transactions for Oracle Standby
<a name="CHAP_Source.Oracle.Amazon-Managed.Privileges.Standby"></a>

When using AWS DMS versions 3.4.6 and higher, perform the following steps to handle open transactions for Oracle Standby. 

1. Create a database link named, `AWSDMS_DBLINK` on the primary database. `DMS_USER` will use the database link to connect to the primary database. Note that the database link is executed from the standby instance to query the open transactions running on the primary database. See the following example. 

   ```
   CREATE PUBLIC DATABASE LINK AWSDMS_DBLINK 
      CONNECT TO DMS_USER IDENTIFIED BY DMS_USER_PASSWORD
      USING '(DESCRIPTION=
               (ADDRESS=(PROTOCOL=TCP)(HOST=PRIMARY_HOST_NAME_OR_IP)(PORT=PORT))
               (CONNECT_DATA=(SERVICE_NAME=SID))
             )';
   ```

1. Verify the connection to the database link using `DMS_USER` is established, as shown in the following example.

   ```
   select 1 from dual@AWSDMS_DBLINK
   ```

### Configuring an AWS-managed Oracle source for AWS DMS
<a name="CHAP_Source.Oracle.Amazon-Managed.Configuration"></a>

Before using an AWS-managed Oracle database as a source for AWS DMS, perform the following tasks for the Oracle database:
+ Enable automatic backups. For more information about enabling automatic backups, see [Enabling automated backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html#USER_WorkingWithAutomatedBackups.Enabling) in the *Amazon RDS User Guide*.
+ Set up supplemental logging.
+ Set up archiving. Archiving the redo logs for your Amazon RDS for Oracle DB instance allows AWS DMS to retrieve the log information using Oracle LogMiner or Binary Reader. 

**To set up archiving**

1. Run the `rdsadmin.rdsadmin_util.set_configuration` command to set up archiving.

   For example, to retain the archived redo logs for 24 hours, run the following command.

   ```
   exec rdsadmin.rdsadmin_util.set_configuration('archivelog retention hours',24);
   commit;
   ```
**Note**  
The commit is required for a change to take effect.

1. Make sure that your storage has enough space for the archived redo logs during the specified retention period. For example, if your retention period is 24 hours, calculate the total size of your accumulated archived redo logs over a typical hour of transaction processing and multiply that total by 24. Compare this calculated 24-hour total with your available storage space and decide if you have enough storage space to handle a full 24 hours transaction processing.

**To set up supplemental logging**

1. Run the following command to enable supplemental logging at the database level.

   ```
   exec rdsadmin.rdsadmin_util.alter_supplemental_logging('ADD');
   ```

1. Run the following command to enable primary key supplemental logging.

   ```
   exec rdsadmin.rdsadmin_util.alter_supplemental_logging('ADD','PRIMARY KEY');
   ```

1. (Optional) Enable key-level supplemental logging at the table level.

   Your source database incurs a small bit of overhead when key-level supplemental logging is enabled. Therefore, if you are migrating only a subset of your tables, you might want to enable key-level supplemental logging at the table level. To enable key-level supplemental logging at the table level, run the following command.

   ```
   alter table table_name add supplemental log data (PRIMARY KEY) columns;
   ```

### Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS
<a name="CHAP_Source.Oracle.Amazon-Managed.CDC"></a>

You can configure AWS DMS to access the source Amazon RDS for Oracle instance redo logs using Binary Reader for CDC. 

**Note**  
To use Oracle LogMiner, the minimum required user account privileges are sufficient. For more information, see [User account privileges required on an AWS-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.Privileges).

To use AWS DMS Binary Reader, specify additional settings and extra connection attributes for the Oracle source endpoint, depending on your AWS DMS version.

Binary Reader support is available in the following versions of Amazon RDS for Oracle:
+ Oracle 11.2 – Versions 11.2.0.4V11 and higher
+ Oracle 12.1 – Versions 12.1.0.2.V7 and higher
+ Oracle 12.2 – All versions
+ Oracle 18.0 – All versions
+ Oracle 19.0 – All versions

**To configure CDC using Binary Reader**

1. Log in to your Amazon RDS for Oracle source database as the master user and run the following stored procedures to create the server-level directories.

   ```
   exec rdsadmin.rdsadmin_master_util.create_archivelog_dir;
   exec rdsadmin.rdsadmin_master_util.create_onlinelog_dir;
   ```

1. Grant the following privileges to the Oracle user account that is used to access the Oracle source endpoint.

   ```
   GRANT READ ON DIRECTORY ONLINELOG_DIR TO dms_user;
   GRANT READ ON DIRECTORY ARCHIVELOG_DIR TO dms_user;
   ```

1. Set the following extra connection attributes on the Amazon RDS Oracle source endpoint:
   + For RDS Oracle versions 11.2 and 12.1, set the following.

     ```
     useLogminerReader=N;useBfile=Y;accessAlternateDirectly=false;useAlternateFolderForOnline=true;
     oraclePathPrefix=/rdsdbdata/db/{$DATABASE_NAME}_A/;usePathPrefix=/rdsdbdata/log/;replacePathPrefix=true;
     ```
   + For RDS Oracle versions 12.2, 18.0, and 19.0, set the following.

     ```
     useLogminerReader=N;useBfile=Y;
     ```

**Note**  
Make sure there's no white space following the semicolon separator (;) for multiple attribute settings, for example `oneSetting;thenAnother`.

For more information configuring a CDC task, see [Configuration for CDC on an Oracle source database](#CHAP_Source.Oracle.CDC.Configuration).

### Using an Amazon RDS Oracle Standby (read replica) as a source with Binary Reader for CDC in AWS DMS
<a name="CHAP_Source.Oracle.Amazon-Managed.StandBy"></a>

Verify the following prerequisites for using Amazon RDS for Oracle Standby as a source when using Binary Reader for CDC in AWS DMS:
+ Use the Oracle master user to set up Binary Reader.
+ Make sure that AWS DMS currently supports using only Oracle Active Data Guard Standby.

After you do so, use the following procedure to use RDS for Oracle Standby as a source when using Binary Reader for CDC.

**To configure an RDS for Oracle Standby as a source when using Binary Reader for CDC**

1. Sign in to RDS for Oracle primary instance as the master user.

1. Run the following stored procedures as documented in the Amazon RDS User Guide to create the server level directories.

   ```
   exec rdsadmin.rdsadmin_master_util.create_archivelog_dir;
   exec rdsadmin.rdsadmin_master_util.create_onlinelog_dir;
   ```

1. Identify the directories created in step 2.

   ```
   SELECT directory_name, directory_path FROM all_directories
   WHERE directory_name LIKE ( 'ARCHIVELOG_DIR_%' )
           OR directory_name LIKE ( 'ONLINELOG_DIR_%' )
   ```

   For example, the preceding code displays a list of directories like the following.  
![\[Table showing directory names and their corresponding paths for archive and online logs.\]](http://docs.aws.amazon.com/dms/latest/userguide/images/datarep-rds-server-level-directories.png)

1. Grant the `Read` privilege on the preceding directories to the Oracle user account that is used to access the Oracle Standby.

   ```
   GRANT READ ON DIRECTORY ARCHIVELOG_DIR_A TO dms_user;
   GRANT READ ON DIRECTORY ARCHIVELOG_DIR_B TO dms_user;
   GRANT READ ON DIRECTORY ONLINELOG_DIR_A TO dms_user;
   GRANT READ ON DIRECTORY ONLINELOG_DIR_B TO dms_user;
   ```

1. Perform an archive log switch on the primary instance. Doing this makes sure that the changes to `ALL_DIRECTORIES` are also ported to the Oracle Standby.

1. Run an `ALL_DIRECTORIES` query on the Oracle Standby to confirm that the changes were applied.

1. Create a source endpoint for the Oracle Standby by using the AWS DMS Management Console or AWS Command Line Interface (AWS CLI). While creating the endpoint, specify the following extra connection attributes.

   ```
   useLogminerReader=N;useBfile=Y;archivedLogDestId=1;additionalArchivedLogDestId=2
   ```

1. After creating the endpoint, use **Test endpoint connection** on the **Create endpoint** page of the console or the AWS CLI `test-connection` command to verify that connectivity is established.

## Limitations on using Oracle as a source for AWS DMS
<a name="CHAP_Source.Oracle.Limitations"></a>

The following limitations apply when using an Oracle database as a source for AWS DMS:
+ AWS DMS supports Oracle Extended data types in AWS DMS version 3.5.0 and higher.
+ AWS DMS does not support Oracle identifiers longer than 30 bytes during CDC replication. This limitation applies regardless of whether the Oracle source version supports extended identifiers.

  During full load, AWS DMS migrates objects with long identifiers successfully. However, during CDC, events for objects with identifiers exceeding 30 bytes are silently skipped without stopping the task. This can result in missing data on the target.

  This limitation applies to all identifier types in the migration scope, such as schemas, tables, views, columns, constraints, and primary keys.
**Note**  
The 30-byte limit is measured in bytes, not characters. Identifiers that use multibyte character sets, such as UTF-8 encoded CJK characters, can exceed 30 bytes with fewer than 30 characters. For example, a 15-character Japanese identifier encoded in UTF-8 can require up to 45 bytes.

  To avoid data loss during CDC replication, verify that all identifiers in the migration scope are within the 30-byte limit. To check the byte length of an object name, use the Oracle built-in function `LENGTHB('object-name')`. Alternatively, run a premigration assessment to validate all objects in the migration scope. For more information, see [Validate the length of the object name included in the task scope](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.AssessmentReport.Oracle.html#CHAP_Tasks.AssessmentReport.Oracle.30ByteLimit).
+ AWS DMS does not support function-based indexes.
+ If you manage supplemental logging and carry out transformations on any of the columns, make sure that supplemental logging is activated for all fields and columns. For more information on setting up supplemental logging, see the following topics:
  + For a self-managed Oracle source database, see [Setting up supplemental logging](#CHAP_Source.Oracle.Self-Managed.Configuration.SupplementalLogging).
  + For an AWS-managed Oracle source database, see [Configuring an AWS-managed Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.Configuration).
+ AWS DMS does not support the multi-tenant container root database (CDB\$1ROOT). It does support a PDB using the Binary Reader.
+ AWS DMS does not support deferred constraints.
+ AWS DMS version 3.5.3 and above fully supports secure LOBs.
+ AWS DMS supports the `rename table table-name to new-table-name` syntax for all supported Oracle versions 11 and higher. This syntax isn't supported for any Oracle version 10 source databases.
+ AWS DMS does not replicate results of the DDL statement `ALTER TABLE ADD column data_type DEFAULT default_value`. Instead of replicating `default_value` to the target, it sets the new column to `NULL`.
+ When using AWS DMS version 3.4.7 or higher, to replicate changes that result from partition or subpartition operations, do the following before starting a DMS task.
  + Manually create the partitioned table structure (DDL); 
  + Make sure the DDL is the same on both Oracle source and Oracle target; 
  + Set the extra connection attribute `enableHomogenousPartitionOps=true`.

  For more information about `enableHomogenousPartitionOps`, see [Endpoint settings when using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.ConnectionAttrib). Also, note that on FULL\$1CDC tasks, DMS does not replicate data changes captured as part of the cached changes. In that use case, recreate the table structure on the Oracle target and reload the tables in question.

  Prior to AWS DMS version 3.4.7:

  DMS does not replicate data changes that result from partition or subpartition operations (`ADD`, `DROP`, `EXCHANGE`, and `TRUNCATE`). Such updates might cause the following errors during replication:
  + For `ADD` operations, updates and deletes on the added data might raise a "0 rows affected" warning.
  + For `DROP` and `TRUNCATE` operations, new inserts might raise "duplicates" errors.
  + `EXCHANGE` operations might raise both a "0 rows affected" warning and "duplicates" errors.

  To replicate changes that result from partition or subpartition operations, reload the tables in question. After adding a new empty partition, operations on the newly added partition are replicated to the target as normal.
+ AWS DMS versions prior to 3.4 don't support data changes on the target that result from running the `CREATE TABLE AS` statement on the source. However, the new table is created on the target.
+ AWS DMS does not capture changes made by the Oracle `DBMS_REDEFINITION` package, for example the table metadata and the `OBJECT_ID` field.
+ When Limited-size LOB mode is enabled, empty BLOB/CLOB columns on the Oracle source are replicated as NULL values. When Full LOB mode is enabled, they are replicated as an empty string (' ').
+ When capturing changes with Oracle 11 LogMiner, an update on a CLOB column with a string length greater than 1982 is lost, and the target is not updated.
+ During change data capture (CDC), AWS DMS does not support batch updates to numeric columns defined as a primary key.
+ AWS DMS does not support certain `UPDATE` commands. The following example is an unsupported `UPDATE` command.

  ```
  UPDATE TEST_TABLE SET KEY=KEY+1;
  ```

  Here, `TEST_TABLE` is the table name and `KEY` is a numeric column defined as a primary key.
+ AWS DMS does not support full LOB mode for loading LONG and LONG RAW columns. Instead, you can use limited LOB mode for migrating these datatypes to an Oracle target. In limited LOB mode, AWS DMS truncates any data to 64 KB that you set to LONG or LONG RAW columns longer than 64 KB.
+ AWS DMS does not support full LOB mode for loading XMLTYPE columns. Instead, you can use limited LOB mode for migrating XMLTYPE columns to an Oracle target. In limited LOB mode, DMS truncates any data larger than the user defined 'Maximum LOB size' variable. The maximum recommended value for 'Maximum LOB size' is 100MB.
+ AWS DMS does not replicate tables whose names contain apostrophes.
+ AWS DMS supports CDC from materialized views. But DMS does not support CDC from any other views.
+ AWS DMS does not support CDC for index-organized tables with an overflow segment.
+ AWS DMS does not support the `Drop Partition` operation for tables partitioned by reference with `enableHomogenousPartitionOps` set to `true`.
+ When you use Oracle LogMiner to access the redo logs, AWS DMS has the following limitations:
  + For Oracle 12 only, AWS DMS does not replicate any changes to LOB columns.
  + AWS DMS does not support XA transactions in replication while using Oracle LogMiner.
  + Oracle LogMiner does not support connections to a pluggable database (PDB). To connect to a PDB, access the redo logs using Binary Reader.
  + SHRINK SPACE operations aren’t supported.
+ When you use Binary Reader, AWS DMS has these limitations:
  + It does not support table clusters.
  + It supports only table-level `SHRINK SPACE` operations. This level includes the full table, partitions, and sub-partitions.
  + It does not support changes to index-organized tables with key compression.
  + It does not support implementing online redo logs on raw devices.
  + Binary Reader supports TDE only for self-managed Oracle databases since RDS for Oracle does not support wallet password retrieval for TDE encryption keys.
+ AWS DMS does not support connections to an Amazon RDS Oracle source using an Oracle Automatic Storage Management (ASM) proxy.
+ AWS DMS does not support virtual columns. 
+ AWS DMS does not support the `ROWID` data type or materialized views based on a ROWID column.

  AWS DMS has partial support for Oracle Materialized Views. For full-loads, DMS can do a full-load copy of an Oracle Materialized View. DMS copies the Materialized View as a base table to the target system and ignores any ROWID columns in the Materialized View. For ongoing replication (CDC), DMS tries to replicate changes to the Materialized View data but the results might not be ideal. Specifically, if the Materialized View is completely refreshed, DMS replicates individual deletes for all the rows, followed by individual inserts for all the rows. That is a very resource intensive exercise and might perform poorly for materialized views with large numbers of rows. For ongoing replication where the materialized views do a fast refresh, DMS tries to process and replicate the fast refresh data changes. In either case, DMS skips any ROWID columns in the materialized view.
+ AWS DMS does not load or capture global temporary tables.
+ For S3 targets using replication, enable supplemental logging on every column so source row updates can capture every column value. An example follows: `alter table yourtablename add supplemental log data (all) columns;`.
+ An update for a row with a composite unique key that contains `null` can't be replicated at the target.
+ AWS DMS does not support use of multiple Oracle TDE encryption keys on the same source endpoint. Each endpoint can have only one attribute for TDE encryption Key Name "`securityDbEncryptionName`", and one TDE password for this key.
+ When replicating from Amazon RDS for Oracle, TDE is supported only with encrypted tablespace and using Oracle LogMiner.
+ AWS DMS does not support multiple table rename operations in quick succession.
+ When using Oracle 19.0 as source, AWS DMS does not support the following features:
  + Data-guard DML redirect
  + Partitioned hybrid tables
  + Schema-only Oracle accounts
+ AWS DMS does not support migration of tables or views of type `BIN$` or `DR$`.
+ Beginning with Oracle 18.x, AWS DMS does not support change data capture (CDC) from Oracle Express Edition (Oracle Database XE).
+ When migrating data from a CHAR column, DMS truncates any trailing spaces. 
+ AWS DMS does not support replication from application containers.
+ AWS DMS does not support performing Oracle Flashback Database and restore points, as these operations affect the consistency of Oracle Redo Log files.
+ Prior to AWS DMS version 3.5.3, Direct-load `INSERT` procedure with the parallel execution option is not supported in the following cases:
  + Uncompressed tables with more than 255 columns
  + Row size exceeds 8K
  + Exadata HCC tables
  + Database running on Big Endian platform
+ A source table with neither primary nor unique key requires ALL COLUMN supplemental logging to be enabled. It creates more redo log activities and may increase DMS CDC latency.
+ AWS DMS does not migrate data from invisible columns in your source database. To include these columns in your migration scope, use the `ALTER TABLE` statement to make these columns visible.
+ For all Oracle versions, AWS DMS does not replicate the result of `UPDATE` operations on `XMLTYPE` and LOB columns.
+ AWS DMS does not support replication from tables with temporal validity constraints.
+ If the Oracle source becomes unavailable during a full load task, AWS DMS might mark the task as completed after multiple reconnection attempts, even though the data migration remains incomplete. In this scenario, the target tables contain only the records migrated before the connection loss, potentially creating data inconsistencies between the source and target systems. To ensure data completeness, you must either restart the full load task entirely or reload the specific tables affected by the connection interruption.

## SSL support for an Oracle endpoint
<a name="CHAP_Security.SSL.Oracle"></a>

AWS DMS Oracle endpoints support SSL V3 for the `none` and `verify-ca` SSL modes. To use SSL with an Oracle endpoint, upload the Oracle wallet for the endpoint instead of .pem certificate files. 

**Topics**
+ [

### Using an existing certificate for Oracle SSL
](#CHAP_Security.SSL.Oracle.Existing)
+ [

### Using a self-signed certificate for Oracle SSL
](#CHAP_Security.SSL.Oracle.SelfSigned)

### Using an existing certificate for Oracle SSL
<a name="CHAP_Security.SSL.Oracle.Existing"></a>

To use an existing Oracle client installation to create the Oracle wallet file from the CA certificate file, do the following steps.

**To use an existing oracle client installation for Oracle SSL with AWS DMS**

1. Set the `ORACLE_HOME` system variable to the location of your `dbhome_1` directory by running the following command.

   ```
   prompt>export ORACLE_HOME=/home/user/app/user/product/12.1.0/dbhome_1                        
   ```

1. Append `$ORACLE_HOME/lib` to the `LD_LIBRARY_PATH` system variable.

   ```
   prompt>export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME/lib                        
   ```

1. Create a directory for the Oracle wallet at `$ORACLE_HOME/ssl_wallet`.

   ```
   prompt>mkdir $ORACLE_HOME/ssl_wallet
   ```

1. Put the CA certificate `.pem` file in the `ssl_wallet` directory. If you use Amazon RDS, you can download the `rds-ca-2015-root.pem` root CA certificate file hosted by Amazon RDS. For more information about downloading this file, see [Using SSL/TLS to encrypt a connection to a DB instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html) in the *Amazon RDS User Guide*.

1. If your CA certificate contains more than one PEM file (like Amazon RDS global or regional bundle), you must split it into separate files and add them into the Oracle wallet using following bash script. This script requires two parameter inputs: the path to the CA certificate and the path to the folder of the previously created Oracle wallet.

   ```
   #!/usr/bin/env bash
   
   certnum=$(grep -c BEGIN <(cat $1))
   
   cnt=0
   temp_cert=""
   while read line
   do
   if [ -n "$temp_cert" -a "$line" == "-----BEGIN CERTIFICATE-----" ]
   then
   cnt=$(expr $cnt + 1)
   printf "\rImporting certificate # $cnt of $certnum"
   orapki wallet add -wallet "$2" -trusted_cert -cert <(echo -n "${temp_cert}") -auto_login_only 1>/dev/null 2>/dev/null
   temp_cert=""
   fi
   temp_cert+="$line"$'\n'
   done < <(cat $1)
   
   cnt=$(expr $cnt + 1)
   printf "\rImporting certificate # $cnt of $certnum"
   orapki wallet add -wallet "$2" -trusted_cert -cert <(echo -n "${temp_cert}") -auto_login_only 1>/dev/null 2>/dev/null
   echo ""
   ```

When you have completed the steps previous, you can import the wallet file with the `ImportCertificate` API call by specifying the certificate-wallet parameter. You can then use the imported wallet certificate when you select `verify-ca` as the SSL mode when creating or modifying your Oracle endpoint.

**Note**  
 Oracle wallets are binary files. AWS DMS accepts these files as-is. 

### Using a self-signed certificate for Oracle SSL
<a name="CHAP_Security.SSL.Oracle.SelfSigned"></a>

To use a self-signed certificate for Oracle SSL, do the steps following, assuming an Oracle wallet password of `oracle123`.

**To use a self-signed certificate for Oracle SSL with AWS DMS**

1. Create a directory you will use to work with the self-signed certificate.

   ```
   mkdir -p /u01/app/oracle/self_signed_cert
   ```

1. Change into the directory you created in the previous step.

   ```
   cd /u01/app/oracle/self_signed_cert
   ```

1. Create a root key.

   ```
   openssl genrsa -out self-rootCA.key 2048
   ```

1. Self-sign a root certificate using the root key you created in the previous step.

   ```
   openssl req -x509 -new -nodes -key self-rootCA.key 
           -sha256 -days 3650 -out self-rootCA.pem
   ```

   Use input parameters like the following.
   + `Country Name (2 letter code) [XX]`, for example: `AU`
   + `State or Province Name (full name) []`, for example: `NSW`
   + `Locality Name (e.g., city) [Default City]`, for example: `Sydney`
   + `Organization Name (e.g., company) [Default Company Ltd]`, for example: `AmazonWebService`
   + `Organizational Unit Name (e.g., section) []`, for example: `DBeng`
   + `Common Name (e.g., your name or your server's hostname) []`, for example: `aws`
   + `Email Address []`, for example: abcd.efgh@amazonwebservice.com

1. Create an Oracle wallet directory for the Oracle database.

   ```
   mkdir -p /u01/app/oracle/wallet
   ```

1. Create a new Oracle wallet.

   ```
   orapki wallet create -wallet "/u01/app/oracle/wallet" -pwd oracle123 -auto_login_local
   ```

1. Add the root certificate to the Oracle wallet.

   ```
   orapki wallet add -wallet "/u01/app/oracle/wallet" -pwd oracle123 -trusted_cert 
   -cert /u01/app/oracle/self_signed_cert/self-rootCA.pem
   ```

1. List the contents of the Oracle wallet. The list should include the root certificate. 

   ```
   orapki wallet display -wallet /u01/app/oracle/wallet -pwd oracle123
   ```

   For example, this might display similar to the following.

   ```
   Requested Certificates:
   User Certificates:
   Trusted Certificates:
   Subject:        CN=aws,OU=DBeng,O= AmazonWebService,L=Sydney,ST=NSW,C=AU
   ```

1. Generate the Certificate Signing Request (CSR) using the ORAPKI utility.

   ```
   orapki wallet add -wallet "/u01/app/oracle/wallet" -pwd oracle123 
   -dn "CN=aws" -keysize 2048 -sign_alg sha256
   ```

1. Run the following command.

   ```
   openssl pkcs12 -in /u01/app/oracle/wallet/ewallet.p12 -nodes -out /u01/app/oracle/wallet/nonoracle_wallet.pem
   ```

   This has output like the following.

   ```
   Enter Import Password:
   MAC verified OK
   Warning unsupported bag type: secretBag
   ```

1. Put 'dms' as the common name.

   ```
   openssl req -new -key /u01/app/oracle/wallet/nonoracle_wallet.pem -out certdms.csr
   ```

   Use input parameters like the following.
   + `Country Name (2 letter code) [XX]`, for example: `AU`
   + `State or Province Name (full name) []`, for example: `NSW`
   + `Locality Name (e.g., city) [Default City]`, for example: `Sydney`
   + `Organization Name (e.g., company) [Default Company Ltd]`, for example: `AmazonWebService`
   + `Organizational Unit Name (e.g., section) []`, for example: `aws`
   + `Common Name (e.g., your name or your server's hostname) []`, for example: `aws`
   + `Email Address []`, for example: abcd.efgh@amazonwebservice.com

   Make sure this is not same as step 4. You can do this, for example, by changing Organizational Unit Name to a different name as shown.

   Enter the additional attributes following to be sent with your certificate request.
   + `A challenge password []`, for example: `oracle123`
   + `An optional company name []`, for example: `aws`

1. Get the certificate signature.

   ```
   openssl req -noout -text -in certdms.csr | grep -i signature
   ```

   The signature key for this post is `sha256WithRSAEncryption` .

1. Run the command following to generate the certificate (`.crt`) file.

   ```
   openssl x509 -req -in certdms.csr -CA self-rootCA.pem -CAkey self-rootCA.key 
   -CAcreateserial -out certdms.crt -days 365 -sha256
   ```

   This displays output like the following.

   ```
   Signature ok
   subject=/C=AU/ST=NSW/L=Sydney/O=awsweb/OU=DBeng/CN=aws
   Getting CA Private Key
   ```

1. Add the certificate to the wallet.

   ```
   orapki wallet add -wallet /u01/app/oracle/wallet -pwd oracle123 -user_cert -cert certdms.crt
   ```

1. View the wallet. It should have two entries. See the code following.

   ```
   orapki wallet display -wallet /u01/app/oracle/wallet -pwd oracle123
   ```

1. Configure the `sqlnet.ora` file (`$ORACLE_HOME/network/admin/sqlnet.ora`).

   ```
   WALLET_LOCATION =
      (SOURCE =
        (METHOD = FILE)
        (METHOD_DATA =
          (DIRECTORY = /u01/app/oracle/wallet/)
        )
      ) 
   
   SQLNET.AUTHENTICATION_SERVICES = (NONE)
   SSL_VERSION = 1.0
   SSL_CLIENT_AUTHENTICATION = FALSE
   SSL_CIPHER_SUITES = (SSL_RSA_WITH_AES_256_CBC_SHA)
   ```

1. Stop the Oracle listener.

   ```
   lsnrctl stop
   ```

1. Add entries for SSL in the `listener.ora` file (`$ORACLE_HOME/network/admin/listener.ora`).

   ```
   SSL_CLIENT_AUTHENTICATION = FALSE
   WALLET_LOCATION =
     (SOURCE =
       (METHOD = FILE)
       (METHOD_DATA =
         (DIRECTORY = /u01/app/oracle/wallet/)
       )
     )
   
   SID_LIST_LISTENER =
    (SID_LIST =
     (SID_DESC =
      (GLOBAL_DBNAME = SID)
      (ORACLE_HOME = ORACLE_HOME)
      (SID_NAME = SID)
     )
    )
   
   LISTENER =
     (DESCRIPTION_LIST =
       (DESCRIPTION =
         (ADDRESS = (PROTOCOL = TCP)(HOST = localhost.localdomain)(PORT = 1521))
         (ADDRESS = (PROTOCOL = TCPS)(HOST = localhost.localdomain)(PORT = 1522))
         (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC1521))
       )
     )
   ```

1. Configure the `tnsnames.ora` file (`$ORACLE_HOME/network/admin/tnsnames.ora`).

   ```
   <SID>=
   (DESCRIPTION=
           (ADDRESS_LIST = 
                   (ADDRESS=(PROTOCOL = TCP)(HOST = localhost.localdomain)(PORT = 1521))
           )
           (CONNECT_DATA =
                   (SERVER = DEDICATED)
                   (SERVICE_NAME = <SID>)
           )
   )
   
   <SID>_ssl=
   (DESCRIPTION=
           (ADDRESS_LIST = 
                   (ADDRESS=(PROTOCOL = TCPS)(HOST = localhost.localdomain)(PORT = 1522))
           )
           (CONNECT_DATA =
                   (SERVER = DEDICATED)
                   (SERVICE_NAME = <SID>)
           )
   )
   ```

1. Restart the Oracle listener.

   ```
   lsnrctl start
   ```

1. Show the Oracle listener status.

   ```
   lsnrctl status
   ```

1. Test the SSL connection to the database from localhost using sqlplus and the SSL tnsnames entry.

   ```
   sqlplus -L ORACLE_USER@SID_ssl
   ```

1. Verify that you successfully connected using SSL.

   ```
   SELECT SYS_CONTEXT('USERENV', 'network_protocol') FROM DUAL;
   
   SYS_CONTEXT('USERENV','NETWORK_PROTOCOL')
   --------------------------------------------------------------------------------
   tcps
   ```

1. Change directory to the directory with the self-signed certificate.

   ```
   cd /u01/app/oracle/self_signed_cert
   ```

1. Create a new client Oracle wallet for AWS DMS to use.

   ```
   orapki wallet create -wallet ./ -auto_login_only
   ```

1. Add the self-signed root certificate to the Oracle wallet.

   ```
   orapki wallet add -wallet ./ -trusted_cert -cert self-rootCA.pem -auto_login_only
   ```

1. List the contents of the Oracle wallet for AWS DMS to use. The list should include the self-signed root certificate.

   ```
   orapki wallet display -wallet ./
   ```

   This has output like the following.

   ```
   Trusted Certificates:
   Subject:        CN=aws,OU=DBeng,O=AmazonWebService,L=Sydney,ST=NSW,C=AU
   ```

1. Upload the Oracle wallet that you just created to AWS DMS.

## Supported encryption methods for using Oracle as a source for AWS DMS
<a name="CHAP_Source.Oracle.Encryption"></a>

In the following table, you can find the transparent data encryption (TDE) methods that AWS DMS supports when working with an Oracle source database. 


| Redo logs access method | TDE tablespace | TDE column | 
| --- | --- | --- | 
| Oracle LogMiner | Yes | Yes | 
| Binary Reader | Yes | Yes | 

AWS DMS supports Oracle TDE when using Binary Reader, on both the column level and the tablespace level. To use TDE encryption with AWS DMS, first identify the Oracle wallet location where the TDE encryption key and TDE password are stored. Then identify the correct TDE encryption key and password for your Oracle source endpoint.

**To identify and specify encryption key and password for TDE encryption**

1. Run the following query to find the Oracle encryption wallet on the Oracle database host.

   ```
   SQL> SELECT WRL_PARAMETER FROM V$ENCRYPTION_WALLET;
   
   WRL_PARAMETER
   --------------------------------------------------------------------------------
   /u01/oracle/product/12.2.0/dbhome_1/data/wallet/
   ```

   Here, `/u01/oracle/product/12.2.0/dbhome_1/data/wallet/` is the wallet location.

1. Get the master key ID for either Non-CDB or CDB source as follows:

   1. For non-CDB source run the following query to retrieve Master encryption key ID:

      ```
      SQL>  select rownum, key_id, activation_time from v$encryption_keys;
      
      ROWNUM KEY_ID                                                 ACTIVATION_TIME
      ------ ------------------------------------------------------ ---------------
           1 AeKask0XZU+NvysflCYBEVwAAAAAAAAAAAAAAAAAAAAAAAAAAAAA   04-SEP-24 10.20.56.605200 PM +00:00
           2 AV7WU9uhoU8rv8daE/HNnSwAAAAAAAAAAAAAAAAAAAAAAAAAAAAA   10-AUG-21 07.52.03.966362 PM +00:00
           3 AckpoJ/f+k8xvzJ+gSuoVH4AAAAAAAAAAAAAAAAAAAAAAAAAAAAA   14-SEP-20 09.26.29.048870 PM +00:00
      ```

      Activation time is useful if you plan to start CDC from some point in the past. For example, using the above results, you can start CDC from some point between 10-AUG-21 07.52.03 PM and 14-SEP-20 09.26.29 PM using the Master Key ID in ROWNUM 2. When the task reaches the redo generated on or after 14-SEP-20 09.26.29 PM it fails, you must modify the source endpoint, provide the Master key ID in ROWNUM 3, and then resume the task.

   1. For CDB source DMS requires CDB\$1ROOT Master encryption key. Connect to CDB\$1ROOT and execute the following query:

      ```
      SQL> select rownum, key_id, activation_time from v$encryption_keys where con_id = 1;
      
      ROWNUM KEY_ID                                               ACTIVATION_TIME
      ------ ---------------------------------------------------- -----------------------------------
           1 Aa2E/Vwb5U+zv5hCncS5ErMAAAAAAAAAAAAAAAAAAAAAAAAAAAAA 29-AUG-24 12.51.19.699060 AM +00:00
      ```

1. From the command line, list the encryption wallet entries on the source Oracle database host.

   ```
   $ mkstore -wrl /u01/oracle/product/12.2.0/dbhome_1/data/wallet/ -list
   Oracle Secret Store entries:
   ORACLE.SECURITY.DB.ENCRYPTION.AWGDC9glSk8Xv+3bVveiVSgAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
   ORACLE.SECURITY.DB.ENCRYPTION.AY1mRA8OXU9Qvzo3idU4OH4AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
   ORACLE.SECURITY.DB.ENCRYPTION.MASTERKEY
   ORACLE.SECURITY.ID.ENCRYPTION.
   ORACLE.SECURITY.KB.ENCRYPTION.
   ORACLE.SECURITY.KM.ENCRYPTION.AY1mRA8OXU9Qvzo3idU4OH4AAAAAAAAAAAAAAAAAAAAAAAAAAAAA
   ```

   Find the entry containing the master key ID that you found in step 2 (`AWGDC9glSk8Xv+3bVveiVSg`). This entry is the TDE encryption key name.

1. View the details of the entry that you found in the previous step.

   ```
   $ mkstore -wrl /u01/oracle/product/12.2.0/dbhome_1/data/wallet/ -viewEntry ORACLE.SECURITY.DB.ENCRYPTION.AWGDC9glSk8Xv+3bVveiVSgAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
   Oracle Secret Store Tool : Version 12.2.0.1.0
   Copyright (c) 2004, 2016, Oracle and/or its affiliates. All rights reserved.
   Enter wallet password:
   ORACLE.SECURITY.DB.ENCRYPTION.AWGDC9glSk8Xv+3bVveiVSgAAAAAAAAAAAAAAAAAAAAAAAAAAAAA = AEMAASAASGYs0phWHfNt9J5mEMkkegGFiD4LLfQszDojgDzbfoYDEACv0x3pJC+UGD/PdtE2jLIcBQcAeHgJChQGLA==
   ```

   Enter the wallet password to see the result.

   Here, the value to the right of `'='` is the TDE password.

1. Specify the TDE encryption key name for the Oracle source endpoint by setting the `securityDbEncryptionName` extra connection attribute.

   ```
   securityDbEncryptionName=ORACLE.SECURITY.DB.ENCRYPTION.AWGDC9glSk8Xv+3bVveiVSgAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
   ```

1. Provide the associated TDE password for this key on the console as part of the Oracle source's **Password** value. Use the following order to format the comma-separated password values, ended by the TDE password value.

   ```
   Oracle_db_password,ASM_Password,AEMAASAASGYs0phWHfNt9J5mEMkkegGFiD4LLfQszDojgDzbfoYDEACv0x3pJC+UGD/PdtE2jLIcBQcAeHgJChQGLA==
   ```

   Specify the password values in this order regardless of your Oracle database configuration. For example, if you're using TDE but your Oracle database isn't using ASM, specify password values in the following comma-separated order.

   ```
   Oracle_db_password,,AEMAASAASGYs0phWHfNt9J5mEMkkegGFiD4LLfQszDojgDzbfoYDEACv0x3pJC+UGD/PdtE2jLIcBQcAeHgJChQGLA==
   ```

If the TDE credentials you specify are incorrect, the AWS DMS migration task does not fail. However, the task also does not read or apply ongoing replication changes to the target database. After starting the task, monitor **Table statistics** on the console migration task page to make sure changes are replicated.

If a DBA changes the TDE credential values for the Oracle database while the task is running, the task fails. The error message contains the new TDE encryption key name. To specify new values and restart the task, use the preceding procedure.

**Important**  
You can’t manipulate a TDE wallet created in an Oracle Automatic Storage Management (ASM) location because OS level commands like `cp`, `mv`, `orapki`, and `mkstore` corrupt the wallet files stored in an ASM location. This restriction is specific to TDE wallet files stored in an ASM location only, but not for TDE wallet files stored in a local OS directory.  
To manipulate a TDE wallet stored in ASM with OS level commands, create a local keystore and merge the ASM keystore into the local keystore as follows:   
Create a local keystore.  

   ```
   ADMINISTER KEY MANAGEMENT create keystore file system wallet location identified by wallet password;
   ```
Merge the ASM keystore into the local keystore.  

   ```
   ADMINISTER KEY MANAGEMENT merge keystore ASM wallet location identified by wallet password into existing keystore file system wallet location identified by wallet password with backup;
   ```
Then, to list the encryption wallet entries and TDE password, run steps 3 and 4 against the local keystore.

## Supported compression methods for using Oracle as a source for AWS DMS
<a name="CHAP_Source.Oracle.Compression"></a>

In the following table, you can find which compression methods AWS DMS supports when working with an Oracle source database. As the table shows, compression support depends both on your Oracle database version and whether DMS is configured to use Oracle LogMiner to access the redo logs.


| Version | Basic | OLTP |  HCC (from Oracle 11g R2 or newer)  | Others | 
| --- | --- | --- | --- | --- | 
| Oracle 10 | No | N/A | N/A | No | 
| Oracle 11 or newer – Oracle LogMiner | Yes | Yes | Yes  | Yes – Any compression method supported by Oracle LogMiner. | 
| Oracle 11 or newer – Binary Reader | Yes | Yes | Yes – For more information, see the following note . | Yes | 

**Note**  
When the Oracle source endpoint is configured to use Binary Reader, the Query Low level of the HCC compression method is supported for full-load tasks only.

## Replicating nested tables using Oracle as a source for AWS DMS
<a name="CHAP_Source.Oracle.NestedTables"></a>

AWS DMS supports the replication of Oracle tables containing columns that are nested tables or defined types. To enable this functionality, add the following extra connection attribute setting to the Oracle source endpoint.

```
allowSelectNestedTables=true;
```

AWS DMS creates the target tables from Oracle nested tables as regular parent and child tables on the target without a unique constraint. To access the correct data on the target, join the parent and child tables. To do this, first manually create a nonunique index on the `NESTED_TABLE_ID` column in the target child table. You can then use the `NESTED_TABLE_ID` column in the join `ON` clause together with the parent column that corresponds to the child table name. In addition, creating such an index improves performance when the target child table data is updated or deleted by AWS DMS. For an example, see [Example join for parent and child tables on the target](#CHAP_Source.Oracle.NestedTables.JoinExample).

We recommend that you configure the task to stop after a full load completes. Then, create these nonunique indexes for all the replicated child tables on the target and resume the task.

If a captured nested table is added to an existing parent table (captured or not captured), AWS DMS handles it correctly. However, the nonunique index for the corresponding target table isn't created. In this case, if the target child table becomes extremely large, performance might be affected. In such a case, we recommend that you stop the task, create the index, then resume the task.

After the nested tables are replicated to the target, have the DBA run a join on the parent and corresponding child tables to flatten the data.

### Prerequisites for replicating Oracle nested tables as a source
<a name="CHAP_Source.Oracle.NestedTables.Prerequisites"></a>

Ensure that you replicate parent tables for all the replicated nested tables. Include both the parent tables (the tables containing the nested table column) and the child (that is, nested) tables in the AWS DMS table mappings.

### Supported Oracle nested table types as a source
<a name="CHAP_Source.Oracle.NestedTables.Types"></a>

AWS DMS supports the following Oracle nested table types as a source:
+ Data type
+ User defined object

### Limitations of AWS DMS support for Oracle nested tables as a source
<a name="CHAP_Source.Oracle.NestedTables.Limitations"></a>

AWS DMS has the following limitations in its support of Oracle nested tables as a source:
+ AWS DMS supports only one level of table nesting.
+ AWS DMS table mapping does not check that both the parent and child table or tables are selected for replication. That is, it's possible to select a parent table without a child or a child table without a parent.

### How AWS DMS replicates Oracle nested tables as a source
<a name="CHAP_Source.Oracle.NestedTables.HowReplicated"></a>

AWS DMS replicates parent and nested tables to the target as follows:
+ AWS DMS creates the parent table identical to the source. It then defines the nested column in the parent as `RAW(16)` and includes a reference to the parent's nested tables in its `NESTED_TABLE_ID` column.
+ AWS DMS creates the child table identical to the nested source, but with an additional column named `NESTED_TABLE_ID`. This column has the same type and value as the corresponding parent nested column and has the same meaning.

### Example join for parent and child tables on the target
<a name="CHAP_Source.Oracle.NestedTables.JoinExample"></a>

To flatten the parent table, run a join between the parent and child tables, as shown in the following example:

1. Create the `Type` table.

   ```
   CREATE OR REPLACE TYPE NESTED_TEST_T AS TABLE OF VARCHAR(50);
   ```

1. Create the parent table with a column of type `NESTED_TEST_T` as defined preceding.

   ```
   CREATE TABLE NESTED_PARENT_TEST (ID NUMBER(10,0) PRIMARY KEY, NAME NESTED_TEST_T) NESTED TABLE NAME STORE AS NAME_KEY;
   ```

1. Flatten the table `NESTED_PARENT_TEST` using a join with the `NAME_KEY` child table where `CHILD.NESTED_TABLE_ID` matches `PARENT.NAME`.

   ```
   SELECT … FROM NESTED_PARENT_TEST PARENT, NAME_KEY CHILD WHERE CHILD.NESTED_
   TABLE_ID = PARENT.NAME;
   ```

## Storing REDO on Oracle ASM when using Oracle as a source for AWS DMS
<a name="CHAP_Source.Oracle.REDOonASM"></a>

For Oracle sources with high REDO generation, storing REDO on Oracle ASM can benefit performance, especially in a RAC configuration since you can configure DMS to distribute ASM REDO reads across all ASM nodes.

To utilize this configuration, use the `asmServer` connection attribute. For example, the following connection string distributes DMS REDO reads across 3 ASM nodes:

```
asmServer=(DESCRIPTION=(CONNECT_TIMEOUT=8)(ENABLE=BROKEN)(LOAD_BALANCE=ON)(FAILOVER=ON)
(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=tcp)(HOST=asm_node1_ip_address)(PORT=asm_node1_port_number))
(ADDRESS=(PROTOCOL=tcp)(HOST=asm_node2_ip_address)(PORT=asm_node2_port_number))
(ADDRESS=(PROTOCOL=tcp)(HOST=asm_node3_ip_address)(PORT=asm_node3_port_number)))
(CONNECT_DATA=(SERVICE_NAME=+ASM)))
```

When using NFS to store Oracle REDO, it’s important to make sure that applicable DNFS (direct NFS) client patches are applied, specifically any patch addressing Oracle bug 25224242. For additional information, review the following Oracle Publication regarding Direct NFS client related patches, [Recommended Patches for Direct NFS Client](https://support.oracle.com/knowledge/Oracle Cloud/1495104_1.html). 

Additionally, to improve NFS read performance, we recommended you increase the value of `rsize` and `wsize` in `fstab` for the the NFS volume, as shown in the following example.

```
NAS_name_here:/ora_DATA1_archive /u09/oradata/DATA1 nfs rw,bg,hard,nointr,tcp,nfsvers=3,_netdev,
timeo=600,rsize=262144,wsize=262144
```

Also, adjust the `tcp-max-xfer-size` value as follows:

```
vserver nfs modify -vserver vserver -tcp-max-xfer-size 262144
```

## Endpoint settings when using Oracle as a source for AWS DMS
<a name="CHAP_Source.Oracle.ConnectionAttrib"></a>

You can use endpoint settings to configure your Oracle source database similar to using extra connection attributes. You specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--oracle-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

The following table shows the endpoint settings that you can use with Oracle as a source.


| Name | Description | 
| --- | --- | 
| AccessAlternateDirectly |  Set this attribute to false in order to use the Binary Reader to capture change data for an Amazon RDS for Oracle as the source. This tells the DMS instance to not access redo logs through any specified path prefix replacement using direct file access. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC). Default value: true  Valid values: true/false Example: `--oracle-settings '{"AccessAlternateDirectly": false}'`  | 
|  `AdditionalArchivedLogDestId`  |  Set this attribute with `ArchivedLogDestId` in a primary-Standby setup. This attribute is useful in a switchover when Oracle Data Guard database is used as a source. In this case, AWS DMS needs to know which destination to get archive redo logs from to read changes. This is because the previous primary is now a Standby instance after switchover. Although AWS DMS supports the use of the Oracle `RESETLOGS` option to open the database, never use `RESETLOGS` unless necessary. For additional information about `RESETLOGS`, see [RMAN Data Repair Concepts](https://docs.oracle.com/en/database/oracle/oracle-database/19/bradv/rman-data-repair-concepts.html#GUID-1805CCF7-4AF2-482D-B65A-998192F89C2B) in the *Oracle® Database Backup and Recovery User's Guide*. Valid values : Archive destination Ids Example: `--oracle-settings '{"AdditionalArchivedLogDestId": 2}'`  | 
|  `AddSupplementalLogging`  |  Set this attribute to set up table-level supplemental logging for the Oracle database. This attribute enables one of the following on all tables selected for a migration task, depending on table metadata: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html) Default value: false  Valid values: true/false  Example: `--oracle-settings '{"AddSupplementalLogging": false}'`  If you use this option, you still need to enable database-level supplemental logging as discussed previously.    | 
|  `AllowSelectNestedTables`  |  Set this attribute to true to enable replication of Oracle tables containing columns that are nested tables or defined types. For more information, see [Replicating nested tables using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.NestedTables). Default value: false  Valid values: true/false Example: `--oracle-settings '{"AllowSelectNestedTables": true}'`  | 
|  `ArchivedLogDestId`  |  Specifies the ID of the destination for the archived redo logs. This value should be the same as a number in the dest\$1id column of the v\$1archived\$1log view. If you work with an additional redo log destination, we recommend that you use the `AdditionalArchivedLogDestId` attribute to specify the additional destination ID. Doing this improves performance by ensuring that the correct logs are accessed from the outset.  Default value: 1 Valid values: Number  Example: `--oracle-settings '{"ArchivedLogDestId": 1}'`  | 
|  `ArchivedLogsOnly`  |  When this field is set to Y, AWS DMS only accesses the archived redo logs. If the archived redo logs are stored on Oracle ASM only, the AWS DMS user account needs to be granted ASM privileges.  Default value: N  Valid values: Y/N  Example: `--oracle-settings '{"ArchivedLogsOnly": Y}'`  | 
|  `asmUsePLSQLArray` (ECA Only)  |  Use this extra connection attribute (ECA) when capturing source changes with BinaryReader. This setting allows DMS to buffer 50 reads at ASM level per single read thread while controlling the number of threads using the `parallelASMReadThreads` attribute. When you set this attribute, AWS DMS binary reader uses an anonymous PL/SQL block to capture redo data and send it back to the replication instance as a large buffer. This reduces the number of round trips to the source. This can significantly improve source capture performance, but it does result in higher PGA memory consumption on the ASM Instance. Stability issues might arise if the memory target is not sufficient enough. You can use the following formula to estimate the total ASM instance PGA memory usage by a single DMS task: `number_of_redo_threads * parallelASMReadThreads * 7 MB` Default value: false Valid values: true/false ECA Example: `asmUsePLSQLArray=true;`  | 
|  `ConvertTimestampWithZoneToUTC`  |  Set this attribute to `true` to convert the timestamp value of 'TIMESTAMP WITH TIME ZONE' and 'TIMESTAMP WITH LOCAL TIME ZONE' columns to UTC. By default the value of this attribute is 'false' and the data will be replicated using the source database timezone. Default value: false Valid values: true/false Example: `--oracle-settings '{"ConvertTimestampWithZoneToUTC": true}'`  | 
|  `EnableHomogenousPartitionOps`  |  Set this attribute to `true` to enable replication of Oracle Partition and subPartition DDL Operations for Oracle *Homogenous* migration. Note that this feature and enhancement was introduced in AWS DMS version 3.4.7. Default value: false Valid values: true/false Example: `--oracle-settings '{"EnableHomogenousPartitionOps": true}'`  | 
|  `EnableHomogenousTablespace`  |  Set this attribute to enable homogenous tablespace replication and create existing tables or indexes under the same tablespace on the target. Default value: false Valid values: true/false Example: `--oracle-settings '{"EnableHomogenousTablespace": true}'`  | 
|  `EscapeCharacter`  |  Set this attribute to an escape character. This escape character allows you to make a single wildcard character behave as a normal character in table mapping expressions. For more information, see [Wildcards in table mapping](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Wildcards.md). Default value: Null  Valid values: Any character other than a wildcard character Example: `--oracle-settings '{"EscapeCharacter": "#"}'` You can only use `escapeCharacter` for table names. It does not escape characters from schema names or column names.  | 
|  `ExposeViews`  |  Use this attribute to pull data once from a view; you can't use it for ongoing replication. When you extract data from a view, the view is shown as a table on the target schema. Default value: false Valid values: true/false Example: `--oracle-settings '{"ExposeViews": true}'`  | 
|  `ExtraArchivedLogDestIds`  |  Specifies the IDs of one more destinations for one or more archived redo logs. These IDs are the values of the dest\$1id column in the v\$1archived\$1log view. Use this setting with the ArchivedLogDestId extra connection attribute in a primary-to-single setup or a primary-to-multiple-standby setup. This setting is useful in a switchover when you use an Oracle Data Guard database as a source. In this case, AWS DMS needs information about what destination to get archive redo logs from to read changes. AWS DMS needs this because after the switchover the previous primary is a standby instance. Valid values: Archive destination Ids Example: `--oracle-settings '{"ExtraArchivedLogDestIds": 1}'`  | 
|  `FailTasksOnLobTruncation`  |  When set to `true`, this attribute causes a task to fail if the actual size of an LOB column is greater than the specified `LobMaxSize`. If a task is set to limited LOB mode and this option is set to `true`, the task fails instead of truncating the LOB data. Default value: false  Valid values: Boolean  Example: `--oracle-settings '{"FailTasksOnLobTruncation": true}'`  | 
|  `filterTransactionsOfUser` (ECA Only)  |  Use this extra connection attribute (ECA) to allows DMS to ignore transactions from a specified user when replicating data from Oracle when using LogMiner. You can pass comma separated user name values, but they must be in all CAPITAL letters. ECA Example: `filterTransactionsOfUser=USERNAME;`  | 
|  `NumberDataTypeScale`  |  Specifies the number scale. You can select a scale up to 38, or you can select -1 for FLOAT, or -2 for VARCHAR. By default, the NUMBER data type is converted to precision 38, scale 10. Default value: 10  Valid values: -2 to 38 (-2 for VARCHAR, -1 for FLOAT) Example: `--oracle-settings '{"NumberDataTypeScale": 12}'`  Select a precision-scale combination, -1 (FLOAT) or -2 (VARCHAR). DMS supports any precision-scale combination supported by Oracle. If precision is 39 or above, select -2 (VARCHAR). The NumberDataTypeScale setting for the Oracle database is used for the NUMBER data type only (without the explicit precision and scale definition). You must note that loss of precision can happen when this setting is incorrectly configured.   | 
|  `OpenTransactionWindow`  |   Provides the timeframe in minutes to check for any open transactions for CDC only task. When you set `OpenTransactionWindow` to 1 or higher, DMS uses `SCN_TO_TIMESTAMP` to convert SCN values to timestamp values. Because of Oracle Database limitations, if you specify an SCN that is too old as the CDC start point, SCN\$1TO\$1TIMESTAMP will fail with an `ORA-08181` error, and you can't start CDC-only tasks. Default value: 0  Valid values: An integer from 0 to 240 Example: `openTransactionWindow=15;`  | 
| OraclePathPrefix | Set this string attribute to the required value in order to use the Binary Reader to capture change data for an Amazon RDS for Oracle as the source. This value specifies the default Oracle root used to access the redo logs. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC).Default value: none Valid value: /rdsdbdata/db/ORCL\$1A/ Example: `--oracle-settings '{"OraclePathPrefix": "/rdsdbdata/db/ORCL_A/"}'`  | 
| ParallelASMReadThreads |  Set this attribute to change the number of threads that DMS configures to perform change data capture (CDC) using Oracle Automatic Storage Management (ASM). You can specify an integer value between 2 (the default) and 8 (the maximum). Use this attribute together with the `ReadAheadBlocks` attribute. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC). Default value: 2  Valid values: An integer from 2 to 8 Example: `--oracle-settings '{"ParallelASMReadThreads": 6;}'`  | 
| ReadAheadBlocks |  Set this attribute to change the number of read-ahead blocks that DMS configures to perform CDC using Oracle Automatic Storage Management (ASM) and non-ASM NAS storage. You can specify an integer value between 1000 (the default) and 2,000,000 (the maximum). Use this attribute together with the `ParallelASMReadThreads` attribute. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC). Default value: 1000  Valid values: An integer from 1000 to 2,000,000 Example: `--oracle-settings '{"ReadAheadBlocks": 150000}'`  | 
|  `ReadTableSpaceName`  |  When set to `true`, this attribute supports tablespace replication. Default value: false  Valid values: Boolean  Example: `--oracle-settings '{"ReadTableSpaceName": true}'`  | 
| ReplacePathPrefix | Set this attribute to true in order to use the Binary Reader to capture change data for an Amazon RDS for Oracle as the source. This setting tells DMS instance to replace the default Oracle root with the specified UsePathPrefix setting to access the redo logs. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC).Default value: false Valid values: true/false Example: `--oracle-settings '{"ReplacePathPrefix": true}'`  | 
|  `RetryInterval`  |  Specifies the number of seconds that the system waits before resending a query.  Default value: 5  Valid values: Numbers starting from 1  Example: `--oracle-settings '{"RetryInterval": 6}'`  | 
|  `SecurityDbEncryptionName`  |  Specifies the name of a key used for the transparent data encryption (TDE) of the columns and tablespace in the Oracle source database. For more information on setting this attribute and its associated password on the Oracle source endpoint, see [Supported encryption methods for using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.Encryption). Default value: ""  Valid values: String  Example: `--oracle-settings '{"SecurityDbEncryptionName": "ORACLE.SECURITY.DB.ENCRYPTION.Adg8m2dhkU/0v/m5QUaaNJEAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}'`  | 
|  `SpatialSdo2GeoJsonFunctionName`  |  For Oracle version 12.1 or earlier sources migrating to PostgreSQL targets, use this attribute to convert SDO\$1GEOMETRY to GEOJSON format. By default, AWS DMS calls the `SDO2GEOJSON` custom function which must be present and accessible to the AWS DMS user. Or you can create your own custom function that mimics the operation of `SDOGEOJSON` and set `SpatialSdo2GeoJsonFunctionName` to call it instead.  Default value: SDO2GEOJSON Valid values: String  Example: `--oracle-settings '{"SpatialSdo2GeoJsonFunctionName": "myCustomSDO2GEOJSONFunction"}'`  | 
|  `StandbyDelayTime`  |  Use this attribute to specify a time in minutes for the delay in standby sync. If the source is an Active Data Guard standby database, use this attribute to specify the time lag between primary and standby databases. In AWS DMS, you can create an Oracle CDC task that uses an Active Data Guard standby instance as a source for replicating ongoing changes. Doing this eliminates the need to connect to an active database that might be in production. Default value:0  Valid values: Number  Example: `--oracle-settings '{"StandbyDelayTime": 1}'` **Note: **When using DMS 3.4.6, 3.4.7 and higher, use of this connection setting is optional. In the latest version of DMS 3.4.6 and version 3.4.7, `dms_user` should have `select` permission on `V_$DATAGUARD_STATS`, allowing DMS to calculate standby delay time.  | 
| UseAlternateFolderForOnline | Set this attribute to true in order to use the Binary Reader to capture change data for an Amazon RDS for Oracle as the source. This tells the DMS instance to use any specified prefix replacement to access all online redo logs. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC).Default value: false Valid values: true/false Example: `--oracle-settings '{"UseAlternateFolderForOnline": true}'`  | 
| UseBfile |  Set this attribute to Y in order to capture change data using the Binary Reader utility. Set `UseLogminerReader` to N to set this attribute to Y. To use the Binary Reader with an Amazon RDS for Oracle as the source, you set additional attributes. For more information on this setting and using Oracle Automatic Storage Management (ASM), see [Using Oracle LogMiner or AWS DMS Binary Reader for CDC](#CHAP_Source.Oracle.CDC). Note: When setting this value as an Extra Connection Attribute (ECA), valid values are 'Y' and 'N'. When setting this value as an endpoint setting, valid values are `true` and `false`. Default value: N  Valid values: Y/N (when setting this value as an ECA); true/false (when setting this value as as an endpoint setting). Example: `--oracle-settings '{"UseBfile": Y}'`  | 
|  `UseLogminerReader`  |  Set this attribute to Y to capture change data using the LogMiner utility (the default). Set this option to N if you want AWS DMS to access the redo logs as a binary file. When you set this option to N, also add the setting useBfile=Y. For more information on this setting and using Oracle Automatic Storage Management (ASM), see [Using Oracle LogMiner or AWS DMS Binary Reader for CDC](#CHAP_Source.Oracle.CDC). Note: When setting this value as an Extra Connection Attribute (ECA), valid values are 'Y' and 'N'. When setting this value as an endpoint setting, valid values are `true` and `false`. Default value: Y  Valid values: Y/N (when setting this value as an ECA); true/false (when setting this value as as an endpoint setting). Example: `--oracle-settings '{"UseLogminerReader": Y}'`  | 
| UsePathPrefix | Set this string attribute to the required value in order to use the Binary Reader to capture change data for an Amazon RDS for Oracle as the source. This value specifies the path prefix used to replace the default Oracle root to access the redo logs. For more information, see [Configuring a CDC task to use Binary Reader with an RDS for Oracle source for AWS DMS](#CHAP_Source.Oracle.Amazon-Managed.CDC).Default value: none Valid value: /rdsdbdata/log/ Example: `--oracle-settings '{"UsePathPrefix": "/rdsdbdata/log/"}'`  | 

## Source data types for Oracle
<a name="CHAP_Source.Oracle.DataTypes"></a>

The Oracle endpoint for AWS DMS supports most Oracle data types. The following table shows the Oracle source data types that are supported when using AWS DMS and the default mapping to AWS DMS data types.

**Note**  
With the exception of the LONG and LONG RAW data types, when replicating from an Oracle source to an Oracle target (a *homogeneous replication*), all of the source and target data types will be identical. But the LONG data type will be mapped to CLOB and the LONG RAW data type will be mapped to BLOB. 

For information on how to view the data type that is mapped in the target, see the section for the target endpoint you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  Oracle data type  |  AWS DMS data type  | 
| --- | --- | 
|  BINARY\$1FLOAT  |  REAL4  | 
|  BINARY\$1DOUBLE  |  REAL8  | 
|  BINARY  |  BYTES  | 
|  FLOAT (P)  |  If precision is less than or equal to 24, use REAL4. If precision is greater than 24, use REAL8.  | 
|  NUMBER (P,S)  |  When scale is greater than 0, use NUMERIC. When scale is 0: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html) When scale is less than 0, use REAL8. | 
|  DATE  |  DATETIME  | 
|  INTERVAL\$1YEAR TO MONTH  |  STRING (with interval year\$1to\$1month indication)  | 
|  INTERVAL\$1DAY TO SECOND  |  STRING (with interval day\$1to\$1second indication)  | 
|  TIMESTAMP  |  DATETIME  | 
|  TIMESTAMP WITH TIME ZONE  |  STRING (with timestamp\$1with\$1timezone indication)  | 
|  TIMESTAMP WITH LOCAL TIME ZONE  |  STRING (with timestamp\$1with\$1local\$1 timezone indication)  | 
|  CHAR  |  STRING  | 
|  VARCHAR2  |  [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html)  | 
|  NCHAR  |  WSTRING  | 
|  NVARCHAR2  |  [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.Oracle.html)  | 
|  RAW  |  BYTES  | 
|  REAL  |  REAL8  | 
|  BLOB  |  BLOB To use this data type with AWS DMS, you must enable the use of BLOB data types for a specific task. AWS DMS supports BLOB data types only in tables that include a primary key.  | 
|  CLOB  |  CLOB To use this data type with AWS DMS, you must enable the use of CLOB data types for a specific task. During CDC, AWS DMS supports CLOB data types only in tables that include a primary key.  | 
|  NCLOB  |  NCLOB To use this data type with AWS DMS, you must enable the use of NCLOB data types for a specific task. During CDC, AWS DMS supports NCLOB data types only in tables that include a primary key.  | 
|  LONG  |  CLOB The LONG data type isn't supported in batch-optimized apply mode (TurboStream CDC mode). To use this data type with AWS DMS, enable the use of LOBs for a specific task. During CDC or full load, AWS DMS supports LOB data types only in tables that have a primary key. Also, AWS DMS does not support full LOB mode for loading LONG columns. Instead, you can use limited LOB mode for migrating LONG columns to an Oracle target. In limited LOB mode, AWS DMS truncates any data to 64 KB that you set to LONG columns longer than 64 KB. For more information about LOB support in AWS DMS, see [Setting LOB support for source databases in an AWS DMS task](CHAP_Tasks.LOBSupport.md)  | 
|  LONG RAW  |  BLOB The LONG RAW data type isn't supported in batch-optimized apply mode (TurboStream CDC mode). To use this data type with AWS DMS, enable the use of LOBs for a specific task. During CDC or full load, AWS DMS supports LOB data types only in tables that have a primary key. Also, AWS DMS does not support full LOB mode for loading LONG RAW columns. Instead, you can use limited LOB mode for migrating LONG RAW columns to an Oracle target. In limited LOB mode, AWS DMS truncates any data to 64 KB that you set to LONG RAW columns longer than 64 KB. For more information about LOB support in AWS DMS, see [Setting LOB support for source databases in an AWS DMS task](CHAP_Tasks.LOBSupport.md)  | 
|  XMLTYPE  |  CLOB  | 
| SDO\$1GEOMETRY | BLOB (when an Oracle to Oracle migration)CLOB (when an Oracle to PostgreSQL migration) | 

Oracle tables used as a source with columns of the following data types are not supported and can't be replicated. Replicating columns with these data types result in a null column.
+ BFILE
+ ROWID
+ REF
+ UROWID
+ User-defined data types
+ ANYDATA
+ VARRAY

**Note**  
Virtual columns are not supported.

### Migrating Oracle spatial data types
<a name="CHAP_Source.Oracle.DataTypes.Spatial"></a>

*Spatial data* identifies the geometry information for an object or location in space. In an Oracle database, the geometric description of a spatial object is stored in an object of type SDO\$1GEOMETRY. Within this object, the geometric description is stored in a single row in a single column of a user-defined table. 

AWS DMS supports migrating the Oracle type SDO\$1GEOMETRY from an Oracle source to either an Oracle or PostgreSQL target.

When you migrate Oracle spatial data types using AWS DMS, be aware of these considerations:
+ When migrating to an Oracle target, make sure to manually transfer USER\$1SDO\$1GEOM\$1METADATA entries that include type information. 
+ When migrating from an Oracle source endpoint to a PostgreSQL target endpoint, AWS DMS creates target columns. These columns have default geometry and geography type information with a 2D dimension and a spatial reference identifier (SRID) equal to zero (0). An example is `GEOMETRY, 2, 0`.
+ For Oracle version 12.1 or earlier sources migrating to PostgreSQL targets, convert `SDO_GEOMETRY` objects to `GEOJSON` format by using the `SDO2GEOJSON` function, or the `spatialSdo2GeoJsonFunctionName` extra connection attribute. For more information, see [Endpoint settings when using Oracle as a source for AWS DMS](#CHAP_Source.Oracle.ConnectionAttrib).
+ AWS DMS supports Oracle Spatial Column migrations for Full LOB mode only. AWS DMS does not support Limited LOB or Inline LOB modes. For more information about LOB mode, see [Setting LOB support for source databases in an AWS DMS task](CHAP_Tasks.LOBSupport.md).
+ Because AWS DMS only supports Full LOB mode for migrating Oracle Spatial Columns, the columns' table needs a primary key and a unique key. If the table does not have a primary key and a unique key, the table is skipped from migration.

# Using a Microsoft SQL Server database as a source for AWS DMS
<a name="CHAP_Source.SQLServer"></a>

Migrate data from one or many Microsoft SQL Server databases using AWS DMS. With a SQL Server database as a source, you can migrate data to another SQL Server database, or to one of the other AWS DMS supported databases. 

For information about versions of SQL Server that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

The source SQL Server database can be installed on any computer in your network. A SQL Server account with appropriate access privileges to the source database for the type of task you chose is required for use with AWS DMS. For more information, see [Permissions for SQL Server tasks](#CHAP_Source.SQLServer.Permissions).

AWS DMS supports migrating data from named instances of SQL Server. You can use the following notation in the server name when you create the source endpoint.

```
IPAddress\InstanceName
```

For example, the following is a correct source endpoint server name. Here, the first part of the name is the IP address of the server, and the second part is the SQL Server instance name (in this example, SQLTest).

```
10.0.0.25\SQLTest
```

Also, obtain the port number that your named instance of SQL Server listens on, and use it to configure your AWS DMS source endpoint. 

**Note**  
Port 1433 is the default for Microsoft SQL Server. But dynamic ports that change each time SQL Server is started, and specific static port numbers used to connect to SQL Server through a firewall are also often used. So, you want to know the actual port number of your named instance of SQL Server when you create the AWS DMS source endpoint.

You can use SSL to encrypt connections between your SQL Server endpoint and the replication instance. For more information on using SSL with a SQL Server endpoint, see [Using SSL with AWS Database Migration Service](CHAP_Security.SSL.md).

You can use CDC for ongoing migration from a SQL Server database. For information about configuring your source SQL server database for CDC, see [Capturing data changes for ongoing replication from SQL Server](CHAP_Source.SQLServer.CDC.md).

For additional details on working with SQL Server source databases and AWS DMS, see the following.

**Topics**
+ [

## Limitations on using SQL Server as a source for AWS DMS
](#CHAP_Source.SQLServer.Limitations)
+ [

## Permissions for SQL Server tasks
](#CHAP_Source.SQLServer.Permissions)
+ [

## Prerequisites for using ongoing replication (CDC) from a SQL Server source
](#CHAP_Source.SQLServer.Prerequisites)
+ [

## Supported compression methods for SQL Server
](#CHAP_Source.SQLServer.Compression)
+ [

## Working with self-managed SQL Server AlwaysOn availability groups
](#CHAP_Source.SQLServer.AlwaysOn)
+ [

## Endpoint settings when using SQL Server as a source for AWS DMS
](#CHAP_Source.SQLServer.ConnectionAttrib)
+ [

## Source data types for SQL Server
](#CHAP_Source.SQLServer.DataTypes)
+ [

# Capturing data changes for ongoing replication from SQL Server
](CHAP_Source.SQLServer.CDC.md)

## Limitations on using SQL Server as a source for AWS DMS
<a name="CHAP_Source.SQLServer.Limitations"></a>

The following limitations apply when using a SQL Server database as a source for AWS DMS:
+ The identity property for a column isn't migrated to a target database column.
+ The SQL Server endpoint doesn't support the use of tables with sparse columns.
+ Windows Authentication isn't supported.
+ Changes to computed fields in a SQL Server aren't replicated.
+ Temporal tables aren't supported.
+ SQL Server partition switching isn't supported.
+ When using the WRITETEXT and UPDATETEXT utilities, AWS DMS doesn't capture events applied on the source database.
+ The following data manipulation language (DML) pattern isn't supported. 

  ```
  SELECT * INTO new_table FROM existing_table
  ```
+ When using SQL Server as a source, column-level encryption isn't supported.
+ AWS DMS doesn't support server level audits on SQL Server 2008 or SQL Server 2008 R2 as sources. This is because of a known issue with SQL Server 2008 and 2008 R2. For example, running the following command causes AWS DMS to fail.

  ```
  USE [master]
  GO 
  ALTER SERVER AUDIT [my_audit_test-20140710] WITH (STATE=on)
  GO
  ```
+ Geometry and Geography columns are not supported in full lob mode when using SQL Server as a source. Instead, use limited lob mode or set the `InlineLobMaxSize` task setting to use inline lob mode.
+ When using a Microsoft SQL Server source database in a replication task, the SQL Server Replication Publisher definitions aren't removed if you remove the task. A Microsoft SQL Server system administrator must delete those definitions from Microsoft SQL Server.
+ Migrating data from schema-bound and non-schema-bound views is supported for full-load only tasks. 
+ Renaming tables using sp\$1rename isn't supported (for example, `sp_rename 'Sales.SalesRegion', 'SalesReg;)`
+ Renaming columns using sp\$1rename isn't supported (for example, `sp_rename 'Sales.Sales.Region', 'RegID', 'COLUMN';`)
+ AWS DMS doesn't support change processing to set and unset column default values (using the `ALTER COLUMN SET DEFAULT` clause with `ALTER TABLE` statements).
+ AWS DMS doesn't support change processing to set column nullability (using the `ALTER COLUMN [SET|DROP] NOT NULL` clause with `ALTER TABLE` statements).
+ With SQL Server 2012 and SQL Server 2014, when using DMS replication with Availability Groups, the distribution database can't be placed in an availability group. SQL 2016 supports placing the distribution database into an availability group, except for distribution databases used in merge, bidirectional, or peer-to-peer replication topologies.
+ For partitioned tables, AWS DMS doesn't support different data compression settings for each partition.
+ When inserting a value into SQL Server spatial data types (GEOGRAPHY and GEOMETRY), you can either ignore the spatial reference system identifier (SRID) property or specify a different number. When replicating tables with spatial data types, AWS DMS replaces the SRID with the default SRID (0 for GEOMETRY and 4326 for GEOGRAPHY).
+ If your database isn't configured for MS-REPLICATION or MS-CDC, you can still capture tables that do not have a Primary Key, but only INSERT/DELETE DML events are captured. UPDATE and TRUNCATE TABLE events are ignored.
+ Columnstore indexes aren't supported.
+ Memory-optimized tables (using In-Memory OLTP) aren't supported.
+ When replicating a table with a primary key that consists of multiple columns, updating the primary key columns during full load isn't supported.
+ Delayed durability isn't supported.
+ The `readBackupOnly=true` endpoint setting (extra connection attribute) doesn't work on RDS for SQL Server source instances because of the way RDS performs backups.
+ `EXCLUSIVE_AUTOMATIC_TRUNCATION` doesn’t work on Amazon RDS SQL Server source instances because RDS users don't have access to run the SQL Server stored procedure, `sp_repldone`.
+ AWS DMS doesn't capture truncate commands.
+ AWS DMS doesn't support replication from databases with accelerated database recovery (ADR) turned on.
+ AWS DMS doesn't support capturing data definition language (DDL) and data manipulation language (DML) statements within a single transaction.
+ AWS DMS doesn't support the replication of data-tier application packages (DACPAC).
+ UPDATE statements that involve primary keys or unique indexes and update multiple data rows, can cause conflicts when you apply changes to the target database. This might happen, for example, when the target database applies updates as INSERT and DELETE statements instead of a single UPDATE statement. With the batch optimized apply mode, the table might be ignored. With the transactional apply mode, the UPDATE operation might result in constraint violations. To avoid this issue, reload the relevant table. Alternatively, locate the problematic records in the Apply Exceptions control table (`dmslogs.awsdms_apply_exceptions`) and edit them manually in the target database. For more information, see [Change processing tuning settings](CHAP_Tasks.CustomizingTasks.TaskSettings.ChangeProcessingTuning.md).
+ AWS DMS doesn't support the replication of tables and schemas, where the name includes a special character from the following set.

  `\\ -- \n \" \b \r ' \t ;` 
+ Data masking isn't supported. AWS DMS migrates masked data without masking.
+ AWS DMS replicates up to 32,767 tables with primary keys and up to 1,000 columns for each table. This is because AWS DMS creates a SQL Server replication article for each replicated table, and SQL Server replication articles have these limitations.
+ When using Change Data Capture (CDC), you must define all columns that make up a unique index as `NOT NULL`. If this requirement is not met, SQL Server system error 22838 will result. 
+ You may lose events if SQL Server archives from the active transaction log to the backup log, or truncates them from the active transaction log.

The following limitations apply when accessing the backup transaction logs:
+ Encrypted backups aren't supported.
+ Backups stored at a URL or on Windows Azure aren't supported.
+ AWS DMS doe snot support direct processing of transaction log backups at the file level from alternative shared folders.
+ For Cloud SQL Server sources other than Amazon RDS for Microsoft SQL Server, AWS DMS supports ongoing replication (CDC) with the active transaction log only. You can't use the backup log with CDC. You may lose events if SQL server archives them from the active transaction log to the backup log, or truncates them from the active transaction log before DMS can read it. 
+ For Amazon RDS for Microsoft SQL Server sources, AWS DMS 3.5.2 and below supports ongoing replication (CDC) with the active transaction log only, because DMS can’t access the backup log with CDC. You may lose events if RDS for SQL Server archives them from the active transaction log to the backup log, or truncate them from the active transaction log before DMS can read it. This limitation does not apply to AWS DMS version 3.5.3 and above.
+ AWS DMS does not support CDC for Amazon RDS Proxy for SQL Server as a source.
+ If the SQL Server source becomes unavailable during a full load task, AWS DMS might mark the task as completed after multiple reconnection attempts, even though the data migration remains incomplete. In this scenario, the target tables contain only the records migrated before the connection loss, potentially creating data inconsistencies between the source and target systems. To ensure data completeness, you must either restart the full load task entirely or reload the specific tables affected by the connection interruption.

## Permissions for SQL Server tasks
<a name="CHAP_Source.SQLServer.Permissions"></a>

**Topics**
+ [

### Permissions for full load only tasks
](#CHAP_Source.SQLServer.Permissions.FullLoad)
+ [

### Permissions for tasks with ongoing replication
](#CHAP_Source.SQLServer.Permissions.Ongoing)

### Permissions for full load only tasks
<a name="CHAP_Source.SQLServer.Permissions.FullLoad"></a>

The following permissions are required to perform full load only tasks. Note that AWS DMS does not create the `dms_user` login. For information about creating a login for SQL Server, see [Create a database user](https://learn.microsoft.com/en-us/sql/relational-databases/security/authentication-access/create-a-database-user?view=sql-server-ver16) topic in *Microsoft's documentation*.

```
USE db_name;
                
                CREATE USER dms_user FOR LOGIN dms_user; 
                ALTER ROLE [db_datareader] ADD MEMBER dms_user; 
                GRANT VIEW DATABASE STATE to dms_user;
                GRANT VIEW DEFINITION to dms_user;
                
                USE master;
                
                GRANT VIEW SERVER STATE TO dms_user;
```

### Permissions for tasks with ongoing replication
<a name="CHAP_Source.SQLServer.Permissions.Ongoing"></a>

Self-managed SQL Server instances can be configured for ongoing replication using DMS with or without using the `sysadmin` role. For SQL Server instances, where you can't grant the `sysadmin` role, ensure that the DMS user has the privileges described as follows.

**Set up permissions for ongoing replication from a self-managed SQL Server database**

1. Create a new SQL Server account with password authentication using SQL Server Management Studio (SSMS) or as described previously in [Permissions for full load only tasks](#CHAP_Source.SQLServer.Permissions.FullLoad), for example, `self_managed_user`.

1. Run the following `GRANT` commands: 

   ```
   GRANT VIEW SERVER STATE TO self_managed_user;
   
   USE msdb;
       GRANT SELECT ON msdb.dbo.backupset TO self_managed_user;
       GRANT SELECT ON msdb.dbo.backupmediafamily TO self_managed_user;
       GRANT SELECT ON msdb.dbo.backupfile TO self_managed_user;
       
   USE db_name;
       CREATE USER self_managed_user FOR LOGIN self_managed_user;
       ALTER ROLE [db_owner] ADD MEMBER self_managed_user;
       GRANT VIEW DEFINITION to self_managed_user;
   ```

1. In addition to the preceding permissions, the user needs one of the following:
   + The user must be a member of the `sysadmin` fixed server role
   + Configurations and permissions as described in [Setting up ongoing replication on a SQL Server in an availability group environment: Without sysadmin role](CHAP_Source.SQLServer.CDC.md#CHAP_SupportScripts.SQLServer.ag) or [Setting up ongoing replication on a standalone SQL Server: Without sysadmin role](CHAP_Source.SQLServer.CDC.md#CHAP_SupportScripts.SQLServer.standalone), depending on your source configuration.

#### Set up permissions for ongoing replication from a cloud SQL Server database
<a name="CHAP_Source.SQLServer.Permissions.Cloud"></a>

A cloud-hosted SQL server instance is an instance running on Amazon RDS for Microsoft SQL Server, an Azure SQL Managed Instance, or any other managed cloud SQL Server instance supported by DMS.

Create a new SQL Server account with password authentication using SQL Server Management Studio (SSMS) or as described previously in [Permissions for full load only tasks](#CHAP_Source.SQLServer.Permissions.FullLoad), for example, `rds_user`.

Run the following grant commands.

```
GRANT VIEW SERVER STATE TO rds_user;
```

For Amazon RDS for Microsoft SQL Server sources, DMS version 3.5.3 and above support reading from transaction log backups. To ensure that DMS is able to access the log backups, in addition to the above, either grant `master` user privileges, or the following privileges on an RDS SQL Server source:

```
USE msdb;
    GRANT EXEC ON msdb.dbo.rds_dms_tlog_download TO rds_user;
    GRANT EXEC ON msdb.dbo.rds_dms_tlog_read TO rds_user;
    GRANT EXEC ON msdb.dbo.rds_dms_tlog_list_current_lsn TO rds_user;
    GRANT EXEC ON msdb.dbo.rds_task_status TO rds_user;
    
USE db_name;
    CREATE USER rds_user FOR LOGIN rds_user;
    ALTER ROLE [db_owner] ADD MEMBER rds_user;
    GRANT VIEW DEFINITION to rds_user;
```

For Amazon Azure SQL Managed Instances grant the following privileges:

```
GRANT SELECT ON msdb.dbo.backupset TO rds_user;
GRANT SELECT ON msdb.dbo.backupmediafamily TO rds_user;
GRANT SELECT ON msdb.dbo.backupfile TO rds_user;
```

## Prerequisites for using ongoing replication (CDC) from a SQL Server source
<a name="CHAP_Source.SQLServer.Prerequisites"></a>

You can use ongoing replication (change data capture, or CDC) for a self-managed SQL Server database on-premises or on Amazon EC2, or a cloud database such as Amazon RDS or a Microsoft Azure SQL managed instance.

The following requirements apply specifically when using ongoing replication with a SQL Server database as a source for AWS DMS:
+ SQL Server must be configured for full backups, and you must perform a backup before beginning to replicate data.
+ The recovery model must be set to **Bulk logged** or **Full**.
+ SQL Server backup to multiple disks isn't supported. If the backup is defined to write the database backup to multiple files over different disks, AWS DMS can't read the data and the AWS DMS task fails.
+ For self-managed SQL Server sources, SQL Server Replication Publisher definitions for the source used in a DMS CDC task aren't removed when you remove the task. A SQL Server system administrator must delete these definitions from SQL Server for self-managed sources.
+ During CDC, AWS DMS needs to look up SQL Server transaction log backups to read changes. AWS DMS doesn't support SQL Server transaction log backups created using third-party backup software that* aren't *in native format. To support transaction log backups that *are* in native format and created using third-party backup software, add the `use3rdPartyBackupDevice=Y` connection attribute to the source endpoint.
+ For self-managed SQL Server sources, be aware that SQL Server doesn't capture changes on newly created tables until they've been published. When tables are added to a SQL Server source, AWS DMS manages creating the publication. However, this process might take several minutes. Operations made to newly created tables during this delay aren't captured or replicated to the target. 
+ AWS DMS change data capture requires full transaction logging to be turned on in SQL Server. To turn on full transaction logging in SQL Server, either enable MS-REPLICATION or CHANGE DATA CAPTURE (CDC).
+ SQL Server *tlog* entries won't be marked for re-use until the MS CDC capture job processes those changes.
+ CDC operations aren't supported on memory-optimized tables. This limitation applies to SQL Server 2014 (when the feature was first introduced) and higher.
+ AWS DMS change data capture requires a distribution database by default on Amazon EC2 or On-Prem SQL server as source. So, ensure that you have activated the distributor while configuring MS replication for tables with primary keys.

## Supported compression methods for SQL Server
<a name="CHAP_Source.SQLServer.Compression"></a>

Note the following about support for SQL Server compression methods in AWS DMS:
+ AWS DMS supports Row/Page compression in SQL Server version 2008 and later.
+ AWS DMS doesn't support the Vardecimal storage format.
+ AWS DMS doesn't support sparse columns and columnar structure compression.

## Working with self-managed SQL Server AlwaysOn availability groups
<a name="CHAP_Source.SQLServer.AlwaysOn"></a>

SQL Server Always On availability groups provide high availability and disaster recovery as an enterprise-level alternative to database mirroring. 

In AWS DMS, you can migrate changes from a single primary or secondary availability group replica.

### Working with the primary availability group replica
<a name="CHAP_Source.SQLServer.AlwaysOn.Primary"></a>

 

**To use the primary availability group as a source in AWS DMS, do the following:**

1. Turn on the distribution option for all SQL Server instances in your availability replicas. For more information, see [Setting up ongoing replication on a self-managed SQL Server](CHAP_Source.SQLServer.CDC.md#CHAP_Source.SQLServer.CDC.MSCDC).

1. In the AWS DMS console, open the SQL Server source database settings. For **Server Name**, specify the Domain Name Service (DNS) name or IP address that was configured for your availability group listener. 

When you start an AWS DMS task for the first time, it might take longer than usual to start. This slowness occurs because the creation of the table articles is being duplicated by the availability group server. 

### Working with a secondary availability group replica
<a name="CHAP_Source.SQLServer.AlwaysOn.Secondary"></a>

**To use a secondary availability group as a source in AWS DMS, do the following:**

1. Use the same credentials for connecting to individual replicas as those used by the AWS DMS source endpoint user.

1. Ensure that your AWS DMS replication instance can resolve DNS names for all existing replicas, and connect to them. You can use the following SQL query to get DNS names for all of your replicas.

   ```
   select ar.replica_server_name, ar.endpoint_url from sys.availability_replicas ar
   JOIN sys.availability_databases_cluster adc
   ON adc.group_id = ar.group_id AND adc.database_name = '<source_database_name>';
   ```

1. When you create the source endpoint, specify the DNS name of the availability group listener for the endpoint's **Server name** or for the endpoint secret's **Server address**. For more information about availability group listeners, see [What is an availability group listener?](https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/availability-group-listener-overview?view=sql-server-ver15) in the SQL Server documentation.

   You can use either a public DNS server or an on-premises DNS server to resolve the availability group listener, the primary replica, and the secondary replicas. To use an on-premises DNS server, configure the Amazon Route 53 Resolver. For more information, see [Using your own on-premises name server](CHAP_BestPractices.md#CHAP_BestPractices.Rte53DNSResolver).

1. Add the following extra connection attributes to your source endpoint.    
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html)

1. Enable the distribution option on all replicas in your availability group. Add all nodes to the distributors list. For more information, see [To set up distribution](CHAP_Source.SQLServer.CDC.md#CHAP_Source.SQLServer.CDC.MSCDC.Setup).

1. Run the following query on the primary read-write replica to enable publication of your database. You run this query only once for your database. 

   ```
   sp_replicationdboption @dbname = N'<source DB name>', @optname = N'publish', @value = N'true';
   ```



#### Limitations
<a name="CHAP_Source.SQLServer.AlwaysOn.Secondary.limitations"></a>

Following are limitations for working with a secondary availability group replica:
+ AWS DMS doesn't support Safeguard when using a read-only availability group replica as a source. For more information, see [Endpoint settings when using SQL Server as a source for AWS DMS](#CHAP_Source.SQLServer.ConnectionAttrib).
+ AWS DMS doesn't support the `setUpMsCdcForTables` extra connection attribute when using a read-only availability group replica as a source. For more information, see [Endpoint settings when using SQL Server as a source for AWS DMS](#CHAP_Source.SQLServer.ConnectionAttrib).
+ AWS DMS can use a self-managed secondary availability group replica as a source database for ongoing replication (change data capture, or CDC) starting from version 3.4.7. Cloud SQL Server Multi-AZ read replicas are not supported. If you use previous versions of AWS DMS, make sure that you use the primary availability group replica as a source database for CDC.

#### Failover to other nodes
<a name="CHAP_Source.SQLServer.AlwaysOn.Secondary.failover"></a>

If you set the `ApplicationIntent` extra connection attribute for your endpoint to `ReadOnly`, your AWS DMS task connects to the read-only node with the highest read-only routing priority. It then fails over to other read-only nodes in your availability group when the highest priority read-only node is unavailable. If you don't set `ApplicationIntent`, your AWS DMS task only connects to the primary (read/write) node in your availability group.

## Endpoint settings when using SQL Server as a source for AWS DMS
<a name="CHAP_Source.SQLServer.ConnectionAttrib"></a>

You can use endpoint settings to configure your SQL Server source database similar to using extra connection attributes. You specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--microsoft-sql-server-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

The following table shows the endpoint settings that you can use with SQL Server as a source.

[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html)

## Source data types for SQL Server
<a name="CHAP_Source.SQLServer.DataTypes"></a>

Data migration that uses SQL Server as a source for AWS DMS supports most SQL Server data types. The following table shows the SQL Server source data types that are supported when using AWS DMS and the default mapping from AWS DMS data types.

For information on how to view the data type that is mapped in the target, see the section for the target endpoint you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  SQL Server data types  |  AWS DMS data types  | 
| --- | --- | 
|  BIGINT  |  INT8  | 
|  BIT  |  BOOLEAN  | 
|  DECIMAL  |  NUMERIC  | 
|  INT  |  INT4  | 
|  MONEY  |  NUMERIC  | 
|  NUMERIC (p,s)  |  NUMERIC   | 
|  SMALLINT  |  INT2  | 
|  SMALLMONEY  |  NUMERIC  | 
|  TINYINT  |  UINT1  | 
|  REAL  |  REAL4  | 
|  FLOAT  |  REAL8  | 
|  DATETIME  |  DATETIME  | 
|  DATETIME2 (SQL Server 2008 and higher)  |  DATETIME  | 
|  SMALLDATETIME  |  DATETIME  | 
|  DATE  |  DATE  | 
|  TIME  |  TIME  | 
|  DATETIMEOFFSET  |  WSTRING  | 
|  CHAR  |  STRING  | 
|  VARCHAR  |  STRING  | 
|  VARCHAR (max)  |  CLOB TEXT To use this data type with AWS DMS, you must enable the use of CLOB data types for a specific task. For SQL Server tables, AWS DMS updates LOB columns in the target even for UPDATE statements that don't change the value of the LOB column in SQL Server. During CDC, AWS DMS supports CLOB data types only in tables that include a primary key.  | 
|  NCHAR  |  WSTRING  | 
|  NVARCHAR (length)  |  WSTRING  | 
|  NVARCHAR (max)  |  NCLOB NTEXT To use this data type with AWS DMS, you must enable the use of SupportLobs for a specific task. For more information about enabling Lob support, see [Setting LOB support for source databases in an AWS DMS task](CHAP_Tasks.LOBSupport.md).  For SQL Server tables, AWS DMS updates LOB columns in the target even for UPDATE statements that don't change the value of the LOB column in SQL Server. During CDC, AWS DMS supports CLOB data types only in tables that include a primary key.  | 
|  BINARY  |  BYTES  | 
|  VARBINARY  |  BYTES  | 
|  VARBINARY (max)  |  BLOB IMAGE For SQL Server tables, AWS DMS updates LOB columns in the target even for UPDATE statements that don't change the value of the LOB column in SQL Server. To use this data type with AWS DMS, you must enable the use of BLOB data types for a specific task. AWS DMS supports BLOB data types only in tables that include a primary key.  | 
|  TIMESTAMP  |  BYTES  | 
|  UNIQUEIDENTIFIER  |  STRING  | 
|  HIERARCHYID   |  Use HIERARCHYID when replicating to a SQL Server target endpoint. Use WSTRING (250) when replicating to all other target endpoints.  | 
|  XML  |  NCLOB For SQL Server tables, AWS DMS updates LOB columns in the target even for UPDATE statements that don't change the value of the LOB column in SQL Server. To use this data type with AWS DMS, you must enable the use of NCLOB data types for a specific task. During CDC, AWS DMS supports NCLOB data types only in tables that include a primary key.  | 
|  GEOMETRY  |  Use GEOMETRY when replicating to target endpoints that support this data type. Use CLOB when replicating to target endpoints that don't support this data type.  | 
|  GEOGRAPHY  |  Use GEOGRAPHY when replicating to target endpoints that support this data type. Use CLOB when replicating to target endpoints that don't support this data type.  | 

AWS DMS doesn't support tables that include fields with the following data types. 
+ CURSOR
+ SQL\$1VARIANT
+ TABLE

**Note**  
User-defined data types are supported according to their base type. For example, a user-defined data type based on DATETIME is handled as a DATETIME data type.

# Capturing data changes for ongoing replication from SQL Server
<a name="CHAP_Source.SQLServer.CDC"></a>

This topic describes how to set up CDC replication on a SQL Server source.

**Topics**
+ [

## Capturing data changes for self-managed SQL Server on-premises or on Amazon EC2
](#CHAP_Source.SQLServer.CDC.Selfmanaged)
+ [

## Setting up ongoing replication on a cloud SQL Server DB instance
](#CHAP_Source.SQLServer.Configuration)

## Capturing data changes for self-managed SQL Server on-premises or on Amazon EC2
<a name="CHAP_Source.SQLServer.CDC.Selfmanaged"></a>

To capture changes from a source Microsoft SQL Server database, make sure that the database is configured for full backups. Configure the database either in full recovery mode or bulk-logged mode.

For a self-managed SQL Server source, AWS DMS uses the following:

**MS-Replication**  
To capture changes for tables with primary keys. You can configure this automatically by giving sysadmin privileges to the AWS DMS endpoint user on the source SQL Server instance. Or you can follow the steps in this section to prepare the source and use a user that doesn't have sysadmin privileges for the AWS DMS endpoint.

**MS-CDC**  
To capture changes for tables without primary keys. Enable MS-CDC at the database level and for all of the tables individually.

When setting up a SQL Server database for ongoing replication (CDC), you can do one of the following:
+ Set up ongoing replication using the sysadmin role.
+ Set up ongoing replication to not use the sysadmin role.

**Note**  
You can use the following script to find all table without a primary or a unique key:  

```
USE [DBname]
SELECT SCHEMA_NAME(schema_id) AS schema_name, name AS table_name
FROM sys.tables
WHERE OBJECTPROPERTY(object_id, 'TableHasPrimaryKey') = 0
        AND  OBJECTPROPERTY(object_id, 'TableHasUniqueCnst') = 0
ORDER BY schema_name, table_name;
```

### Setting up ongoing replication on a self-managed SQL Server
<a name="CHAP_Source.SQLServer.CDC.MSCDC"></a>

This section contains information about setting up ongoing replication on a self-managed SQL server with or without using the sysadmin role.

**Topics**
+ [

#### Setting up ongoing replication on a self-managed SQL Server: Using sysadmin role
](#CHAP_Source.SQLServer.CDC.MSCDC.Sysadmin)
+ [

#### Setting up ongoing replication on a standalone SQL Server: Without sysadmin role
](#CHAP_SupportScripts.SQLServer.standalone)
+ [

#### Setting up ongoing replication on a SQL Server in an availability group environment: Without sysadmin role
](#CHAP_SupportScripts.SQLServer.ag)

#### Setting up ongoing replication on a self-managed SQL Server: Using sysadmin role
<a name="CHAP_Source.SQLServer.CDC.MSCDC.Sysadmin"></a>

AWS DMS ongoing replication for SQL Server uses native SQL Server replication for tables with primary keys, and change data capture (CDC) for tables without primary keys.

Before setting up ongoing replication, see [Prerequisites for using ongoing replication (CDC) from a SQL Server source](CHAP_Source.SQLServer.md#CHAP_Source.SQLServer.Prerequisites). 

For tables with primary keys, AWS DMS can generally configure the required artifacts on the source. However, for SQL Server source instances that are self-managed, make sure to first configure the SQL Server distribution manually. After you do so, AWS DMS source users with sysadmin permission can automatically create the publication for tables with primary keys.

To check if distribution has already been configured, run the following command.

```
sp_get_distributor
```

If the result is `NULL` for column distribution, distribution isn't configured. You can use the following procedure to set up distribution.<a name="CHAP_Source.SQLServer.CDC.MSCDC.Setup"></a>

**To set up distribution**

1. Connect to your SQL Server source database using the SQL Server Management Studio (SSMS) tool.

1. Open the context (right-click) menu for the **Replication** folder, and choose **Configure Distribution**. The Configure Distribution wizard appears. 

1. Follow the wizard to enter the default values and create the distribution.<a name="CHAP_Source.SQLServer.CDC.MSCDC.Setup.CDC"></a>

**To set up CDC**

AWS DMS version 3.4.7 and greater can set up MS CDC for your database and all of your tables automatically if you aren't using a read-only replica. To use this feature, set the `SetUpMsCdcForTables` ECA to true. For information about ECAs, see [Endpoint settings](CHAP_Source.SQLServer.md#CHAP_Source.SQLServer.ConnectionAttrib).

For versions of AWS DMS earlier than 3.4.7, or for a read-only replica as a source, perform the following steps:

1. For tables without primary keys, set up MS-CDC for the database. To do so, use an account that has the sysadmin role assigned to it, and run the following command.

   ```
   use [DBname]
   EXEC sys.sp_cdc_enable_db
   ```

1. Next, set up MS-CDC for each of the source tables. For each table with unique keys but no primary key, run the following query to set up MS-CDC.

   ```
   exec sys.sp_cdc_enable_table
   @source_schema = N'schema_name',
   @source_name = N'table_name',
   @index_name = N'unique_index_name',
   @role_name = NULL,
   @supports_net_changes = 1
   GO
   ```

1. For each table with no primary key or no unique keys, run the following query to set up MS-CDC.

   ```
   exec sys.sp_cdc_enable_table
   @source_schema = N'schema_name',
   @source_name = N'table_name',
   @role_name = NULL
   GO
   ```

For more information on setting up MS-CDC for specific tables, see the [SQL Server documentation](https://msdn.microsoft.com/en-us/library/cc627369.aspx). 

#### Setting up ongoing replication on a standalone SQL Server: Without sysadmin role
<a name="CHAP_SupportScripts.SQLServer.standalone"></a>

This section describes how to set up ongoing replication for a standalone SQL Server database source that doesn't require the user account to have sysadmin privileges.

**Note**  
After running the steps in this section, the non-sysadmin DMS user will have permissions to do the following:  
Read changes from the online transactions log file
Disk access to read changes from transactional log backup files
Add or alter the publication which DMS uses
Add articles to the publication

1. Set up Microsoft SQL Server for Replication as described in [Capturing data changes for ongoing replication from SQL Server](#CHAP_Source.SQLServer.CDC).

1. Enable MS-REPLICATION on the source database. This can either be done manually or by running the task once as a sysadmin user.

1. Create the `awsdms` schema on the source database using the following script:

   ```
   use master
   go
   create schema awsdms
   go
   
   
   -- Create the table valued function [awsdms].[split_partition_list] on the Master database, as follows:
   USE [master]
   GO
   
   set ansi_nulls on
   go
   
   set quoted_identifier on
   go
   
   if (object_id('[awsdms].[split_partition_list]','TF')) is not null
   
   drop function [awsdms].[split_partition_list];
   
   go
   
   create function [awsdms].[split_partition_list]
   
   (
   
   @plist varchar(8000), --A delimited list of partitions
   
   @dlm nvarchar(1) --Delimiting character
   
   )
   
   returns @partitionsTable table --Table holding the BIGINT values of the string fragments
   
   (
   
   pid bigint primary key
   
   )   
   
   as
   
   begin
   
   declare @partition_id bigint;
   
   declare @dlm_pos integer;
   
   declare @dlm_len integer;
   
   set @dlm_len = len(@dlm);
   
   while (charindex(@dlm,@plist)>0)
   
   begin
   
   set @dlm_pos = charindex(@dlm,@plist);
   
   set @partition_id = cast( ltrim(rtrim(substring(@plist,1,@dlm_pos-1))) as bigint);
   
   insert into @partitionsTable (pid) values (@partition_id)
   
   set @plist = substring(@plist,@dlm_pos+@dlm_len,len(@plist));
   
   end
   
   set @partition_id = cast (ltrim(rtrim(@plist)) as bigint);
   
   insert into @partitionsTable (pid) values ( @partition_id );
   
   return
   
   end
   
   GO
   ```

1. Create the `[awsdms].[rtm_dump_dblog]` procedure on the Master database using the following script:

   ```
   use [MASTER]
   
   go
   
   if (object_id('[awsdms].[rtm_dump_dblog]','P')) is not null drop procedure [awsdms].[rtm_dump_dblog];
   go
   
   
   set ansi_nulls on
   go
   
   set quoted_identifier on
   GO
   
   
   
   CREATE procedure [awsdms].[rtm_dump_dblog]
   
   (
   
   @start_lsn varchar(32),
   
   @seqno integer,
   
   @filename varchar(260),
   
   @partition_list varchar(8000), -- A comma delimited list: P1,P2,... Pn
   
   @programmed_filtering integer,
   
   @minPartition bigint,
   
   @maxPartition bigint
   
   )
   
   as begin
   
   declare @start_lsn_cmp varchar(32); -- Stands against the GT comparator
   
   SET NOCOUNT ON -- – Disable "rows affected display"
   
   set @start_lsn_cmp = @start_lsn;
   
   if (@start_lsn_cmp) is null
   
   set @start_lsn_cmp = '00000000:00000000:0000';
   
   if (@partition_list is null)
   
   begin
   
   RAISERROR ('Null partition list waspassed',16,1);
   
   return
   
   end
   
   if (@start_lsn) is not null
   
   set @start_lsn = '0x'+@start_lsn;
   
   if (@programmed_filtering=0)
   
   
   
   SELECT
   
   [Current LSN],
   
   [operation],
   
   [Context],
   
   [Transaction ID],
   
   [Transaction Name],
   
   [Begin Time],
   
   [End Time],
   
   [Flag Bits],
   
   [PartitionID],
   
   [Page ID],
   
   [Slot ID],
   
   [RowLog Contents 0],
   
   [Log Record],
   
   [RowLog Contents 1]
   
   FROM
   
   fn_dump_dblog (
   
   @start_lsn, NULL, N'DISK', @seqno, @filename,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default)
   
   where [Current LSN] collate SQL_Latin1_General_CP1_CI_AS > @start_lsn_cmp collate SQL_Latin1_General_CP1_CI_AS
   
   and
   
   (
   
   ( [operation] in ('LOP_BEGIN_XACT','LOP_COMMIT_XACT','LOP_ABORT_XACT') )
   
   or
   
   ( [operation] in ('LOP_INSERT_ROWS','LOP_DELETE_ROWS','LOP_MODIFY_ROW')
   
   and
   
   ( ( [context] in ('LCX_HEAP','LCX_CLUSTERED','LCX_MARK_AS_GHOST') ) or ([context] = 'LCX_TEXT_MIX' and (datalength([RowLog Contents 0]) in (0,1))))
   
   and [PartitionID] in ( select * from master.awsdms.split_partition_list (@partition_list,','))
   
   )
   
   or
   
   ([operation] = 'LOP_HOBT_DDL')
   
   )
   
   
   else
   
   
   SELECT
   
   [Current LSN],
   
   [operation],
   
   [Context],
   
   [Transaction ID],
   
   [Transaction Name],
   
   [Begin Time],
   
   [End Time],
   
   [Flag Bits],
   
   [PartitionID],
   
   [Page ID],
   
   [Slot ID],
   
   [RowLog Contents 0],
   
   [Log Record],
   
   [RowLog Contents 1] -- After Image
   
   FROM
   
   fn_dump_dblog (
   
   @start_lsn, NULL, N'DISK', @seqno, @filename,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default,
   
   default, default, default, default, default, default, default)
   
   where [Current LSN] collate SQL_Latin1_General_CP1_CI_AS > @start_lsn_cmp collate SQL_Latin1_General_CP1_CI_AS
   
   and
   
   (
   
   ( [operation] in ('LOP_BEGIN_XACT','LOP_COMMIT_XACT','LOP_ABORT_XACT') )
   
   or
   
   ( [operation] in ('LOP_INSERT_ROWS','LOP_DELETE_ROWS','LOP_MODIFY_ROW')
   
   and
   
   ( ( [context] in ('LCX_HEAP','LCX_CLUSTERED','LCX_MARK_AS_GHOST') ) or ([context] = 'LCX_TEXT_MIX' and (datalength([RowLog Contents 0]) in (0,1))))
   
   and ([PartitionID] is not null) and ([PartitionID] >= @minPartition and [PartitionID]<=@maxPartition)
   
   )
   
   or
   
   ([operation] = 'LOP_HOBT_DDL')
   
   )
   
   
   
   SET NOCOUNT OFF -- Re-enable "rows affected display"
   
   end
   
   GO
   ```

1. Create the certificate on the Master database using the following script:

   ```
   Use [master]
   Go
   
   CREATE CERTIFICATE [awsdms_rtm_dump_dblog_cert] ENCRYPTION BY PASSWORD = N'@5trongpassword'
   
   WITH SUBJECT = N'Certificate for FN_DUMP_DBLOG Permissions';
   ```

1. Create the login from the certificate using the following script: 

   ```
   Use [master]
   Go
   
   CREATE LOGIN awsdms_rtm_dump_dblog_login FROM CERTIFICATE [awsdms_rtm_dump_dblog_cert];
   ```

1. Add the login to the sysadmin server role using the following script:

   ```
   ALTER SERVER ROLE [sysadmin] ADD MEMBER [awsdms_rtm_dump_dblog_login];
   ```

1. Add the signature to [master].[awsdms].[rtm\$1dump\$1dblog] using the certificate, using the following script: 

   ```
   Use [master]
   GO
   ADD SIGNATURE
   TO [master].[awsdms].[rtm_dump_dblog] BY CERTIFICATE [awsdms_rtm_dump_dblog_cert] WITH PASSWORD = '@5trongpassword';
   ```
**Note**  
If you recreate the stored procedure, you need to add the signature again.

1. Create the [awsdms].[rtm\$1position\$11st\$1timestamp] on the Master database using the following script:

   ```
   use [master]
       if object_id('[awsdms].[rtm_position_1st_timestamp]','P') is not null
       DROP PROCEDURE [awsdms].[rtm_position_1st_timestamp];
       go
       create procedure [awsdms].[rtm_position_1st_timestamp]
       (
       @dbname                sysname,      -- Database name
       @seqno                 integer,      -- Backup set sequence/position number within file
       @filename              varchar(260), -- The backup filename
       @1stTimeStamp          varchar(40)   -- The timestamp to position by
       ) 
       as begin
   
       SET NOCOUNT ON       -- Disable "rows affected display"
   
       declare @firstMatching table
       (
       cLsn varchar(32),
       bTim datetime
       )
   
       declare @sql nvarchar(4000)
       declare @nl                       char(2)
       declare @tb                       char(2)
       declare @fnameVar                 nvarchar(254) = 'NULL'
   
       set @nl  = char(10); -- New line
       set @tb  = char(9)   -- Tab separator
   
       if (@filename is not null)
       set @fnameVar = ''''+@filename +''''
   
       set @sql='use ['+@dbname+'];'+@nl+
       'select top 1 [Current LSN],[Begin Time]'+@nl+
       'FROM fn_dump_dblog (NULL, NULL, NULL, '+ cast(@seqno as varchar(10))+','+ @fnameVar+','+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default,'+@nl+
       @tb+'default, default, default, default, default, default, default)'+@nl+
       'where operation=''LOP_BEGIN_XACT''' +@nl+
       'and [Begin Time]>= cast('+''''+@1stTimeStamp+''''+' as datetime)'+@nl
   
       --print @sql
       delete from  @firstMatching 
       insert into @firstMatching  exec sp_executesql @sql    -- Get them all
   
       select top 1 cLsn as [matching LSN],convert(varchar,bTim,121) as [matching Timestamp] from @firstMatching;
   
       SET NOCOUNT OFF      -- Re-enable "rows affected display"
   
       end
       GO
   ```

1. Create the certificate on the Master database using the following script:

   ```
   Use [master]
   Go
   CREATE CERTIFICATE [awsdms_rtm_position_1st_timestamp_cert]
   ENCRYPTION BY PASSWORD = '@5trongpassword'
   WITH SUBJECT = N'Certificate for FN_POSITION_1st_TIMESTAMP Permissions';
   ```

1. Create the login from the certificate using the following script:

   ```
   Use [master]
   Go
   CREATE LOGIN awsdms_rtm_position_1st_timestamp_login FROM CERTIFICATE [awsdms_rtm_position_1st_timestamp_cert];
   ```

1. Add the login to the sysadmin role using the following script:

   ```
   ALTER SERVER ROLE [sysadmin] ADD MEMBER [awsdms_rtm_position_1st_timestamp_login];
   ```

1. Add the signature to [master].[awsdms].[rtm\$1position\$11st\$1timestamp] using the certificate, using the following script:

   ```
   Use [master]
       GO
       ADD SIGNATURE
       TO [master].[awsdms].[rtm_position_1st_timestamp]
       BY CERTIFICATE [awsdms_rtm_position_1st_timestamp_cert]
       WITH PASSWORD = '@5trongpassword';
   ```

1. Grant the DMS user execute access to the new stored procedure using the following script:

   ```
   use master
   go
   GRANT execute on [awsdms].[rtm_position_1st_timestamp] to dms_user;
   ```

1. Create a user with the following permissions and roles in each of the following databases:
**Note**  
You should create the dmsnosysadmin user account with the same SID on each replica. The following SQL query can help verify the dmsnosysadmin account SID value on each replica. For more information about creating a user, see [ CREATE USER (Transact-SQL) ](https://learn.microsoft.com/en-us/sql/t-sql/statements/create-user-transact-sql) in the [Microsoft SQL server documentation](https://learn.microsoft.com/en-us/sql/). For more information about creating SQL user accounts for Azure SQL database, see [ Active geo-replication ](https://learn.microsoft.com/en-us/azure/azure-sql/database/active-geo-replication-overview).

   ```
   use master
   go
   grant select on sys.fn_dblog to [DMS_user]
   grant view any definition to [DMS_user]
   grant view server state to [DMS_user]--(should be granted to the login).
   grant execute on sp_repldone to [DMS_user]
   grant execute on sp_replincrementlsn to [DMS_user]
   grant execute on sp_addpublication to [DMS_user]
   grant execute on sp_addarticle to [DMS_user]
   grant execute on sp_articlefilter to [DMS_user]
   grant select on [awsdms].[split_partition_list] to [DMS_user]
   grant execute on [awsdms].[rtm_dump_dblog] to [DMS_user]
   ```

   ```
   use msdb
   go
   grant select on msdb.dbo.backupset to self_managed_user
   grant select on msdb.dbo.backupmediafamily to self_managed_user
   grant select on msdb.dbo.backupfile to self_managed_user
   ```

   Run the following script on the source database:

   ```
   use Source_DB
       Go
       EXEC sp_addrolemember N'db_owner', N'DMS_user'
   ```

1. Lastly, add an Extra Connection Attribute (ECA) to the source SQL Server endpoint:

   ```
   enableNonSysadminWrapper=true;
   ```

#### Setting up ongoing replication on a SQL Server in an availability group environment: Without sysadmin role
<a name="CHAP_SupportScripts.SQLServer.ag"></a>

This section describes how to set up ongoing replication for a SQL Server database source in an availability group environment that doesn't require the user account to have sysadmin privileges.

**Note**  
After running the steps in this section, the non-sysadmin DMS user will have permissions to do the following:  
Read changes from the online transactions log file
Disk access to read changes from transactional log backup files
Add or alter the publication which DMS uses
Add articles to the publication

**To set up ongoing replication without using the sysadmin user in an Availability Group environment**

1. Set up Microsoft SQL Server for Replication as described in [Capturing data changes for ongoing replication from SQL Server](#CHAP_Source.SQLServer.CDC).

1. Enable MS-REPLICATION on the source database. This can either be done manually or by running the task once using a sysadmin user.
**Note**  
You should either configure the MS-REPLICATION distributor as local or in a way that allows access to non-sysadmin users via the associated linked server.

1. If the **Exclusively use sp\$1repldone within a single task** endpoint option is enabled, stop the MS-REPLICATION Log Reader job.

1. Perform the following steps on each replica:

   1. Create the `[awsdms]`[awsdms] schema in the master database:

      ```
      CREATE SCHEMA [awsdms]
      ```

   1. Create the `[awsdms].[split_partition_list]` table valued function on the Master database:

      ```
      USE [master]
      GO
      
      SET ansi_nulls on
      GO
        
      SET quoted_identifier on
      GO
      
      IF (object_id('[awsdms].[split_partition_list]','TF')) is not null
        DROP FUNCTION [awsdms].[split_partition_list];
      GO
      
      CREATE FUNCTION [awsdms].[split_partition_list] 
      ( 
        @plist varchar(8000),    --A delimited list of partitions    
        @dlm nvarchar(1)    --Delimiting character
      ) 
      RETURNS @partitionsTable table --Table holding the BIGINT values of the string fragments
      (
        pid bigint primary key
      ) 
      AS 
      BEGIN
        DECLARE @partition_id bigint;
        DECLARE @dlm_pos integer;
        DECLARE @dlm_len integer;  
        SET @dlm_len = len(@dlm);
        WHILE (charindex(@dlm,@plist)>0)
        BEGIN 
          SET @dlm_pos = charindex(@dlm,@plist);
          SET @partition_id = cast( ltrim(rtrim(substring(@plist,1,@dlm_pos-1))) as bigint);
          INSERT into @partitionsTable (pid) values (@partition_id)
          SET @plist = substring(@plist,@dlm_pos+@dlm_len,len(@plist));
        END 
        SET @partition_id = cast (ltrim(rtrim(@plist)) as bigint);
        INSERT into @partitionsTable (pid) values (  @partition_id  );
        RETURN
      END
      GO
      ```

   1. Create the `[awsdms].[rtm_dump_dblog]` procedure on the Master database:

      ```
      USE [MASTER] 
      GO
      
      IF (object_id('[awsdms].[rtm_dump_dblog]','P')) is not null
        DROP PROCEDURE [awsdms].[rtm_dump_dblog]; 
      GO
      
      SET ansi_nulls on
      GO 
      
      SET quoted_identifier on 
      GO
                                          
      CREATE PROCEDURE [awsdms].[rtm_dump_dblog]
      (
        @start_lsn            varchar(32),
        @seqno                integer,
        @filename             varchar(260),
        @partition_list       varchar(8000), -- A comma delimited list: P1,P2,... Pn
        @programmed_filtering integer,
        @minPartition         bigint,
        @maxPartition         bigint
      ) 
      AS 
      BEGIN
      
        DECLARE @start_lsn_cmp varchar(32); -- Stands against the GT comparator
      
        SET NOCOUNT ON  -- Disable "rows affected display"
      
        SET @start_lsn_cmp = @start_lsn;
        IF (@start_lsn_cmp) is null
          SET @start_lsn_cmp = '00000000:00000000:0000';
      
        IF (@partition_list is null)
          BEGIN
            RAISERROR ('Null partition list was passed',16,1);
            return
            --set @partition_list = '0,';    -- A dummy which is never matched
          END
      
        IF (@start_lsn) is not null
          SET @start_lsn = '0x'+@start_lsn;
      
        IF (@programmed_filtering=0)
          SELECT
            [Current LSN],
            [operation],
            [Context],
            [Transaction ID],
            [Transaction Name],
            [Begin Time],
            [End Time],
            [Flag Bits],
            [PartitionID],
            [Page ID],
            [Slot ID],
            [RowLog Contents 0],
            [Log Record],
            [RowLog Contents 1] -- After Image
          FROM
            fn_dump_dblog (
              @start_lsn, NULL, N'DISK', @seqno, @filename,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default,
              default, default, default, default, default, default, default)
          WHERE 
            [Current LSN] collate SQL_Latin1_General_CP1_CI_AS > @start_lsn_cmp collate SQL_Latin1_General_CP1_CI_AS -- This aims for implementing FN_DBLOG based on GT comparator.
            AND
            (
              (  [operation] in ('LOP_BEGIN_XACT','LOP_COMMIT_XACT','LOP_ABORT_XACT') )
              OR
              (  [operation] in ('LOP_INSERT_ROWS','LOP_DELETE_ROWS','LOP_MODIFY_ROW')
                AND
                ( ( [context]   in ('LCX_HEAP','LCX_CLUSTERED','LCX_MARK_AS_GHOST') ) or ([context] = 'LCX_TEXT_MIX') )
                AND       
                [PartitionID] in ( select * from master.awsdms.split_partition_list (@partition_list,','))
              )
            OR
            ([operation] = 'LOP_HOBT_DDL')
          )
          ELSE
            SELECT
              [Current LSN],
              [operation],
              [Context],
              [Transaction ID],
              [Transaction Name],
              [Begin Time],
              [End Time],
              [Flag Bits],
              [PartitionID],
              [Page ID],
              [Slot ID],
              [RowLog Contents 0],
              [Log Record],
              [RowLog Contents 1] -- After Image
            FROM
              fn_dump_dblog (
                @start_lsn, NULL, N'DISK', @seqno, @filename,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default,
                default, default, default, default, default, default, default)
            WHERE [Current LSN] collate SQL_Latin1_General_CP1_CI_AS > @start_lsn_cmp collate SQL_Latin1_General_CP1_CI_AS -- This aims for implementing FN_DBLOG based on GT comparator.
            AND
            (
              (  [operation] in ('LOP_BEGIN_XACT','LOP_COMMIT_XACT','LOP_ABORT_XACT') )
              OR
              (  [operation] in ('LOP_INSERT_ROWS','LOP_DELETE_ROWS','LOP_MODIFY_ROW')
                AND
                ( ( [context]   in ('LCX_HEAP','LCX_CLUSTERED','LCX_MARK_AS_GHOST') ) or ([context] = 'LCX_TEXT_MIX') )
                AND ([PartitionID] is not null) and ([PartitionID] >= @minPartition and [PartitionID]<=@maxPartition)
              )
              OR
              ([operation] = 'LOP_HOBT_DDL')
            )
            SET NOCOUNT OFF -- Re-enable "rows affected display"
      END
      GO
      ```

   1. Create a certificate on the Master Database:

      ```
      USE [master]
      GO
      CREATE CERTIFICATE [awsdms_rtm_dump_dblog_cert]
        ENCRYPTION BY PASSWORD = N'@hardpassword1'
        WITH SUBJECT = N'Certificate for FN_DUMP_DBLOG Permissions'
      ```

   1. Create a login from the certificate:

      ```
      USE [master]
      GO
      CREATE LOGIN awsdms_rtm_dump_dblog_login FROM CERTIFICATE
        [awsdms_rtm_dump_dblog_cert];
      ```

   1. Add the login to the sysadmin server role:

      ```
      ALTER SERVER ROLE [sysadmin] ADD MEMBER [awsdms_rtm_dump_dblog_login];
      ```

   1. Add the signature to the [master].[awsdms].[rtm\$1dump\$1dblog] procedure using the certificate:

      ```
      USE [master]
      GO
      
      ADD SIGNATURE
        TO [master].[awsdms].[rtm_dump_dblog]
        BY CERTIFICATE [awsdms_rtm_dump_dblog_cert]
        WITH PASSWORD = '@hardpassword1';
      ```
**Note**  
If you recreate the stored procedure, you need to add the signature again.

   1. Create the `[awsdms].[rtm_position_1st_timestamp]` procedure on the Master database:

      ```
      USE [master]
      IF object_id('[awsdms].[rtm_position_1st_timestamp]','P') is not null
        DROP PROCEDURE [awsdms].[rtm_position_1st_timestamp];
      GO
      CREATE PROCEDURE [awsdms].[rtm_position_1st_timestamp]
      (
        @dbname                sysname,      -- Database name
        @seqno                 integer,      -- Backup set sequence/position number within file
        @filename              varchar(260), -- The backup filename
        @1stTimeStamp          varchar(40)   -- The timestamp to position by
      ) 
      AS 
      BEGIN
        SET NOCOUNT ON       -- Disable "rows affected display"
      
        DECLARE @firstMatching table
        (
          cLsn varchar(32),
          bTim datetime
        )
        DECLARE @sql nvarchar(4000)
        DECLARE @nl                       char(2)
        DECLARE @tb                       char(2)
        DECLARE @fnameVar                 sysname = 'NULL'
      
        SET @nl  = char(10); -- New line
        SET @tb  = char(9)   -- Tab separator
      
        IF (@filename is not null)
          SET @fnameVar = ''''+@filename +''''
        SET @filename = ''''+@filename +''''
        SET @sql='use ['+@dbname+'];'+@nl+
          'SELECT TOP 1 [Current LSN],[Begin Time]'+@nl+
          'FROM fn_dump_dblog (NULL, NULL, NULL, '+ cast(@seqno as varchar(10))+','+ @filename +','+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default,'+@nl+
          @tb+'default, default, default, default, default, default, default)'+@nl+
          'WHERE operation=''LOP_BEGIN_XACT''' +@nl+
          'AND [Begin Time]>= cast('+''''+@1stTimeStamp+''''+' as datetime)'+@nl
      
          --print @sql
          DELETE FROM @firstMatching 
          INSERT INTO @firstMatching  exec sp_executesql @sql    -- Get them all
          SELECT TOP 1 cLsn as [matching LSN],convert(varchar,bTim,121) AS[matching Timestamp] FROM @firstMatching;
      
          SET NOCOUNT OFF      -- Re-enable "rows affected display"
      
      END
      GO
      ```

   1. Create a certificate on the Master database:

      ```
      USE [master]
      GO
      CREATE CERTIFICATE [awsdms_rtm_position_1st_timestamp_cert]
        ENCRYPTION BY PASSWORD = N'@hardpassword1'
        WITH SUBJECT = N'Certificate for FN_POSITION_1st_TIMESTAMP Permissions';
      ```

   1. Create a login from the certificate:

      ```
      USE [master]
      GO
      CREATE LOGIN awsdms_rtm_position_1st_timestamp_login FROM CERTIFICATE
        [awsdms_rtm_position_1st_timestamp_cert];
      ```

   1. Add the login to the sysadmin server role:

      ```
      ALTER SERVER ROLE [sysadmin] ADD MEMBER [awsdms_rtm_position_1st_timestamp_login];
      ```

   1. Add the signature to the `[master].[awsdms].[rtm_position_1st_timestamp]` procedure using the certificate:

      ```
      USE [master]
      GO
      ADD SIGNATURE
        TO [master].[awsdms].[rtm_position_1st_timestamp]
        BY CERTIFICATE [awsdms_rtm_position_1st_timestamp_cert]
        WITH PASSWORD = '@hardpassword1';
      ```
**Note**  
If you recreate the stored procedure, you need to add the signature again.

   1. Create a user with the following permissions/roles in each of the following databases:
**Note**  
You should create the dmsnosysadmin user account with the same SID on each replica. The following SQL query can help verify the dmsnosysadmin account SID value on each replica. For more information about creating a user, see [ CREATE USER (Transact-SQL) ](https://learn.microsoft.com/en-us/sql/t-sql/statements/create-user-transact-sql) in the [Microsoft SQL server documentation](https://learn.microsoft.com/en-us/sql/). For more information about creating SQL user accounts for Azure SQL database, see [ Active geo-replication ](https://learn.microsoft.com/en-us/azure/azure-sql/database/active-geo-replication-overview).

      ```
      SELECT @@servername servername, name, sid, create_date, modify_date
        FROM sys.server_principals
        WHERE name = 'dmsnosysadmin';
      ```

   1. Grant permissions on the master database on each replica:

      ```
      USE master
      GO 
      
      GRANT select on sys.fn_dblog to dmsnosysadmin;
      GRANT view any definition to dmsnosysadmin;
      GRANT view server state to dmsnosysadmin -- (should be granted to the login).
      GRANT execute on sp_repldone to dmsnosysadmin;
      GRANT execute on sp_replincrementlsn to dmsnosysadmin;
      GRANT execute on sp_addpublication to dmsnosysadmin;
      GRANT execute on sp_addarticle to dmsnosysadmin;
      GRANT execute on sp_articlefilter to dmsnosysadmin;
      GRANT select on [awsdms].[split_partition_list] to dmsnosysadmin;
      GRANT execute on [awsdms].[rtm_dump_dblog] to dmsnosysadmin;
      GRANT execute on [awsdms].[rtm_position_1st_timestamp] to dmsnosysadmin;
      ```

   1. Grant permissions on the msdb database on each replica:

      ```
      USE msdb
      GO
      GRANT select on msdb.dbo.backupset TO self_managed_user
      GRANT select on msdb.dbo.backupmediafamily TO self_managed_user
      GRANT select on msdb.dbo.backupfile TO self_managed_user
      ```

   1. Add the `db_owner` role to `dmsnosysadmin` on the source database. Because the database is synchronized, you can add the role on the primary replica only.

      ```
      use <source DB>
      GO 
      EXEC sp_addrolemember N'db_owner', N'dmsnosysadmin'
      ```

## Setting up ongoing replication on a cloud SQL Server DB instance
<a name="CHAP_Source.SQLServer.Configuration"></a>

This section describes how to set up CDC on a cloud-hosted SQL Server database instance. A cloud-hosted SQL server instance is an instance running on Amazon RDS for SQL Server, an Azure SQL Manged Instance, or any other managed cloud SQL Server instance. For information about limitations for ongoing replication for each database type, see [Limitations on using SQL Server as a source for AWS DMS](CHAP_Source.SQLServer.md#CHAP_Source.SQLServer.Limitations). 

Before setting up ongoing replication, see [Prerequisites for using ongoing replication (CDC) from a SQL Server source](CHAP_Source.SQLServer.md#CHAP_Source.SQLServer.Prerequisites). 

Unlike self-managed Microsoft SQL Server sources, Amazon RDS for SQL Server doesn't support MS-Replication. Therefore, AWS DMS needs to use MS-CDC for tables with or without primary keys.

Amazon RDS doesn't grant sysadmin privileges for setting replication artifacts that AWS DMS uses for ongoing changes in a source SQL Server instance. Make sure to turn on MS-CDC for the Amazon RDS instance (using master user privileges) as in the following procedure.

**To turn on MS-CDC for a cloud SQL Server DB instance**

1. Run one of the following queries at the database level.

   For an RDS for SQL Server DB instance, use this query.

   ```
   exec msdb.dbo.rds_cdc_enable_db 'DB_name'
   ```

   For an Azure SQL managed DB instance, use this query.

   ```
   USE DB_name 
   GO 
   EXEC sys.sp_cdc_enable_db 
   GO
   ```

1. For each table with a primary key, run the following query to turn on MS-CDC.

   ```
   exec sys.sp_cdc_enable_table
   @source_schema = N'schema_name',
   @source_name = N'table_name',
   @role_name = NULL,
   @supports_net_changes = 1
   GO
   ```

   For each table with unique keys but no primary key, run the following query to turn on MS-CDC.

   ```
   exec sys.sp_cdc_enable_table
   @source_schema = N'schema_name',
   @source_name = N'table_name',
   @index_name = N'unique_index_name',
   @role_name = NULL,
   @supports_net_changes = 1
   GO
   ```

    For each table with no primary key nor unique keys, run the following query to turn on MS-CDC.

   ```
   exec sys.sp_cdc_enable_table
   @source_schema = N'schema_name',
   @source_name = N'table_name',
   @role_name = NULL
   GO
   ```

1. Set the retention period:
   + For RDS for SQL Server instances that are replicating using DMS version 3.5.3 and above, make sure that the retention period is set to the default value of 5 seconds. If you’re upgrading or moving from DMS 3.5.2 and below to DMS 3.5.3 and above, change the polling interval value after the tasks are running on the new or upgraded instance. The following script sets the retention period to 5 seconds:

     ```
     use dbname
     EXEC sys.sp_cdc_change_job @job_type = 'capture' ,@pollinginterval = 5
     exec sp_cdc_stop_job 'capture'
     exec sp_cdc_start_job 'capture'
     ```
   + The parameter `@pollinginterval` is measured in seconds with a recommended value set to 86399. This means that the transaction log retains changes for 86,399 seconds (one day) when `@pollinginterval = 86399`. The procedure `exec sp_cdc_start_job 'capture'` initiates the settings.
**Note**  
With some versions of SQL Server, if the value of `pollinginterval` is set to more than 3599 seconds, the value resets to the default five seconds. When this happens, T-Log entries are purged before AWS DMS can read them. To determine which SQL Server versions are affected by this known issue, see [this Microsoft KB article](https://support.microsoft.com/en-us/topic/kb4459220-fix-incorrect-results-occur-when-you-convert-pollinginterval-parameter-from-seconds-to-hours-in-sys-sp-cdc-scan-in-sql-server-dac8aefe-b60b-7745-f987-582dda2cfa78).

     If you are using Amazon RDS with Multi-AZ, make sure that you also set your secondary to have the right values in case of failover.

     ```
     exec rdsadmin..rds_set_configuration 'cdc_capture_pollinginterval' , <5 or preferred value>
     ```

**To maintain the retention period when an AWS DMS replication task is stopped for more than one hour**
**Note**  
The following steps aren’t needed for a RDS for SQL Server source replicating using DMS 3.5.3 and above.

1. Stop the job truncating the transaction logs by using the following command. 

   ```
   exec sp_cdc_stop_job 'capture'
   ```

1. Find your task on the AWS DMS console and resume the task.

1. Choose the **Monitoring** tab, and check the `CDCLatencySource` metric. 

1. After the `CDCLatencySource` metric equals 0 (zero) and stays there, restart the job truncating the transaction logs using the following command.

   ```
   exec sp_cdc_start_job 'capture'
   ```

Remember to start the job that truncates SQL Server transaction logs. Otherwise, storage on your SQL Server instance might fill up.

### Recommended settings when using RDS for SQL Server as a source for AWS DMS
<a name="CHAP_Source.SQLServer.Configuration.Settings"></a>

#### For AWS DMS 3.5.3 and above
<a name="CHAP_Source.SQLServer.Configuration.Settings.353"></a>

**Note**  
The initial release of the RDS for SQL Server log backup feature is enabled by default for endpoints that you created or modified after the release of DMS version 3.5.3. To use this feature for existing endpoints, modify the endpoint without making any changes.

AWS DMS version 3.5.3 introduces support for reading from log backups. DMS primarily relies on reading from the active transaction logs to replicate events. If a transaction is backed up before DMS can read it from the active log, the task accesses the RDS backups on-demand and reads from the subsequent backup logs until it catches up to the active transaction log. To ensure that DMS has access to log backups, set the RDS automated backup retention period to at least one day. For information about setting the automated backup retention period, see [ Backup retention period](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ManagingAutomatedBackups.html#USER_WorkingWithAutomatedBackups.BackupRetention) in the *Amazon RDS User Guide*.

A DMS task accessing log backups utilizes storage on the RDS instance. Note that the task only accesses the log backups needed for replication. Amazon RDS removes these downloaded backups in a couple of hours. This removal doesn't affect the Amazon RDS backups retained in Amazon S3, or Amazon RDS `RESTORE DATABASE` functionality. It is advisable to allocate additional storage on your RDS for SQL Server source if you intend to replicate using DMS. One way to estimate the amount of storage needed is to identify the backup from which DMS will start or resume replicating from, and add up the file sizes of all the subsequent backups using the RDS `tlog backup` metadata function. For more information about the `tlog backup` function, see [ Listing available transaction log backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER.SQLServer.AddlFeat.TransactionLogAccess.html#USER.SQLServer.AddlFeat.TransactionLogAccess.Listing) in the *Amazon RDS User Guide*. 

Alternately, you may choose to enable storage autoscaling and/ or trigger storage scaling based on the CloudWatch `FreeStorageSpace` metric for your Amazon RDS instance.

We strongly recommend that you don’t start or resume from a point too far back in the transaction log backups, as it might lead to storage on your SQL Server instance filling up. In such cases, it is advisable to start a full load. Replicating from the transaction log backup is slower than reading from the active transaction logs. For more information, see [Transaction log backup processing for RDS for SQL Server](CHAP_Troubleshooting_Latency_Source_SQLServer.md#CHAP_Troubleshooting_Latency_Source_SQLServer_backup).

Note that accessing the log backups requires additional privileges. For more information, see as detailed in [Set up permissions for ongoing replication from a cloud SQL Server database](CHAP_Source.SQLServer.md#CHAP_Source.SQLServer.Permissions.Cloud) Make sure that you grant these privileges before the task starts replicating.

#### For AWS DMS 3.5.2 and below
<a name="CHAP_Source.SQLServer.Configuration.Settings.352"></a>

When you work with Amazon RDS for SQL Server as a source, the MS-CDC capture job relies on the parameters `maxscans` and `maxtrans`. These parameters govern the maximum number of scans that the MS-CDC capture does on the transaction log and the number of transactions that are processed for each scan.

For databases, where a number of transactions is greater than `maxtrans*maxscans`, increasing the `polling_interval` value can cause an accumulation of active transaction log records. In turn, this accumulation can lead to an increase in the size of the transaction log.

Note that AWS DMS does not rely on the MS-CDC capture job. The MS-CDC capture job marks the transaction log entries as having been processed. This allows the transaction log backup job to remove the entries from the transaction log.

We recommend that you monitor the size of the transaction log and the success of the MS-CDC jobs. If the MS-CDC jobs fail, the transaction log could grow excessively and cause AWS DMS replication failures. You can monitor MS-CDC capture job errors using the `sys.dm_cdc_errors` dynamic management view in the source database. You can monitor the transaction log size using the `DBCC SQLPERF(LOGSPACE)` management command.

**To address the transaction log increase that is caused by MS-CDC**

1. Check the `Log Space Used %` for the database AWS DMS is replicating from and validate that it increases continuously.

   ```
   DBCC SQLPERF(LOGSPACE)
   ```

1. Identify what is blocking the transaction log backup process.

   ```
   Select log_reuse_wait, log_reuse_wait_desc, name from sys.databases where name = db_name();
   ```

   If the `log_reuse_wait_desc` value equals `REPLICATION`, the log backup retention is caused by the latency in MS-CDC.

1. Increase the number of events processed by the capture job by increasing the `maxtrans` and `maxscans` parameter values.

   ```
   EXEC sys.sp_cdc_change_job @job_type = 'capture' ,@maxtrans = 5000, @maxscans = 20 
   exec sp_cdc_stop_job 'capture'
   exec sp_cdc_start_job 'capture'
   ```

To address this issue, set the values of `maxscans` and `maxtrans` so that `maxtrans*maxscans` is equal to the average number of events generated for tables that AWS DMS replicates from the source database for each day.

If you set these parameters higher than the recommended value, the capture jobs process all events in the transaction logs. If you set these parameters below the recommended value, MS-CDC latency increases and your transaction log grows.

Identifying appropriate values for `maxscans` and `maxtrans` can be difficult because changes in workload produce varying number of events. In this case, we recommend that you set up monitoring on MS-CDC latency. For more information, see [ Monitor the process](https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server?view=sql-server-ver15#Monitor) in SQL Server documentation. Then configure `maxtrans` and `maxscans` dynamically based on the monitoring results.

If the AWS DMS task is unable to find the log sequence numbers (LSNs) needed to resume or continue the task, the task may fail and require a complete reload.

**Note**  
When using AWS DMS to replicate data from an RDS for SQL Server source, you may encounter errors when trying to resume replication after a stop-start event of the Amazon RDS instance. This is due to the SQL Server Agent process restarting the capture job process when it restarts after the stop-start event. This bypasses the MS-CDC polling interval.  
Because of this, on databases with transaction volumes lower than the MS-CDC capture job processing, this can cause data to be processed or marked as replicated and backed up before AWS DMS can resume from where it stopped, resulting in the following error:  

```
[SOURCE_CAPTURE ]E: Failed to access LSN '0000dbd9:0006f9ad:0003' in the backup log sets since BACKUP/LOG-s are not available. [1020465] (sqlserver_endpoint_capture.c:764)
```
To mitigate this issue, set the `maxtrans` and `maxscans` values as recommended prior.

# Using Microsoft Azure SQL database as a source for AWS DMS
<a name="CHAP_Source.AzureSQL"></a>

With AWS DMS, you can use Microsoft Azure SQL Database as a source in much the same way as you do SQL Server. AWS DMS supports, as a source, the same list of database versions that are supported for SQL Server running on-premises or on an Amazon EC2 instance. 

For more information, see [Using a Microsoft SQL Server database as a source for AWS DMS](CHAP_Source.SQLServer.md).

**Note**  
AWS DMS doesn't support change data capture operations (CDC) with Azure SQL Database.

# Using Microsoft Azure SQL Managed Instance as a source for AWS DMS
<a name="CHAP_Source.AzureMgd"></a>

With AWS DMS, you can use Microsoft Azure SQL Managed Instance as a source in much the same way as you do SQL Server. AWS DMS supports, as a source, the same list of database versions that are supported for SQL Server running on-premises or on an Amazon EC2 instance. 

For more information, see [Using a Microsoft SQL Server database as a source for AWS DMS](CHAP_Source.SQLServer.md).

# Using Microsoft Azure Database for PostgreSQL flexible server as a source for AWS DMS
<a name="CHAP_Source.AzureDBPostgreSQL"></a>

With AWS DMS, you can use Microsoft Azure Database for PostgreSQL flexible server as a source in much the same way as you do PostgreSQL.

For information about versions of Microsoft Azure Database for PostgreSQL flexible server that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

## Setting up Microsoft Azure for PostgreSQL flexible server for logical replication and decoding
<a name="CHAP_Source.AzureDBPostgreSQL.setup"></a>

You can use logical replication and decoding features in Microsoft Azure Database for PostgreSQL flexible server during database migration.

For logical decoding, DMS uses either the `test_decoding` or `pglogical` plugin. If the `pglogical` plugin is available on a source PostgreSQL database, DMS creates a replication slot using `pglogical`, otherwise the `test_decoding` plugin is used. 

To configure your Microsoft Azure for PostgreSQL flexible server as a source endpoint for DMS, perform the following steps: 

1. Open the Server Parameters page on the portal.

1. Set the `wal_level` server parameter to `LOGICAL`.

1. If you want to use the `pglogical` extension, set the `shared_preload_libraries` and `azure.extensions` parameters to `pglogical`.

1. Set the `max_replication_slots` parameter to the maximum number of DMS tasks that you plan to run concurrently. In Microsoft Azure, the default value for this parameter is 10. This parameter's maximum value depends on the available memory of your PostgreSQL instance, allowing for between 2 and 8 replication slots per GB of memory.

1. Set the `max_wal_senders` parameter to a value greater than 1. The `max_wal_senders` parameter sets the number of concurrent tasks that can run. The default value is 10.

1. Set the `max_worker_processes` parameter value to at least 16. Otherwise, you may see errors such as the following:

   ```
   WARNING: out of background worker slots.
   ```

1. Save the changes. Restart the server to apply the changes.

1. Confirm that your PostgreSQL instance allows network traffic from your connecting resource.

1. Grant an existing user replication permissions, or create a new user with replication permissions, using the following commands. 
   + Grant an existing user replication permissions using the following command:

     ```
     ALTER USER <existing_user> WITH REPLICATION;
     ```
   + Create a new user with replication permissions using the following command: 

     ```
     CREATE USER aws_dms_user PASSWORD 'aws_dms_user_password';
     GRANT azure_pg_admin to aws_dms_user;
     ALTER ROLE aws_dms_user REPLICATION LOGIN;
     ```

For more information about logical replication with PostgreSQL, see the following topics:
+ [Enabling change data capture (CDC) using logical replication](CHAP_Source.PostgreSQL.md#CHAP_Source.PostgreSQL.Security)
+ [Using native CDC start points to set up a CDC load of a PostgreSQL source](CHAP_Source.PostgreSQL.md#CHAP_Source.PostgreSQL.v10)
+ [ Logical replication and logical decoding in Azure Database for PostgreSQL - Flexible Server](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-logical) in the [Azure Database for PostgreSQL documentation](https://learn.microsoft.com/en-us/azure/postgresql/).

# Using Microsoft Azure Database for MySQL flexible server as a source for AWS DMS
<a name="CHAP_Source.AzureDBMySQL"></a>

With AWS DMS, you can use Microsoft Azure Database for MySQL flexible server as a source in much the same way as you do MySQL.

For information about versions of Microsoft Azure Database for MySQL flexible server that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

For more information about using a customer-managed MySQL-compatible database with AWS DMS, see [Using a self-managed MySQL-compatible database as a source for AWS DMS](CHAP_Source.MySQL.md#CHAP_Source.MySQL.CustomerManaged).

## Limitations when using Azure MySQL as a source for AWS Database Migration Service
<a name="CHAP_Source.AzureDBMySQL.limitations"></a>
+ The defaut value for the Azure MySQL flexible server system variable `sql_generate_invisible_primary_key` is `ON`, and the server automatically adds a generated invisible primary key (GIPK) to any table that is created without an explicit primary key. AWS DMS doesn’t support ongoing replication for MySQL tables with GIPK constraints.

# Using OCI MySQL Heatwave as a source for AWS DMS
<a name="CHAP_Source.heatwave"></a>

With AWS DMS, you can use OCI MySQL Heatwave as a source in much the same way as you do MySQL. Using OCI MySQL Heatwave as a source requires a few additional configuration changes.

For information about versions of OCI MySQL Heatwave that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

## Setting up OCI MySQL Heatwave for logical replication
<a name="CHAP_Source.heatwave.setup"></a>

To configure your OCI MySQL Heatwave instance as a source endpoint for DMS, do the following:

1. Sign in to OCI Console, and open the main hamburger menu (≡) in the top left corner.

1. Choose **Databases**, **DB Systems**.

1. Open the **Configurations** menu.

1. Choose **Create configuration**.

1. Enter a configuration name, such as **dms\$1configuration**.

1. Choose the shape of your current OCI MySQL Heatwave instance. You can find the shape on the instance's **DB system configuration** properties tab under the **DB system configuration:Shape** section.

1. In the **User variables** section, choose the `binlog_row_value_options` system variable. Its default value is `PARTIAL_JSON`. Clear the value.

1. Choose the **Create** button.

1. Open your OCI MySQLHeatwave instance, and choose the **Edit** button.

1. In the **Configuration** section, choose the **Change configuration** button, and choose the shape configuration that you created in step 4.

1. Once the changes take effect, your instance is ready for logical replication.

# Using Google Cloud for MySQL as a source for AWS DMS
<a name="CHAP_Source.GC"></a>

With AWS DMS, you can use Google Cloud for MySQL as a source in much the same way as you do MySQL. 

For information about versions of GCP MySQL that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

For more information, see [Using a MySQL-compatible database as a source for AWS DMS](CHAP_Source.MySQL.md).

**Note**  
Support for GCP MySQL 8.0 as a source is available in AWS DMS version 3.4.6.  
AWS DMS doesn't support the SSL mode `verify-full` for GCP for MySQL instances.  
The GCP MySQL security setting `Allow only SSL connections` isn't supported, because it requires both server and client certificate verification. AWS DMS only supports server certificate verification.  
AWS DMS supports the default GCP CloudSQL for MySQL value of `CRC32` for the `binlog_checksum` database flag.

# Using Google Cloud for PostgreSQL as a source for AWS DMS
<a name="CHAP_Source.GCPostgres"></a>

With AWS DMS, you can use Google Cloud for PostgreSQL as a source in much the same way as you do self-managed PostgreSQL databases.

For information about versions of GCP PostgreSQL that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

For more information, see [Using a PostgreSQL database as an AWS DMS source](CHAP_Source.PostgreSQL.md).

## Set up Google Cloud for PostgreSQL for logical replication and decoding
<a name="CHAP_Source.GCPostgres.setup"></a>

You can use logical replication and decoding features in Google Cloud SQL for PostgreSQL during database migration.

For logical decoding, DMS uses one of the following plugins:
+ `test_decoding`
+ `pglogical`

If the `pglogical` plugin is available on a source PostgreSQL database, DMS creates a replication slot using `pglogical`, otherwise the `test_decoding` plugin is used. 

Note the following about using logical decoding with AWS DMS:

1. With Google Cloud SQL for PostgreSQL, enable logical decoding by setting the `cloudsql.logical_decoding` flag to `on`.

1. To enable `pglogical`, set the `cloudsql.enable_pglogical` flag to `on`, and restart the database.

1. To use logical decoding features, you create a PostgreSQL user with the `REPLICATION` attribute. When you are using the `pglogical` extension, the user must have the `cloudsqlsuperuser` role. To create a user with the `cloudsqlsuperuser` role, do the following:

   ```
   CREATE USER new_aws_dms_user WITH REPLICATION
   IN ROLE cloudsqlsuperuser LOGIN PASSWORD 'new_aws_dms_user_password';
   ```

   To set this attribue on an existing user, do the following:

   ```
   ALTER USER existing_user WITH REPLICATION;
   ```

1. Set the `max_replication_slots` parameter to the maximum number of DMS tasks that you plan to run concurrently. In Google Cloud SQL, the default value for this parameter is 10. This parameter's maximum value depends on the available memory of your PostgreSQL instance, allowing for between 2 and 8 replication slots per GB of memory.

For more information about logical replication with PostgreSQL, see the following topics:
+ [Enabling change data capture (CDC) using logical replication](CHAP_Source.PostgreSQL.md#CHAP_Source.PostgreSQL.Security)
+ [Using native CDC start points to set up a CDC load of a PostgreSQL source](CHAP_Source.PostgreSQL.md#CHAP_Source.PostgreSQL.v10)
+ [ Set up logical replication and decoding](https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication) in the [Cloud SQL for PostgreSQL documentation](https://cloud.google.com/sql/docs/postgres).

# Using a PostgreSQL database as an AWS DMS source
<a name="CHAP_Source.PostgreSQL"></a>

You can migrate data from one or many PostgreSQL databases using AWS DMS. With a PostgreSQL database as a source, you can migrate data to either another PostgreSQL database or one of the other supported databases. 

For information about versions of PostgreSQL that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

AWS DMS supports PostgreSQL for these types of databases: 
+ On-premises databases
+ Databases on an Amazon EC2 instance
+ Databases on an Amazon RDS DB instance
+ Databases on an DB instance based on Amazon Aurora PostgreSQL-Compatible Edition
+ Databases on an DB instance based on Amazon Aurora PostgreSQL-Compatible Serverless Edition

**Note**  
DMS supports Amazon Aurora PostgreSQL—Serverless V1 as a source for Full load only. But you can use Amazon Aurora PostgreSQL—Serverless V2 as a source for Full load, Full load \$1 CDC, and CDC only tasks.

You can use Secure Socket Layers (SSL) to encrypt connections between your PostgreSQL endpoint and the replication instance. For more information on using SSL with a PostgreSQL endpoint, see [Using SSL with AWS Database Migration Service](CHAP_Security.SSL.md).

As an additional security requirement when using PostgreSQL as a source, the user account specified must be a registered user in the PostgreSQL database.

To configure a PostgreSQL database as an AWS DMS source endpoint, do the following:
+ Create a PostgreSQL user with appropriate permissions to provide AWS DMS access to your PostgreSQL source database.
**Note**  
If your PostgreSQL source database is self-managed, see [Working with self-managed PostgreSQL databases as a source in AWS DMS](#CHAP_Source.PostgreSQL.Prerequisites) for more information.
If your PostgreSQL source database is managed by Amazon RDS, see [Working with AWS-managed PostgreSQL databases as a DMS source](#CHAP_Source.PostgreSQL.RDSPostgreSQL) for more information.
+ Create a PostgreSQL source endpoint that conforms with your chosen PostgreSQL database configuration.
+ Create a task or set of tasks to migrate your tables.

  To create a full-load-only task, no further endpoint configuration is needed.

  Before you create a task for change data capture (a CDC-only or full-load and CDC task), see [Enabling CDC using a self-managed PostgreSQL database as a AWS DMS source](#CHAP_Source.PostgreSQL.Prerequisites.CDC) or [Enabling CDC with an AWS-managed PostgreSQL DB instance with AWS DMS](#CHAP_Source.PostgreSQL.RDSPostgreSQL.CDC).

**Topics**
+ [

## Working with self-managed PostgreSQL databases as a source in AWS DMS
](#CHAP_Source.PostgreSQL.Prerequisites)
+ [

## Working with AWS-managed PostgreSQL databases as a DMS source
](#CHAP_Source.PostgreSQL.RDSPostgreSQL)
+ [

## Enabling change data capture (CDC) using logical replication
](#CHAP_Source.PostgreSQL.Security)
+ [

## Using native CDC start points to set up a CDC load of a PostgreSQL source
](#CHAP_Source.PostgreSQL.v10)
+ [

## Migrating from PostgreSQL to PostgreSQL using AWS DMS
](#CHAP_Source.PostgreSQL.Homogeneous)
+ [

## Migrating from Babelfish for Amazon Aurora PostgreSQL using AWS DMS
](#CHAP_Source.PostgreSQL.Babelfish)
+ [

## Removing AWS DMS artifacts from a PostgreSQL source database
](#CHAP_Source.PostgreSQL.CleanUp)
+ [

## Additional configuration settings when using a PostgreSQL database as a DMS source
](#CHAP_Source.PostgreSQL.Advanced)
+ [

## Read replica as a source for PostgreSQL
](#CHAP_Source.PostgreSQL.ReadReplica)
+ [

## Using the `MapBooleanAsBoolean` PostgreSQL endpoint setting
](#CHAP_Source.PostgreSQL.ConnectionAttrib.Endpointsetting)
+ [

## Endpoint settings and Extra Connection Attributes (ECAs) when using PostgreSQL as a DMS source
](#CHAP_Source.PostgreSQL.ConnectionAttrib)
+ [

## Limitations on using a PostgreSQL database as a DMS source
](#CHAP_Source.PostgreSQL.Limitations)
+ [

## Source data types for PostgreSQL
](#CHAP_Source-PostgreSQL-DataTypes)

## Working with self-managed PostgreSQL databases as a source in AWS DMS
<a name="CHAP_Source.PostgreSQL.Prerequisites"></a>

With a self-managed PostgreSQL database as a source, you can migrate data to either another PostgreSQL database, or one of the other target databases supported by AWS DMS. The database source can be an on-premises database or a self-managed engine running on an Amazon EC2 instance. You can use a DB instance for both full-load tasks and change data capture (CDC) tasks.

### Prerequisites to using a self-managed PostgreSQL database as an AWS DMS source
<a name="CHAP_Source.PostgreSQL.Prerequisites.SelfManaged"></a>

Before migrating data from a self-managed PostgreSQL source database, do the following: 
+ Make sure that you use a PostgreSQL database that is version 9.4.x or higher.
+ For full-load plus CDC tasks or CDC-only tasks, grant superuser permissions for the user account specified for the PostgreSQL source database. The user account needs superuser permissions to access replication-specific functions in the source. DMS user account needs SELECT permissions on all columns to migrate tables successfully. In the case of missing permissions on columns, DMS creates target table using regular DMS data type mappings which leads to metadata differences and task failures.
+ Add the IP address of the AWS DMS replication server to the `pg_hba.conf` configuration file and enable replication and socket connections. An example follows.

  ```
              # Replication Instance
              host all all 12.3.4.56/00 md5
              # Allow replication connections from localhost, by a user with the
              # replication privilege.
              host replication dms 12.3.4.56/00 md5
  ```

  PostgreSQL's `pg_hba.conf` configuration file controls client authentication. (HBA stands for host-based authentication.) The file is traditionally stored in the database cluster's data directory. 
+ If you're configuring a database as a source for logical replication using AWS DMS see [Enabling CDC using a self-managed PostgreSQL database as a AWS DMS source](#CHAP_Source.PostgreSQL.Prerequisites.CDC)

**Note**  
Some AWS DMS transactions are idle for some time before the DMS engine uses them again. By using the parameter `idle_in_transaction_session_timeout` in PostgreSQL versions 9.6 and higher, you can cause idle transactions to time out and fail. Don't end idle transactions when you use AWS DMS. 

### Enabling CDC using a self-managed PostgreSQL database as a AWS DMS source
<a name="CHAP_Source.PostgreSQL.Prerequisites.CDC"></a>

AWS DMS supports change data capture (CDC) using logical replication. To enable logical replication of a self-managed PostgreSQL source database, set the following parameters and values in the `postgresql.conf` configuration file:
+ Set `wal_level = logical`.
+ Set `max_replication_slots` to a value greater than 1.

  Set the `max_replication_slots` value according to the number of tasks that you want to run. For example, to run five tasks you set a minimum of five slots. Slots open automatically as soon as a task starts and remain open even when the task is no longer running. Make sure to manually delete open slots. Note that DMS automatically drops replication slots when the task is deleted, if DMS created the slot.
+ Set `max_wal_senders` to a value greater than 1.

  The `max_wal_senders` parameter sets the number of concurrent tasks that can run.
+ The `wal_sender_timeout` parameter ends replication connections that are inactive longer than the specified number of milliseconds. The default for an on-premises PostgreSQL database is 60000 milliseconds (60 seconds). Setting the value to 0 (zero) disables the timeout mechanism, and is a valid setting for DMS.

  When setting `wal_sender_timeout` to a non-zero value, a DMS task with CDC requires a minimum of 10000 milliseconds (10 seconds), and fails if the value is less than 10000. Keep the value less than 5 minutes to avoid causing a delay during a Multi-AZ failover of a DMS replication instance.

Some parameters are static, and you can only set them at server start. Any changes to their entries in the configuration file (for a self-managed database) or DB parameter group (for an RDS for PostgreSQL database) are ignored until the server is restarted. For more information, see the [PostgreSQL documentation](https://www.postgresql.org/docs/current/intro-whatis.html).

For more information about enabling CDC, see [Enabling change data capture (CDC) using logical replication](#CHAP_Source.PostgreSQL.Security).

## Working with AWS-managed PostgreSQL databases as a DMS source
<a name="CHAP_Source.PostgreSQL.RDSPostgreSQL"></a>

You can use an AWS-managed PostgreSQL DB instance as a source for AWS DMS. You can perform both full-load tasks and change data capture (CDC) tasks using an AWS-managed PostgreSQL source. 

### Prerequisites for using an AWS-managed PostgreSQL database as a DMS source
<a name="CHAP_Source.PostgreSQL.RDSPostgreSQL.Prerequisites"></a>

Before migrating data from an AWS-managed PostgreSQL source database, do the following:
+ We recommend that you use an AWS user account with the minimum required permissions for the PostgreSQL DB instance as the user account for the PostgreSQL source endpoint for AWS DMS. Using the master account is not recommended. The account must have the `rds_superuser` role and the `rds_replication` role. The `rds_replication` role grants permissions to manage logical slots and to stream data using logical slots.

  Make sure to create several objects from the master user account for the account that you use. For information about creating these, see [Migrating an Amazon RDS for PostgreSQL database without using the master user account](#CHAP_Source.PostgreSQL.RDSPostgreSQL.NonMasterUser).
+ If your source database is in a virtual private cloud (VPC), choose the VPC security group that provides access to the DB instance where the database resides. This is needed for the DMS replication instance to connect successfully to the source DB instance. When the database and DMS replication instance are in same VPC, add the appropriate security group to its own inbound rules.

**Note**  
Some AWS DMS transactions are idle for some time before the DMS engine uses them again. By using the parameter `idle_in_transaction_session_timeout` in PostgreSQL versions 9.6 and higher, you can cause idle transactions to time out and fail. Don't end idle transactions when you use AWS DMS.

### Enabling CDC with an AWS-managed PostgreSQL DB instance with AWS DMS
<a name="CHAP_Source.PostgreSQL.RDSPostgreSQL.CDC"></a>

AWS DMS supports CDC on Amazon RDS PostgreSQL databases when the DB instance is configured to use logical replication. The following table summarizes the logical replication compatibility of each AWS-managed PostgreSQL version. 


|  PostgreSQL version  |  AWS DMS full load support   |  AWS DMS CDC support  | 
| --- | --- | --- | 
|  Aurora PostgreSQL version 2.1 with PostgreSQL 10.5 compatibility (or lower)  |  Yes  |  No  | 
|  Aurora PostgreSQL version 2.2 with PostgreSQL 10.6 compatibility (or higher)   |  Yes  |  Yes  | 
|  RDS for PostgreSQL with PostgreSQL 10.21 compatibility (or higher)  |  Yes  |  Yes  | 

**To enable logical replication for an RDS PostgreSQL DB instance**

1. Use the AWS master user account for the PostgreSQL DB instance as the user account for the PostgreSQL source endpoint. The master user account has the required roles that allow it to set up CDC. 

   If you use an account other than the master user account, make sure to create several objects from the master account for the account that you use. For more information, see [Migrating an Amazon RDS for PostgreSQL database without using the master user account](#CHAP_Source.PostgreSQL.RDSPostgreSQL.NonMasterUser).

1. Set the `rds.logical_replication` parameter in your DB CLUSTER parameter group to 1. This static parameter requires a reboot of the DB instance to take effect. As part of applying this parameter, AWS DMS sets the `wal_level`, `max_wal_senders`, `max_replication_slots`, and `max_connections` parameters. These parameter changes can increase write ahead log (WAL) generation, so only set `rds.logical_replication` when you use logical replication slots.

1. The `wal_sender_timeout` parameter ends replication connections that are inactive longer than the specified number of milliseconds. The default for an AWS-managed PostgreSQL database is 30000 milliseconds (30 seconds). Setting the value to 0 (zero) disables the timeout mechanism, and is a valid setting for DMS.

   When setting `wal_sender_timeout` to a non-zero value, a DMS task with CDC requires a minimum of 10000 milliseconds (10 seconds), and fails if the value is between 0 and 10000. Keep the value less than 5 minutes to avoid causing a delay during a Multi-AZ failover of a DMS replication instance.

1.  Ensure the value of the `max_worker_processes` parameter in your DB Cluster Parameter Group is equal to, or higher than the total combined values of `max_logical_replication_workers`, `autovacuum_max_workers`, and `max_parallel_workers`. A high number of background worker processes might impact application workloads on small instances. So, monitor performance of your database if you set `max_worker_processes` higher than the default value.

1.  When using Aurora PostgreSQL as a source with CDC, set `synchronous_commit` to `ON`.

**To use PostgreSQL MultiAZ DB Cluster Read Replica for CDC (ongoing replication)**

1. Set the `rds.logical_replication` and `sync_replication_slots` parameters in your DB CLUSTER parameter group to 1. This static parameters require a reboot of the DB instance to take effect.

1. Run the following command to create the `awsdms_ddl_audit` table on Writer and to replace the `objects_schema` with the name of the schema to use:

   ```
   CREATE TABLE objects_schema.awsdms_ddl_audit
   (
     c_key    bigserial primary key,
     c_time   timestamp,    -- Informational
     c_user   varchar(64),  -- Informational: current_user
     c_txn    varchar(16),  -- Informational: current transaction
     c_tag    varchar(24),  -- Either 'CREATE TABLE' or 'ALTER TABLE' or 'DROP TABLE'
     c_oid    integer,      -- For future use - TG_OBJECTID
     c_name   varchar(64),  -- For future use - TG_OBJECTNAME
     c_schema varchar(64),  -- For future use - TG_SCHEMANAME. For now - holds current_schema
     c_ddlqry  text         -- The DDL query associated with the current DDL event
   );
   ```

1. Run the following command to create the `awsdms_intercept_ddl` function and to replace the `objects_schema` with the name of the schema to use:

   ```
   CREATE OR REPLACE FUNCTION objects_schema.awsdms_intercept_ddl()
     RETURNS event_trigger
   LANGUAGE plpgsql
   SECURITY DEFINER
     AS $$
     declare _qry text;
   BEGIN
     if (tg_tag='CREATE TABLE' or tg_tag='ALTER TABLE' or tg_tag='DROP TABLE' or tg_tag = 'CREATE TABLE AS') then
            SELECT current_query() into _qry;
            insert into objects_schema.awsdms_ddl_audit
            values
            (
            default,current_timestamp,current_user,cast(TXID_CURRENT()as varchar(16)),tg_tag,0,'',current_schema,_qry
            );
            delete from objects_schema.awsdms_ddl_audit;
   end if;
   END;
   $$;
   ```

1. Run the following command to create the `awsdms_intercept_ddl` event trigger:

   ```
   CREATE EVENT TRIGGER awsdms_intercept_ddl ON ddl_command_end EXECUTE PROCEDURE objects_schema.awsdms_intercept_ddl();
   ```

   Ensure that all the users and roles that access these events have the necessary DDL permissions. For example:

   ```
   grant all on public.awsdms_ddl_audit to public;
   grant all on public.awsdms_ddl_audit_c_key_seq to public;
   ```

1. Create replication slot on Writer:

   ```
   SELECT * FROM pg_create_logical_replication_slot('dms_read_replica_slot', 'test_decoding', false, true);
   ```

1. Ensure the replication slot is available on Reader:

   ```
   select * from pg_catalog.pg_replication_slots where slot_name = 'dms_read_replica_slot';
   
   slot_name            |plugin       |slot_type|datoid|database|temporary|active|active_pid|xmin|catalog_xmin|restart_lsn|confirmed_flush_lsn|wal_status|safe_wal_size|two_phase|inactive_since               |conflicting|invalidation_reason|failover|synced|
   ---------------------+-------------+---------+------+--------+---------+------+----------+----+------------+-----------+-------------------+----------+-------------+---------+-----------------------------+-----------+-------------------+--------+------+
   dms_read_replica_slot|test_decoding|logical  |     5|postgres|false    |false |          |    |3559        |0/180011B8 |0/180011F0         |reserved  |             |true     |2025-02-10 15:45:04.083 +0100|false      |                   |false   |false |
   ```

1. Create DMS source endpoint for Read Replica and set logical replication slot name via the Extra Connection Attribute:

   ```
   slotName=dms_read_replica_slot;
   ```

1. Create and start the CDC/FL\$1CDC task.
**Note**  
For CDC/FL\$1CDC migrations DMS considers task start time as a CDC start position. All older LSNs from replication slot are ignored.

### Migrating an Amazon RDS for PostgreSQL database without using the master user account
<a name="CHAP_Source.PostgreSQL.RDSPostgreSQL.NonMasterUser"></a>

In some cases, you might not use the master user account for the Amazon RDS PostgreSQL DB instance that you are using as a source. In these cases, you create several objects to capture data definition language (DDL) events. You create these objects in the account other than the master account and then create a trigger in the master user account.

**Note**  
If you set the `CaptureDdls` endpoint setting to `false` on the source endpoint, you don't have to create the following table and trigger on the source database.

Use the following procedure to create these objects.

**To create objects**

1. Choose the schema where the objects are to be created. The default schema is `public`. Ensure that the schema exists and is accessible by the `OtherThanMaster` account. 

1. Log in to the PostgreSQL DB instance using the user account other than the master account, here the `OtherThanMaster` account.

1. Create the table `awsdms_ddl_audit` by running the following command, replacing `objects_schema` in the following code with the name of the schema to use.

   ```
   CREATE TABLE objects_schema.awsdms_ddl_audit
   (
     c_key    bigserial primary key,
     c_time   timestamp,    -- Informational
     c_user   varchar(64),  -- Informational: current_user
     c_txn    varchar(16),  -- Informational: current transaction
     c_tag    varchar(24),  -- Either 'CREATE TABLE' or 'ALTER TABLE' or 'DROP TABLE'
     c_oid    integer,      -- For future use - TG_OBJECTID
     c_name   varchar(64),  -- For future use - TG_OBJECTNAME
     c_schema varchar(64),  -- For future use - TG_SCHEMANAME. For now - holds current_schema
     c_ddlqry  text         -- The DDL query associated with the current DDL event
   );
   ```

1. Create the function `awsdms_intercept_ddl` by running the following command, replacing `objects_schema` in the code following with the name of the schema to use.

   ```
   CREATE OR REPLACE FUNCTION objects_schema.awsdms_intercept_ddl()
     RETURNS event_trigger
   LANGUAGE plpgsql
   SECURITY DEFINER
     AS $$
     declare _qry text;
   BEGIN
     if (tg_tag='CREATE TABLE' or tg_tag='ALTER TABLE' or tg_tag='DROP TABLE' or tg_tag = 'CREATE TABLE AS') then
            SELECT current_query() into _qry;
            insert into objects_schema.awsdms_ddl_audit
            values
            (
            default,current_timestamp,current_user,cast(TXID_CURRENT()as varchar(16)),tg_tag,0,'',current_schema,_qry
            );
            delete from objects_schema.awsdms_ddl_audit;
   end if;
   END;
   $$;
   ```

1. Log out of the `OtherThanMaster` account and log in with an account that has the `rds_superuser` role assigned to it.

1. Create the event trigger `awsdms_intercept_ddl` by running the following command.

   ```
   CREATE EVENT TRIGGER awsdms_intercept_ddl ON ddl_command_end 
   EXECUTE PROCEDURE objects_schema.awsdms_intercept_ddl();
   ```

1. Make sure that all users and roles that access these events have the necessary DDL permissions. For example:

   ```
   grant all on public.awsdms_ddl_audit to public;
   grant all on public.awsdms_ddl_audit_c_key_seq to public;
   ```

When you have completed the procedure preceding, you can create the AWS DMS source endpoint using the `OtherThanMaster` account.

**Note**  
These events are triggered by `CREATE TABLE`, `ALTER TABLE`, and `DROP TABLE` statements.

## Enabling change data capture (CDC) using logical replication
<a name="CHAP_Source.PostgreSQL.Security"></a>

You can use PostgreSQL's native logical replication feature to enable change data capture (CDC) during database migration for PostgreSQL sources. You can use this feature with a self-managed PostgreSQL and also an Amazon RDS for PostgreSQL SQL DB instance. This approach reduces downtime and help ensure that the target database is in sync with the source PostgreSQL database.

AWS DMS supports CDC for PostgreSQL tables with primary keys. If a table does not have a primary key, the write-ahead logs (WAL) don't include a before image of the database row. In this case, DMS can't update the table. Here, you can use additional configuration settings and use table replica identity as a workaround. However, this approach can generate extra logs. We recommend that you use table replica identity as a workaround only after careful testing. For more information, see [Additional configuration settings when using a PostgreSQL database as a DMS source](#CHAP_Source.PostgreSQL.Advanced).

**Note**  
REPLICA IDENTITY FULL is supported with a logical decoding plugin, but isn't supported with a pglogical plugin. For more information, see [pglogical documentation](https://github.com/2ndQuadrant/pglogical#primary-key-or-replica-identity-required).

For full load and CDC and CDC only tasks, AWS DMS uses logical replication slots to retain WAL logs for replication until the logs are decoded. On restart (not resume) for a full load and CDC task or a CDC task, the replication slot gets recreated.

**Note**  
For logical decoding, DMS uses either test\$1decoding or pglogical plugin. If the pglogical plugin is available on a source PostgreSQL database, DMS creates a replication slot using pglogical, otherwise a test\$1decoding plugin is used. For more information about the test\$1decoding plugin, see [PostgreSQL Documentation](https://www.postgresql.org/docs/9.4/test-decoding.html).  
If the database parameter `max_slot_wal_keep_size` is set to a non default value, and the `restart_lsn` of a replication slot falls behind the current LSN by more than this size, the DMS task fails due to removal of required WAL files.

### Configuring the pglogical plugin
<a name="CHAP_Source.PostgreSQL.Security.Pglogical"></a>

Implemented as a PostgreSQL extension, the pglogical plugin is a logical replication system and model for selective data replication. The following table identifies source PostgreSQL database versions that support the pglogical plugin.


|  PostgreSQL source   |  Supports pglogical  | 
| --- | --- | 
|  Self-managed PostgreSQL 9.4 or higher  |  Yes  | 
|  Amazon RDS PostgreSQL 9.5 or lower  |  No  | 
|  Amazon RDS PostgreSQL 9.6 or higher  |  Yes  | 
|  Aurora PostgreSQL 1.x till 2.5.x  |  No  | 
|  Aurora PostgreSQL 2.6.x or higher  |  Yes  | 
|  Aurora PostgreSQL 3.3.x or higher  |  Yes  | 

Before configuring pglogical for use with AWS DMS, first enable logical replication for change data capture (CDC) on your PostgreSQL source database. 
+ For information about enabling logical replication for CDC on *self-managed* PostgreSQL source databases, see [Enabling CDC using a self-managed PostgreSQL database as a AWS DMS source](#CHAP_Source.PostgreSQL.Prerequisites.CDC)
+ For information about enabling logical replication for CDC on *AWS-managed* PostgreSQL source databases, see [Enabling CDC with an AWS-managed PostgreSQL DB instance with AWS DMS](#CHAP_Source.PostgreSQL.RDSPostgreSQL.CDC).

After logical replication is enabled on your PostgreSQL source database, use the following steps to configure pglogical for use with DMS.

**To use the pglogical plugin for logical replication on a PostgreSQL source database with AWS DMS**

1. Create a pglogical extension on your source PostgreSQL database:

   1. Set the correct parameter:
      + For self-managed PostgreSQL databases, set the database parameter `shared_preload_libraries= 'pglogical'`.
      + For PostgreSQL on Amazon RDS and Amazon Aurora PostgreSQL-Compatible Edition databases, set the parameter `shared_preload_libraries` to `pglogical` in the same RDS parameter group.

   1. Restart your PostgreSQL source database.

   1. On the PostgreSQL database, run the command, `create extension pglogical;`

1. Run the following command to verify that pglogical installed successfully:

   `select * FROM pg_catalog.pg_extension`

You can now create a AWS DMS task that performs change data capture for your PostgreSQL source database endpoint.

**Note**  
If you don't enable pglogical on your PostgreSQL source database, AWS DMS uses the `test_decoding` plugin by default. When pglogical is enabled for logical decoding, AWS DMS uses pglogical by default. But you can set the extra connection attribute, `PluginName` to use the `test_decoding` plugin instead.

## Using native CDC start points to set up a CDC load of a PostgreSQL source
<a name="CHAP_Source.PostgreSQL.v10"></a>

To enable native CDC start points with PostgreSQL as a source, set the `slotName` extra connection attribute to the name of an existing logical replication slot when you create the endpoint. This logical replication slot holds ongoing changes from the time of endpoint creation, so it supports replication from a previous point in time. 

PostgreSQL writes the database changes to WAL files that are discarded only after AWS DMS successfully reads changes from the logical replication slot. Using logical replication slots can protect logged changes from being deleted before they are consumed by the replication engine. 

However, depending on rate of change and consumption, changes being held in a logical replication slot can cause elevated disk usage. We recommend that you set space usage alarms in the source PostgreSQL instance when you use logical replication slots. For more information on setting the `slotName` extra connection attribute, see [Endpoint settings and Extra Connection Attributes (ECAs) when using PostgreSQL as a DMS source](#CHAP_Source.PostgreSQL.ConnectionAttrib).

The following procedure walks through this approach in more detail.

**To use a native CDC start point to set up a CDC load of a PostgreSQL source endpoint**

1. Identify the logical replication slot used by an earlier replication task (a parent task) that you want to use as a start point. Then query the `pg_replication_slots` view on your source database to make sure that this slot does not have any active connections. If it does, resolve and close them before proceeding.

   For the following steps, assume that your logical replication slot is `abc1d2efghijk_34567890_z0yx98w7_6v54_32ut_1srq_1a2b34c5d67ef`. 

1. Create a new source endpoint that includes the following extra connection attribute setting.

   ```
   slotName=abc1d2efghijk_34567890_z0yx98w7_6v54_32ut_1srq_1a2b34c5d67ef;
   ```

1. Create a new CDC-only task using the console, AWS CLI or AWS DMS API. For example, using the CLI you might run the following `create-replication-task` command. 

   ```
   aws dms create-replication-task --replication-task-identifier postgresql-slot-name-test 
   --source-endpoint-arn arn:aws:dms:us-west-2:012345678901:endpoint:ABCD1EFGHIJK2LMNOPQRST3UV4 
   --target-endpoint-arn arn:aws:dms:us-west-2:012345678901:endpoint:ZYX9WVUTSRQONM8LKJIHGF7ED6 
   --replication-instance-arn arn:aws:dms:us-west-2:012345678901:rep:AAAAAAAAAAA5BB4CCC3DDDD2EE 
   --migration-type cdc --table-mappings "file://mappings.json" --cdc-start-position "4AF/B00000D0" 
   --replication-task-settings "file://task-pg.json"
   ```

   In the preceding command, the following options are set:
   + The `source-endpoint-arn` option is set to the new value that you created in step 2.
   + The `replication-instance-arn` option is set to the same value as for the parent task from step 1.
   + The `table-mappings` and `replication-task-settings` options are set to the same values as for the parent task from step 1.
   + The `cdc-start-position` option is set to a start position value. To find this start position, either query the `pg_replication_slots` view on your source database or view the console details for the parent task in step 1. For more information, see [Determining a CDC native start point](CHAP_Task.CDC.md#CHAP_Task.CDC.StartPoint.Native).

   To enable custom CDC start mode when creating a new CDC-only task using the AWS DMS console, do the following:
   + In the **Task settings** section, for **CDC start mode for source transactions**, choose **Enable custom CDC start mode**.
   + For **Custom CDC start point for source transactions**, choose **Specify a log sequence number**. Specify the system change number or choose **Specify a recovery checkpoint**, and provide a Recovery checkpoint.

   When this CDC task runs, AWS DMS raises an error if the specified logical replication slot does not exist. It also raises an error if the task isn't created with a valid setting for `cdc-start-position`.

When using native CDC start points with the pglogical plugin and you want to use a new replication slot, complete the setup steps following before creating a CDC task. 

**To use a new replication slot not previously created as part of another DMS task**

1. Create a replication slot, as shown following:

   ```
   SELECT * FROM pg_create_logical_replication_slot('replication_slot_name', 'pglogical');
   ```

1. After the database creates the replication slot, get and note the **restart\$1lsn** and **confirmed\$1flush\$1lsn** values for the slot:

   ```
   select * from pg_replication_slots where slot_name like 'replication_slot_name';
   ```

   Note that the Native CDC Start position for a CDC task created after the replication slot can't be older than the **confirmed\$1flush\$1lsn** value.

   For information about the **restart\$1lsn** and **confirmed\$1flush\$1lsn** values, see [pg\$1replication\$1slots](https://www.postgresql.org/docs/14/view-pg-replication-slots.html) 

1. Create a pglogical node.

   ```
   SELECT pglogical.create_node(node_name := 'node_name', dsn := 'your_dsn_name');
   ```

1. Create two replication sets using the `pglogical.create_replication_set` function. The first replication set tracks updates and deletes for tables that have primary keys. The second replication set tracks only inserts, and has the same name as the first replication set, with the added prefix 'i'.

   ```
   SELECT pglogical.create_replication_set('replication_slot_name', false, true, true, false);
   SELECT pglogical.create_replication_set('ireplication_slot_name', true, false, false, true);
   ```

1. Add a table to the replication set.

   ```
   SELECT pglogical.replication_set_add_table('replication_slot_name', 'schemaname.tablename', true);
   SELECT pglogical.replication_set_add_table('ireplication_slot_name', 'schemaname.tablename', true);
   ```

1. Set the extra connection attribute (ECA) following when you create your source endpoint.

   ```
   PluginName=PGLOGICAL;slotName=slot_name;
   ```

You can now create a CDC only task with a PostgreSQL native start point using the new replication slot. For more information about the pglogical plugin, see the [pglogical 3.7 documentation](https://www.enterprisedb.com/docs/pgd/3.7/pglogical/)

## Migrating from PostgreSQL to PostgreSQL using AWS DMS
<a name="CHAP_Source.PostgreSQL.Homogeneous"></a>

When you migrate from a database engine other than PostgreSQL to a PostgreSQL database, AWS DMS is almost always the best migration tool to use. But when you are migrating from a PostgreSQL database to a PostgreSQL database, PostgreSQL tools can be more effective.

### Using PostgreSQL native tools to migrate data
<a name="CHAP_Source.PostgreSQL.Homogeneous.Native"></a>

We recommend that you use PostgreSQL database migration tools such as `pg_dump` under the following conditions: 
+ You have a homogeneous migration, where you are migrating from a source PostgreSQL database to a target PostgreSQL database. 
+ You are migrating an entire database.
+ The native tools allow you to migrate your data with minimal downtime. 

The pg\$1dump utility uses the COPY command to create a schema and data dump of a PostgreSQL database. The dump script generated by pg\$1dump loads data into a database with the same name and recreates the tables, indexes, and foreign keys. To restore the data to a database with a different name, use the `pg_restore` command and the `-d` parameter.

If you are migrating data from a PostgreSQL source database running on EC2 to an Amazon RDS for PostgreSQL target, you can use the pglogical plugin.

For more information about importing a PostgreSQL database into Amazon RDS for PostgreSQL or Amazon Aurora PostgreSQL-Compatible Edition, see [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Procedural.Importing.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Procedural.Importing.html).

### Using DMS to migrate data from PostgreSQL to PostgreSQL
<a name="CHAP_Source.PostgreSQL.Homogeneous.DMS"></a>

 AWS DMS can migrate data, for example, from a source PostgreSQL database that is on premises to a target Amazon RDS for PostgreSQL or Aurora PostgreSQL instance. Core or basic PostgreSQL data types most often migrate successfully.

**Note**  
When replicating partitioned tables from a PostgreSQL source to PostgreSQL target, you don’t need to mention the parent table as part of the selection criteria in the DMS task. Mentioning the parent table causes data to be duplicated in child tables on the target, possibly causing a PK violation. By selecting child tables alone in the table mapping selection criteria, the parent table is automatically populated.

Data types that are supported on the source database but aren't supported on the target might not migrate successfully. AWS DMS streams some data types as strings if the data type is unknown. Some data types, such as XML and JSON, can successfully migrate as small files but can fail if they are large documents. 

When performing data type migration, be aware of the following:
+ In some cases, the PostgreSQL NUMERIC(p,s) data type does not specify any precision and scale. For DMS versions 3.4.2 and earlier, DMS uses a precision of 28 and a scale of 6 by default, NUMERIC(28,6). For example, the value 0.611111104488373 from the source is converted to 0.611111 on the PostgreSQL target.
+ A table with an ARRAY data type must have a primary key. A table with an ARRAY data type missing a primary key gets suspended during full load.

The following table shows source PostgreSQL data types and whether they can be migrated successfully.


| Data type | Migrates successfully | Partially migrates | does not migrate | Comments | 
| --- | --- | --- | --- | --- | 
| INTEGER | X |  |  |  | 
| SMALLINT | X |  |  |  | 
| BIGINT | X |  |  |  | 
| NUMERIC/DECIMAL(p,s) |  | X |  | Where 0<p<39 and 0<s | 
| NUMERIC/DECIMAL |  | X |  | Where p>38 or p=s=0 | 
| REAL | X |  |  |  | 
| DOUBLE | X |  |  |  | 
| SMALLSERIAL | X |  |  |  | 
| SERIAL | X |  |  |  | 
| BIGSERIAL | X |  |  |  | 
| MONEY | X |  |  |  | 
| CHAR |  | X |  | Without specified precision | 
| CHAR(n) | X |  |  |  | 
| VARCHAR |  | X |  | Without specified precision | 
| VARCHAR(n) | X |  |  |  | 
| TEXT | X |  |  |  | 
| BYTEA | X |  |  |  | 
| TIMESTAMP | X |  |  | Positive and negative infinity values are truncated to '9999-12-31 23:59:59' and '4713-01-01 00:00:00 BC' respectively. | 
| TIMESTAMP WITH TIME ZONE |  | X |  |  | 
| DATE | X |  |  |  | 
| TIME | X |  |  |  | 
| TIME WITH TIME ZONE | X |  |  |  | 
| INTERVAL |  | X |  |  | 
| BOOLEAN | X |  |  |  | 
| ENUM |  |  | X |  | 
| CIDR | X |  |  |  | 
| INET |  |  | X |  | 
| MACADDR |  |  | X |  | 
| TSVECTOR |  |  | X |  | 
| TSQUERY |  |  | X |  | 
| XML |  | X |  |  | 
| POINT | X |  |  | PostGIS spatial data type | 
| LINE |  |  | X |  | 
| LSEG |  |  | X |  | 
| BOX |  |  | X |  | 
| PATH |  |  | X |  | 
| POLYGON | X |  |  | PostGIS spatial data type | 
| CIRCLE |  |  | X |  | 
| JSON |  | X |  |  | 
| ARRAY | X |  |  | Requires Primary Key | 
| COMPOSITE |  |  | X |  | 
| RANGE |  |  | X |  | 
| LINESTRING | X |  |  | PostGIS spatial data type | 
| MULTIPOINT | X |  |  | PostGIS spatial data type | 
| MULTILINESTRING | X |  |  | PostGIS spatial data type | 
| MULTIPOLYGON | X |  |  | PostGIS spatial data type | 
| GEOMETRYCOLLECTION | X |  |  | PostGIS spatial data type | 

### Migrating PostGIS spatial data types
<a name="CHAP_Source.PostgreSQL.DataTypes.Spatial"></a>

*Spatial data* identifies the geometry information of an object or location in space. PostgreSQL object-relational databases support PostGIS spatial data types. 

Before migrating PostgreSQL spatial data objects, ensure that the PostGIS plugin is enabled at the global level. Doing this ensures that AWS DMS creates the exact source spatial data columns for the PostgreSQL target DB instance.

For PostgreSQL to PostgreSQL homogeneous migrations, AWS DMS supports the migration of PostGIS geometric and geographic (geodetic coordinates) data object types and subtypes such as the following:
+  POINT 
+  LINESTRING 
+  POLYGON 
+  MULTIPOINT 
+  MULTILINESTRING 
+  MULTIPOLYGON 
+  GEOMETRYCOLLECTION 

## Migrating from Babelfish for Amazon Aurora PostgreSQL using AWS DMS
<a name="CHAP_Source.PostgreSQL.Babelfish"></a>

You can migrate Babelfish for Aurora PostgreSQL source tables to any supported target endpoints using AWS DMS.

When you create your AWS DMS source endpoint using the DMS console, API, or CLI commands, you set the source to **Amazon Aurora PostgreSQL**, and the database name to **babelfish\$1db**. In the **Endpoint Settings** section, make sure that the **DatabaseMode** is set to **Babelfish**, and **BabelfishDatabaseName** is set to the name of the source Babelfish T-SQL database. Instead of using the Babelfish TCP port **1433**, use the Aurora PostgreSQL TCP port **5432**.

You must create your tables before migrating data to make sure that DMS uses the correct data types and table metadata. If you don't create your tables on the target before running migration, DMS may create the tables with incorrect data types and permissions.

### Adding transformation rules to your migration task
<a name="CHAP_Source.PostgreSQL.Babelfish.Transform"></a>

When you create a migration task for a Babelfish source, you need to include transformation rules that ensure that DMS uses the pre-created target tables.

If you set multi-database migration mode when you defined your Babelfish for PostgreSQL cluster, add a transformation rule that renames the schema name to the T-SQL schema. For example, if the T-SQL schema name is `dbo`, and your Babelfish for PostgreSQL schema name is `mydb_dbo`, rename the schema to `dbo` using a transformation rule. To find the PostgreSQL schema name, see [Babelfish architecture](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/babelfish-architecture.html) in the *Amazon Aurora User Guide*. 

If you use single-database mode, you don't need to use a transformation rule to rename database schemas. PostgreSQL schema names have a one-to-one mapping to the schema names in the T-SQL database.

The following transformation rule example shows how to rename the schema name from `mydb_dbo` back to `dbo`:

```
{
    "rules": [
        {
            "rule-type": "transformation",
            "rule-id": "566251737",
            "rule-name": "566251737",
            "rule-target": "schema",
            "object-locator": {
                "schema-name": "mydb_dbo"
            },
            "rule-action": "rename",
            "value": "dbo",
            "old-value": null
        },
        {
            "rule-type": "selection",
            "rule-id": "566111704",
            "rule-name": "566111704",
            "object-locator": {
                "schema-name": "mydb_dbo",
                "table-name": "%"
            },
            "rule-action": "include",
            "filters": []
        }
    ]
}
```

### Limitations for using a PostgreSQL source endpoint with Babelfish tables
<a name="CHAP_Source.PostgreSQL.Babelfish.Limitations"></a>

The following limitations apply when using a PostgreSQL source endpoint with Babelfish tables:
+ DMS only supports migration from Babelfish version 16.2/15.6 and later, and DMS version 3.5.3 and later.
+ DMS does not replicate Babelfish table definition changes to the target endpoint. A workaround for this limitation is to first apply the table definition changes on the target, and then change the table definition on the Babelfish source.
+ When you create Babelfish tables with the BYTEA data type, DMS converts them to the `varbinary(max)` data type when migrating to SQL Server as a target.
+ DMS does not support Full LOB mode for binary data types. Use limited LOB mode for binary data types instead.
+ DMS does not support data validation for Babelfish as a source.
+ For the **Target table preparation mode** task setting, use only the **Do nothing** or **Truncate** modes. Don't use the **Drop tables on target** mode. When using **Drop tables on target**, DMS may create the tables with incorrect data types.
+ When using ongoing replication (CDC or Full load and CDC), set the `PluginName` extra connection attribute (ECA) to `TEST_DECODING`.
+ DMS does not support replication (CDC or Full load and CDC) of partitioned table for Babelfish as a source.

## Removing AWS DMS artifacts from a PostgreSQL source database
<a name="CHAP_Source.PostgreSQL.CleanUp"></a>

To capture DDL events, AWS DMS creates various artifacts in the PostgreSQL database when a migration task starts. When the task completes, you might want to remove these artifacts.

To remove the artifacts, issue the following statements (in the order they appear), where `{AmazonRDSMigration}` is the schema in which the artifacts were created. Dropping a schema should be done with extreme caution. Never drop an operational schema, especially not a public one.

```
drop event trigger awsdms_intercept_ddl;
```

The event trigger does not belong to a specific schema.

```
drop function {AmazonRDSMigration}.awsdms_intercept_ddl()
drop table {AmazonRDSMigration}.awsdms_ddl_audit
drop schema {AmazonRDSMigration}
```

## Additional configuration settings when using a PostgreSQL database as a DMS source
<a name="CHAP_Source.PostgreSQL.Advanced"></a>

You can add additional configuration settings when migrating data from a PostgreSQL database in two ways:
+ You can add values to the extra connection attribute to capture DDL events and to specify the schema in which the operational DDL database artifacts are created. For more information, see [Endpoint settings and Extra Connection Attributes (ECAs) when using PostgreSQL as a DMS source](#CHAP_Source.PostgreSQL.ConnectionAttrib).
+ You can override connection string parameters. Choose this option to do either of the following:
  + Specify internal AWS DMS parameters. Such parameters are rarely required so aren't exposed in the user interface.
  + Specify pass-through (passthru) values for the specific database client. AWS DMS includes pass-through parameters in the connection sting passed to the database client.
+ By using the table-level parameter `REPLICA IDENTITY` in PostgreSQL versions 9.4 and higher, you can control information written to write-ahead logs (WALs). In particular, it does so for WALs that identify rows that are updated or deleted. `REPLICA IDENTITY FULL` records the old values of all columns in the row. Use `REPLICA IDENTITY FULL` carefully for each table as `FULL` generates an extra number of WALs that might not be necessary. For more information, see [ALTER TABLE-REPLICA IDENTITY](https://www.postgresql.org/docs/devel/sql-altertable.html) 

## Read replica as a source for PostgreSQL
<a name="CHAP_Source.PostgreSQL.ReadReplica"></a>

Use PostgreSQL read replicas as CDC sources in AWS DMS to reduce primary database load. This feature is available from PostgreSQL 16.x and requires AWS DMS version 3.6.1 or later. Using read replicas for CDC processing reduces the operational impact on your primary database.

**Note**  
Amazon RDS PostgreSQL version 16.x has limitations for read replica logical replication in the Three-AZ (TAZ) configurations. For full read replica logical replication support in TAZ deployments, you must use PostgreSQL version 17.x or later.

### Prerequisites
<a name="CHAP_Source.PostgreSQL.ReadReplica.prereq"></a>

Before using a read replica as a CDC source for AWS DMS, you must enable logical replication on both the primary Database instance and its read replica to create logical decoding on a read replica. Perform the following actions:
+ Enable logical replication on both, your primary database instance and its read replica along with any other required database parameters. For more information, see [Working with AWS-managed PostgreSQL databases as a DMS source](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.RDSPostgreSQL).
+ For CDC-only tasks, create a replication slot on the primary (writer) instance. For more information, see [Using native CDC start points to set up a CDC load of a PostgreSQL source](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.v10). This action is necessary as read replicas do not support the creation of replication slot.
+ For PostgreSQL version 16, the slot must be manually created on the read replica.
+ For PostgreSQL version 17 and above, the replication slot must be created on the primary, and it is automatically synchronized to the read replica.
+ When using Full Load \$1 CDC or CDC-only tasks, AWS DMS can automatically manage logical replication slots on primary instances but not on read replicas. For PostgreSQL version 16 read replicas, you must manually drop and recreate replication slots before restarting a task (not resuming). Skipping this step can cause task failures or incorrect CDC starting positions. From PostgreSQL version 17 onwards, logical slot synchronization from the primary instance automates this process.

After completing the prerequisites, you can set up your AWS DMS source endpoint with replication `SlotName` of the read replica source in the endpoint settings and configure your AWS DMS task using native CDC start points. For more information see, [Endpoint settings and Extra Connection Attributes (ECAs) when using PostgreSQL as a DMS source](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.ConnectionAttrib) and [Using native CDC start points to set up a CDC load of a PostgreSQL source](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.v10).

## Using the `MapBooleanAsBoolean` PostgreSQL endpoint setting
<a name="CHAP_Source.PostgreSQL.ConnectionAttrib.Endpointsetting"></a>

You can use PostgreSQL endpoint settings to map a boolean as a boolean from your PostgreSQL source to a Amazon Redshift target. By default, a BOOLEAN type is migrated as varchar(5). You can specify `MapBooleanAsBoolean` to let PostgreSQL to migrate the boolean type as boolean as shown in the example following.

```
--postgre-sql-settings '{"MapBooleanAsBoolean": true}'
```

Note that you must set this setting on both the source and target endpoints for it to take effect.

Since MySQL does not have a BOOLEAN type, use a transformation rule rather than this setting when migrating BOOLEAN data to MySQL.

## Endpoint settings and Extra Connection Attributes (ECAs) when using PostgreSQL as a DMS source
<a name="CHAP_Source.PostgreSQL.ConnectionAttrib"></a>

You can use endpoint settings and extra connection attributes (ECAs) to configure your PostgreSQL source database. You specify endpoint settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--postgre-sql-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

The following table shows the endpoint settings and ECAs that you can use with PostgreSQL as a source.

[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html)

## Limitations on using a PostgreSQL database as a DMS source
<a name="CHAP_Source.PostgreSQL.Limitations"></a>

The following limitations apply when using PostgreSQL as a source for AWS DMS:
+ AWS DMS does not work with Amazon RDS for PostgreSQL 10.4 or Amazon Aurora PostgreSQL 10.4 either as source or target.
+ A captured table must have a primary key. If a table does not have a primary key, AWS DMS ignores DELETE and UPDATE record operations for that table. As a workaround, see [Enabling change data capture (CDC) using logical replication](#CHAP_Source.PostgreSQL.Security). 

  **Note:** We don't recommend migrating without a Primary Key/Unique Index, otherwise additional limitations apply such as "NO" Batch apply capability, Full LOB capability, Data Validation and inability to replicate to Redshift target efficiently.
+ AWS DMS ignores an attempt to update a primary key segment. In these cases, the target identifies the update as one that didn't update any rows. However, because the results of updating a primary key in PostgreSQL are unpredictable, no records are written to the exceptions table.
+ AWS DMS does not support the **Start Process Changes from Timestamp** run option.
+ AWS DMS does not replicate changes that result from partition or subpartition operations (`ADD`, `DROP`, or `TRUNCATE`).
+ Replication of multiple tables with the same name where each name has a different case (for example, table1, TABLE1, and Table1) can cause unpredictable behavior. Because of this issue, AWS DMS does not support this type of replication.
+ In most cases, AWS DMS supports change processing of CREATE, ALTER, and DROP DDL statements for tables. AWS DMS does not support this change processing if the tables are held in an inner function or procedure body block or in other nested constructs.

  For example, the following change isn't captured.

  ```
  CREATE OR REPLACE FUNCTION attu.create_distributors1() RETURNS void
  LANGUAGE plpgsql
  AS $$
  BEGIN
  create table attu.distributors1(did serial PRIMARY KEY,name
  varchar(40) NOT NULL);
  END;
  $$;
  ```
+ Currently, `boolean` data types in a PostgreSQL source are migrated to a SQL Server target as `bit` data type with inconsistent values. As a workaround, pre-create the table with a `VARCHAR(1)` data type for the column (or have AWS DMS create the table). Then have downstream processing treat an "F" as False and a "T" as True.
+ AWS DMS does not support change processing of TRUNCATE operations.
+ The OID LOB data type isn't migrated to the target.
+ AWS DMS supports the PostGIS data type for only homogeneous migrations.
+ If your source is a PostgreSQL database that is on-premises or on an Amazon EC2 instance, ensure that the test\$1decoding output plugin is installed on your source endpoint. You can find this plugin in the PostgreSQL contrib package. For more information about the test-decoding plugin, see the [PostgreSQL documentation](https://www.postgresql.org/docs/10/static/test-decoding.html).
+ AWS DMS does not support change processing to set and unset column default values (using the ALTER COLUMN SET DEFAULT clause on ALTER TABLE statements).
+ AWS DMS does not support change processing to set column nullability (using the ALTER COLUMN [SET\$1DROP] NOT NULL clause on ALTER TABLE statements).
+ When logical replication is enabled, the maximum number of changes kept in memory per transaction is 4 MB. After that, changes are spilled to disk. As a result `ReplicationSlotDiskUsage` increases, and `restart_lsn` doesn’t advance until the transaction is completed or stopped and the rollback finishes. Because it is a long transaction, it can take a long time to rollback. So, avoid long running transactions or many sub-transactions when logical replication is enabled. Instead, break the transaction into several smaller transactions. 

  On Aurora PostgreSQL versions 13 and later, you can tune the `logical_decoding_work_mem` parameter to control when DMS spills change data to disk. For more information, see [Spill files in Aurora PostgreSQL](CHAP_Troubleshooting_Latency_Source_PostgreSQL.md#CHAP_Troubleshooting_Latency_Source_PostgreSQL_Spill).
+ A table with an ARRAY data type must have a primary key. A table with an ARRAY data type missing a primary key gets suspended during full load.
+ AWS DMS does not support migrating table metadata related to table partitioning or [table inheritance](https://www.postgresql.org/docs/15/ddl-inherit.html). When AWS DMS encounters a partitioned table or a table that uses inheritance, the following behavior is observed:
  + AWS DMS identifies and reports both parent and child tables involved in partitioning or inheritance on the source database.
  + **Table Creation on Target**: On the target database, AWS DMS creates the table as a standard (non-partitioned, non-inherited) table, preserving the structure and properties of the selected table(s) but not the partitioning or inheritance logic.
  + **Record Differentiation in Inherited Tables**: For tables that use inheritance, AWS DMS does not distinguish records belonging to child tables when populating the parent table. As a result, it does not utilize SQL queries with syntax such as: `SELECT * FROM ONLY parent_table_name`.
+ To replicate partitioned tables from a PostgreSQL source to a PostgreSQL target, first manually create the parent and child tables on the target. Then define a separate task to replicate to those tables. In such a case, set the task configuration to **Truncate before loading**.
+ The PostgreSQL `NUMERIC` data type isn't fixed in size. When transferring data that is a `NUMERIC` data type but without precision and scale, DMS uses `NUMERIC(28,6)` (a precision of 28 and scale of 6) by default. As an example, the value 0.611111104488373 from the source is converted to 0.611111 on the PostgreSQL target.
+ AWS DMS supports Aurora PostgreSQL Serverless V1 as a source for full load tasks only. AWS DMS supports Aurora PostgreSQL Serverless V2 as a source for full load, full load and CDC, and CDC-only tasks.
+ AWS DMS does not support replication of a table with a unique index created with a coalesce function.
+ If primary key definition on source and target does not match, results of replication may be unpredictable.
+ When using the Parallel Load feature, table segmentation according to partitions or sub-partitions isn't supported. For more information about Parallel Load, see [Using parallel load for selected tables, views, and collections](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Tablesettings.md#CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Tablesettings.ParallelLoad) 
+ AWS DMS does not support Deferred Constraints.
+ AWS DMS version 3.4.7 supports PostgreSQL 14.x as a source with these limitations:
  + AWS DMS does not support change processing of two phase commits.
  + AWS DMS does not support logical replication to stream long in-progress transactions.
+ AWS DMS does not support CDC for Amazon RDS Proxy for PostgreSQL as a source.
+ When using [source filters](CHAP_Tasks.CustomizingTasks.Filters.md) that don't contain a Primary Key column, `DELETE` operations won't be captured.
+ If your source database is also a target for another third–party replication system, DDL changes might not migrate during CDC. Because that situation can prevent the `awsdms_intercept_ddl` event trigger from firing. To work around the situation, modify that trigger on your source database as follows:

  ```
  alter event trigger awsdms_intercept_ddl enable always;
  ```
+ AWS DMS does not support replication of changes made to primary key definitions in the source database. If the primary key structure is altered during an active replication task, subsequent changes to affected tables are not replicated to the target.
+ In DDL replication as part of a script, the maximum total number of DDL commands per script is 8192 and maximum total number of lines per script is 8192 lines.
+ AWS DMS does not support Materialized Views.
+ For full load and CDC tasks using a read replica as the source, AWS DMS cannot create replication slots on read replicas.

## Source data types for PostgreSQL
<a name="CHAP_Source-PostgreSQL-DataTypes"></a>

The following table shows the PostgreSQL source data types that are supported when using AWS DMS and the default mapping to AWS DMS data types.

For information on how to view the data type that is mapped in the target, see the section for the target endpoint you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  PostgreSQL data types  |  DMS data types  | 
| --- | --- | 
|  INTEGER  |  INT4  | 
|  SMALLINT  |  INT2  | 
|  BIGINT  |  INT8  | 
|  NUMERIC (p,s)  |  If precision is from 0 through 38, then use NUMERIC. If precision is 39 or greater, then use STRING.  | 
|  DECIMAL(P,S)  |  If precision is from 0 through 38, then use NUMERIC. If precision is 39 or greater, then use STRING.  | 
|  REAL  |  REAL4  | 
|  DOUBLE  |  REAL8  | 
|  SMALLSERIAL  |  INT2  | 
|  SERIAL  |  INT4  | 
|  BIGSERIAL  |  INT8  | 
|  MONEY  |  NUMERIC(38,4) The MONEY data type is mapped to FLOAT in SQL Server.  | 
|  CHAR  |  WSTRING (1)  | 
|  CHAR(N)  |  WSTRING (n)  | 
|  VARCHAR(N)  |  WSTRING (n)  | 
|  TEXT  |  NCLOB  | 
|  CITEXT  |  NCLOB  | 
|  BYTEA  |  BLOB  | 
|  TIMESTAMP  |  DATETIME  | 
|  TIMESTAMP WITH TIME ZONE  |  DATETIME  | 
|  DATE  |  DATE  | 
|  TIME  |  TIME  | 
|  TIME WITH TIME ZONE  |  TIME  | 
|  INTERVAL  |  STRING (128)—1 YEAR, 2 MONTHS, 3 DAYS, 4 HOURS, 5 MINUTES, 6 SECONDS  | 
|  BOOLEAN  |  CHAR (5) false or true  | 
|  ENUM  |  STRING (64)  | 
|  CIDR  |  STRING (50)  | 
|  INET  |  STRING (50)  | 
|  MACADDR  |  STRING (18)  | 
|  BIT (n)  |  STRING (n)  | 
|  BIT VARYING (n)  |  STRING (n)  | 
|  UUID  |  STRING  | 
|  TSVECTOR  |  CLOB  | 
|  TSQUERY  |  CLOB  | 
|  XML  |  CLOB  | 
|  POINT  |  STRING (255) "(x,y)"  | 
|  LINE  |  STRING (255) "(x,y,z)"  | 
|  LSEG  |  STRING (255) "((x1,y1),(x2,y2))"  | 
|  BOX  |  STRING (255) "((x1,y1),(x2,y2))"  | 
|  PATH  |  CLOB "((x1,y1),(xn,yn))"  | 
|  POLYGON  |  CLOB "((x1,y1),(xn,yn))"  | 
|  CIRCLE  |  STRING (255) "(x,y),r"  | 
|  JSON  |  NCLOB  | 
|  JSONB  |  NCLOB  | 
|  ARRAY  |  NCLOB  | 
|  COMPOSITE  |  NCLOB  | 
|  HSTORE  |  NCLOB  | 
|  INT4RANGE  |  STRING (255)  | 
|  INT8RANGE  |  STRING (255)  | 
|  NUMRANGE  |  STRING (255)  | 
|  STRRANGE  |  STRING (255)  | 

### Working with LOB source data types for PostgreSQL
<a name="CHAP_Source-PostgreSQL-DataTypes-LOBs"></a>

PostgreSQL column sizes affect the conversion of PostgreSQL LOB data types to AWS DMS data types. To work with this, take the following steps for the following AWS DMS data types:
+ BLOB – Set **Limit LOB size to** the **Maximum LOB size (KB)** value at task creation.
+ CLOB – Replication handles each character as a UTF8 character. Therefore, find the length of the longest character text in the column, shown here as `max_num_chars_text`. Use this length to specify the value for **Limit LOB size to**. If the data includes 4-byte characters, multiply by 2 to specify the **Limit LOB size to** value, which is in bytes. In this case, **Limit LOB size to** is equal to `max_num_chars_text` multiplied by 2.
+ NCLOB – Replication handles each character as a double-byte character. Therefore, find the length of the longest character text in the column (`max_num_chars_text`) and multiply by 2. You do this to specify the value for **Limit LOB size to**. In this case, **Limit LOB size to** is equal to `max_num_chars_text` multiplied by 2. If the data includes 4-byte characters, multiply by 2 again. In this case, **Limit LOB size to** is equal to `max_num_chars_text` multiplied by 4.

# Using a MySQL-compatible database as a source for AWS DMS
<a name="CHAP_Source.MySQL"></a>

You can migrate data from any MySQL-compatible database (MySQL, MariaDB, or Amazon Aurora MySQL) using AWS Database Migration Service. 

For information about versions of MySQL that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

You can use SSL to encrypt connections between your MySQL-compatible endpoint and the replication instance. For more information on using SSL with a MySQL-compatible endpoint, see [Using SSL with AWS Database Migration Service](CHAP_Security.SSL.md).

In the following sections, the term "self-managed" applies to any database that is installed either on-premises or on Amazon EC2. The term "AWS-managed" applies to any database on Amazon RDS, Amazon Aurora, or Amazon S3.

For additional details on working with MySQL-compatible databases and AWS DMS, see the following sections.

**Topics**
+ [

## Migrating from MySQL to MySQL using AWS DMS
](#CHAP_Source.MySQL.Homogeneous)
+ [

## Using any MySQL-compatible database as a source for AWS DMS
](#CHAP_Source.MySQL.Prerequisites)
+ [

## Using a self-managed MySQL-compatible database as a source for AWS DMS
](#CHAP_Source.MySQL.CustomerManaged)
+ [

## Using an AWS-managed MySQL-compatible database as a source for AWS DMS
](#CHAP_Source.MySQL.AmazonManaged)
+ [

## Limitations on using a MySQL database as a source for AWS DMS
](#CHAP_Source.MySQL.Limitations)
+ [

## Support for XA transactions
](#CHAP_Source.MySQL.XA)
+ [

## Endpoint settings when using MySQL as a source for AWS DMS
](#CHAP_Source.MySQL.ConnectionAttrib)
+ [

## Source data types for MySQL
](#CHAP_Source.MySQL.DataTypes)

**Note**  
When configuring AWS Database Migration Service (AWS DMS) mapping rules, it is important to avoid using wildcards (%) for database or schema names. Instead, you must explicitly specify only the user-created databases that need to be migrated. Using a wildcard character includes all databases in the migration process, including system databases which are not required on the target instance. Since the MySQL Amazon RDS master user lacks the necessary permissions to import data into target system databases, attempting to migrate these system databases fails.

## Migrating from MySQL to MySQL using AWS DMS
<a name="CHAP_Source.MySQL.Homogeneous"></a>

For a heterogeneous migration, where you are migrating from a database engine other than MySQL to a MySQL database, AWS DMS is almost always the best migration tool to use. But for a homogeneous migration, where you are migrating from a MySQL database to a MySQL database, we recommend that you use a homogeneous data migrations migration project. homogeneous data migrations uses native database tools to provide an improved data migration performance and accuracy when compared to AWS DMS.

## Using any MySQL-compatible database as a source for AWS DMS
<a name="CHAP_Source.MySQL.Prerequisites"></a>

Before you begin to work with a MySQL database as a source for AWS DMS, make sure that you have the following prerequisites. These prerequisites apply to either self-managed or AWS-managed sources.

You must have an account for AWS DMS that has the Replication Admin role. The role needs the following privileges:
+ **REPLICATION CLIENT** – This privilege is required for CDC tasks only. In other words, full-load-only tasks don't require this privilege. 
**Note**  
For MariaDB version 10.5.2\$1, you can use BINLOG MONITOR – it is a replacement for REPLICATION CLIENT.
+ **REPLICATION SLAVE** – This privilege is required for CDC tasks only. In other words, full-load-only tasks don't require this privilege.
+ **SUPER** – This privilege is required only in MySQL versions before 5.6.6.

The AWS DMS user must also have SELECT privileges for the source tables designated for replication.

Grant the following privileges if you use MySQL-specific premigration assessments:

```
grant select on mysql.user to <dms_user>;
grant select on mysql.db to <dms_user>;
grant select on mysql.tables_priv to <dms_user>;
grant select on mysql.role_edges to <dms_user>  #only for MySQL version 8.0.11 and higher
grant select on performance_schema.replication_connection_status to <dms_user>;  #Required for primary instance validation - MySQL version 5.7 and higher only
```

If you're using an RDS source and plan to run MySQL-specific premigration assessments, add the following permission:

```
grant select on mysql.rds_configuration to <dms_user>;  #Required for binary log retention check
```

If parameter `BatchEnable` is `true` it is required to grant:

```
grant create temporary tables on `<schema>`.* to <dms_user>;
```

## Using a self-managed MySQL-compatible database as a source for AWS DMS
<a name="CHAP_Source.MySQL.CustomerManaged"></a>

You can use the following self-managed MySQL-compatible databases as sources for AWS DMS:
+ MySQL Community Edition
+ MySQL Standard Edition
+ MySQL Enterprise Edition
+ MySQL Cluster Carrier Grade Edition
+ MariaDB Community Edition
+ MariaDB Enterprise Edition
+ MariaDB Column Store

To use CDC, make sure to enable binary logging. To enable binary logging, the following parameters must be configured in MySQL's `my.ini` (Windows) or `my.cnf` (UNIX) file.


| Parameter | Value | 
| --- | --- | 
| `server_id` | Set this parameter to a value of 1 or greater. | 
| `log-bin` | Set the path to the binary log file, such as `log-bin=E:\MySql_Logs\BinLog`. Don't include the file extension. | 
| `binlog_format` | Set this parameter to `ROW`. We recommend this setting during replication because in certain cases when `binlog_format` is set to `STATEMENT`, it can cause inconsistency when replicating data to the target. The database engine also writes similar inconsistent data to the target when `binlog_format` is set to `MIXED`, because the database engine automatically switches to `STATEMENT`-based logging which can result in writing inconsistent data on the target database. | 
| `expire_logs_days` | Set this parameter to a value of 1 or greater. To prevent overuse of disk space, we recommend that you don't use the default value of 0. | 
| `binlog_checksum` | Set this parameter to `NONE` for DMS version 3.4.7 or prior. | 
| `binlog_row_image` | Set this parameter to `FULL`. | 
| `log_slave_updates` | Set this parameter to `TRUE` if you are using a MySQL or MariaDB read-replica as a source. | 

If you are using a MySQL or MariaDB read-replica as a source for a DMS migration task using **Migrate existing data and replicate ongoing changes** mode, there is a possibility of data loss. DMS won't write a transaction during either full load or CDC under the following conditions:
+ The transaction had been committed to the primary instance before the DMS task started.
+ The transaction hadn't been committed to the replica until after the DMS task started, due to lag between the primary instance and the replica.

The longer the lag between the primary instance and the replica, the greater potential there is for data loss.

If your source uses the NDB (clustered) database engine, the following parameters must be configured to enable CDC on tables that use that storage engine. Add these changes in MySQL's `my.ini` (Windows) or `my.cnf` (UNIX) file.


| Parameter | Value | 
| --- | --- | 
| `ndb_log_bin` | Set this parameter to `ON`. This value ensures that changes in clustered tables are logged to the binary log. | 
| `ndb_log_update_as_write` | Set this parameter to `OFF`. This value prevents writing UPDATE statements as INSERT statements in the binary log. | 
| `ndb_log_updated_only` | Set this parameter to `OFF`. This value ensures that the binary log contains the entire row and not just the changed columns. | 

## Using an AWS-managed MySQL-compatible database as a source for AWS DMS
<a name="CHAP_Source.MySQL.AmazonManaged"></a>

You can use the following AWS-managed MySQL-compatible databases as sources for AWS DMS:
+ MySQL Community Edition
+ MariaDB Community Edition
+ Amazon Aurora MySQL-Compatible Edition

When using an AWS-managed MySQL-compatible database as a source for AWS DMS, make sure that you have the following prerequisites for CDC:
+ To enable binary logs for RDS for MySQL and for RDS for MariaDB, enable automatic backups at the instance level. To enable binary logs for an Aurora MySQL cluster, change the variable `binlog_format` in the parameter group.

  For more information about setting up automatic backups, see [Working with automated backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html) in the *Amazon RDS User Guide*.

  For more information about setting up binary logging for an Amazon RDS for MySQL database, see [ Setting the binary logging format](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQL.BinaryFormat.html) in the *Amazon RDS User Guide*. 

  For more information about setting up binary logging for an Aurora MySQL cluster, see [ How do I turn on binary logging for my Amazon Aurora MySQL cluster?](https://aws.amazon.com/premiumsupport/knowledge-center/enable-binary-logging-aurora/). 
+ If you plan to use CDC, turn on binary logging. For more information on setting up binary logging for an Amazon RDS for MySQL database, see [Setting the binary logging format](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.MySQL.BinaryFormat.html) in the *Amazon RDS User Guide*.
+ Ensure that the binary logs are available to AWS DMS. Because AWS-managed MySQL-compatible databases purge the binary logs as soon as possible, you should increase the length of time that the logs remain available. For example, to increase log retention to 24 hours, run the following command. 

  ```
   call mysql.rds_set_configuration('binlog retention hours', 24);
  ```
+ Set the `binlog_format` parameter to `"ROW"`.
**Note**  
On MySQL or MariaDB, `binlog_format` is a dynamic parameter, so you don't have to reboot to make the new value take effect. However, the new value will only apply to new sessions. If you switch the `binlog_format` to `ROW` for replication purposes, your database can still create subsequent binary logs using the `MIXED` format, if those sessions started before you changed the value. This may prevent AWS DMS from properly capturing all changes on the source database. When you change the `binlog_format` setting on a MariaDB or MySQL database, be sure to restart the database to close all existing sessions, or restart any application performing DML (Data Manipulation Language) operations. Forcing your database to restart all sessions after changing the `binlog_format` parameter to `ROW` will ensure that your database writes all subsequent source database changes using the correct format, so that AWS DMS can properly capture those changes.
+ Set the `binlog_row_image` parameter to `"Full"`. 
+ Set the `binlog_checksum` parameter to `"NONE"` for DMS version 3.4.7 or prior. For more information about setting parameters in Amazon RDS MySQL, see [Working with automated backups](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html) in the *Amazon RDS User Guide*.
+ If you are using an Amazon RDS MySQL or Amazon RDS MariaDB read replica as a source, enable backups on the read replica, and ensure the `log_slave_updates` parameter is set to `TRUE`.

## Limitations on using a MySQL database as a source for AWS DMS
<a name="CHAP_Source.MySQL.Limitations"></a>

When using a MySQL database as a source, consider the following:
+  Change data capture (CDC) isn't supported for Amazon RDS MySQL 5.5 or lower. For Amazon RDS MySQL, you must use version 5.6, 5.7, or 8.0 to enable CDC. CDC is supported for self-managed MySQL 5.5 sources. 
+ For CDC, `CREATE TABLE`, `ADD COLUMN`, and `DROP COLUMN` changing the column data type, and `renaming a column` are supported. However, `DROP TABLE`, `RENAME TABLE`, and updates made to other attributes, such as column default value, column nullability, character set and so on, are not supported.
+  For partitioned tables on the source, when you set **Target table preparation mode** to **Drop tables on target**, AWS DMS creates a simple table without any partitions on the MySQL target. To migrate partitioned tables to a partitioned table on the target, precreate the partitioned tables on the target MySQL database.
+  Using an `ALTER TABLE table_name ADD COLUMN column_name` statement to add columns to the beginning (FIRST) or the middle of a table (AFTER) isn't supported for relational targets. Columns are always added to the end of the table. When the target is Amazon S3 or Amazon Kinesis Data Streams, adding columns using FIRST or AFTER is supported.
+ CDC isn't supported when a table name contains uppercase and lowercase characters, and the source engine is hosted on an operating system with case-insensitive file names. An example is Microsoft Windows or OS X using HFS\$1.
+ You can use Aurora MySQL-Compatible Edition Serverless v1 for full load, but you can't use it for CDC. This is because you can't enable the prerequisites for MySQL. For more information, see [ Parameter groups and Aurora Serverless v1](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.how-it-works.html#aurora-serverless.parameter-groups). 

  Aurora MySQL-Compatible Edition Serverless v2 supports CDC.
+  The AUTO\$1INCREMENT attribute on a column isn't migrated to a target database column.
+  Capturing changes when the binary logs aren't stored on standard block storage isn't supported. For example, CDC does not work when the binary logs are stored on Amazon S3.
+  AWS DMS creates target tables with the InnoDB storage engine by default. If you need to use a storage engine other than InnoDB, you must manually create the table and migrate to it using [do nothing](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_GettingStarted.html) mode.
+ You can't use Aurora MySQL replicas as a source for AWS DMS unless your DMS migration task mode is **Migrate existing data**—full load only.
+  If the MySQL-compatible source is stopped during full load, the AWS DMS task does not stop with an error. The task ends successfully, but the target might be out of sync with the source. If this happens, either restart the task or reload the affected tables.
+  Indexes created on a portion of a column value aren't migrated. For example, the index CREATE INDEX first\$1ten\$1chars ON customer (name(10)) isn't created on the target.
+ In some cases, the task is configured to not replicate LOBs ("SupportLobs" is false in task settings or **Don't include LOB columns** is chosen in the task console). In these cases, AWS DMS does not migrate any MEDIUMBLOB, LONGBLOB, MEDIUMTEXT, and LONGTEXT columns to the target.

  BLOB, TINYBLOB, TEXT, and TINYTEXT columns aren't affected and are migrated to the target.
+ Temporal data tables or system—versioned tables are not supported on MariaDB source and target databases.
+ If migrating between two Amazon RDS Aurora MySQL clusters, the RDS Aurora MySQL source endpoint must be a read/write instance, not a replica instance. 
+ AWS DMS currently does not support views migration for MariaDB.
+ AWS DMS does not support DDL changes for partitioned tables for MySQL. To skip table suspension for partition DDL changes during CDC, set `skipTableSuspensionForPartitionDdl` to `true`.
+ AWS DMS only supports XA transactions in version 3.5.0 and higher. Previous versions do not support XA transactions. AWS DMS does not support XA transactions in MariaDB version 10.6 or higher For more information, see [Support for XA transactions](#CHAP_Source.MySQL.XA) following.
+ AWS DMS does not use GTIDs for replication, even if the source data contains them. 
+ AWS DMS does not support Aurora MySQL enhanced binary log.
+ AWS DMS does not support binary log transaction compression.
+ AWS DMS does not propagate ON DELETE CASCADE and ON UPDATE CASCADE events for MySQL databases using the InnoDB storage engine. For these events, MySQL does not generate binlog events to reflect the cascaded operations on the child tables. Consequently, AWS DMS can't replicate the corresponding changes to the child tables. For more information, see [Indexes, Foreign Keys, or Cascade Updates or Deletes Not Migrated](CHAP_Troubleshooting.md#CHAP_Troubleshooting.MySQL.FKsAndIndexes).
+ AWS DMS does not capture changes to computed (`VIRTUAL` and `GENERATED ALWAYS`) columns. To work around this limitation, do the following:
  + Pre-create the target table in the target database, and create the AWS DMS task with the `DO_NOTHING` or `TRUNCATE_BEFORE_LOAD` full-load task setting.
  + Add a transformation rule to remove the computed column from the task scope. For information about transformation rules, see [Transformation rules and actions](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.md).
+ Due to internal MySQL limitation, AWS DMS can process BINLOGs no larger than 4GB size. BINLOGs larger than 4GB may result in DMS task failures or other unpredictable behavior. You must reduce the size of transactions to avoid BINLOGs larger than 4GB.
+ AWS DMS does not support back-ticks (```) or single quotes (`'`) in schema, table, and column names.
+ AWS DMS does not migrate data from invisible columns in your source database. To include these columns in your migration scope, use the ALTER TABLE statement to make these columns visible.

## Support for XA transactions
<a name="CHAP_Source.MySQL.XA"></a>

An Extended Architecture (XA) transaction is a transaction that can be used to group a series of operations from multiple transactional resources into a single, reliable global transaction. An XA transaction uses a two-phase commit protocol. In general, capturing changes while there are open XA transactions might lead to loss of data. If your database does not use XA transactions, you can ignore this permission and the configuration `IgnoreOpenXaTransactionsCheck` by using the deafult value `TRUE`. To start replicating from a source that has XA transactions, do the following:
+ Ensure that the AWS DMS endpoint user has the following permission:

  ```
  grant XA_RECOVER_ADMIN on *.* to 'userName'@'%';
  ```
+ Set the endpoint setting `IgnoreOpenXaTransactionsCheck` to `false`.

**Note**  
AWS DMS doesn’t support XA transactions on MariaDB Source DB version 10.6 or higher.

## Endpoint settings when using MySQL as a source for AWS DMS
<a name="CHAP_Source.MySQL.ConnectionAttrib"></a>

You can use endpoint settings to configure your MySQL source database similar to using extra connection attributes. You specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--my-sql-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

The following table shows the endpoint settings that you can use with MySQL as a source.

[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.html)

## Source data types for MySQL
<a name="CHAP_Source.MySQL.DataTypes"></a>

The following table shows the MySQL database source data types that are supported when using AWS DMS and the default mapping from AWS DMS data types.

For information on how to view the data type that is mapped in the target, see the section for the target endpoint you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  MySQL data types  |  AWS DMS data types  | 
| --- | --- | 
|  INT  |  INT4  | 
|  BIGINT  |  INT8  | 
|  MEDIUMINT  |  INT4  | 
|  TINYINT  |  INT1  | 
|  SMALLINT  |  INT2  | 
|  UNSIGNED TINYINT  |  UINT1  | 
|  UNSIGNED SMALLINT  |  UINT2  | 
|  UNSIGNED MEDIUMINT  |  UINT4  | 
|  UNSIGNED INT  |  UINT4  | 
|  UNSIGNED BIGINT  |  UINT8  | 
|  DECIMAL(10)  |  NUMERIC (10,0)  | 
|  BINARY  |  BYTES(1)  | 
|  BIT  |  BOOLEAN  | 
|  BIT(64)  |  BYTES(8)  | 
|  BLOB  |  BYTES(65535)  | 
|  LONGBLOB  |  BLOB  | 
|  MEDIUMBLOB  |  BLOB  | 
|  TINYBLOB  |  BYTES(255)  | 
|  DATE  |  DATE  | 
|  DATETIME  |  DATETIME DATETIME without a parenthetical value is replicated without milliseconds. DATETIME with a parenthetical value of 1 to 5 (such as `DATETIME(5)`) is replicated with milliseconds. When replicating a DATETIME column, the time remains the same on the target. It is not converted to UTC.  | 
|  TIME  |  STRING  | 
|  TIMESTAMP  |  DATETIME When replicating a TIMESTAMP column, the time is converted to UTC on the target.  | 
|  YEAR  |  INT2  | 
|  DOUBLE  |  REAL8  | 
|  FLOAT  |  REAL(DOUBLE) If the FLOAT values are not in the range following, use a transformation to map FLOAT to STRING. For more information about transformations, see [Transformation rules and actions](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.md). The supported FLOAT range is -1.79E\$1308 to -2.23E-308, 0, and 2.23E-308 to 1.79E\$1308  | 
|  VARCHAR (45)  |  WSTRING (45)  | 
|  VARCHAR (2000)  |  WSTRING (2000)  | 
|  VARCHAR (4000)  |  WSTRING (4000)  | 
|  VARBINARY (4000)  |  BYTES (4000)  | 
|  VARBINARY (2000)  |  BYTES (2000)  | 
|  CHAR  |  WSTRING  | 
|  TEXT  |  WSTRING  | 
|  LONGTEXT  |  NCLOB  | 
|  MEDIUMTEXT  |  NCLOB  | 
|  TINYTEXT  |  WSTRING(255)  | 
|  GEOMETRY  |  BLOB  | 
|  POINT  |  BLOB  | 
|  LINESTRING  |  BLOB  | 
|  POLYGON  |  BLOB  | 
|  MULTIPOINT  |  BLOB  | 
|  MULTILINESTRING  |  BLOB  | 
|  MULTIPOLYGON  |  BLOB  | 
|  GEOMETRYCOLLECTION  |  BLOB  | 
|  ENUM  |  WSTRING (*length*) Here, *length* is the length of the longest value in the ENUM.  | 
|  SET  |  WSTRING (*length*) Here, *length* is the total length of all values in the SET, including commas.  | 
|  JSON  |  CLOB  | 

**Note**  
In some cases, you might specify the DATETIME and TIMESTAMP data types with a "zero" value (that is, 0000-00-00). If so, make sure that the target database in the replication task supports "zero" values for the DATETIME and TIMESTAMP data types. Otherwise, these values are recorded as null on the target.

# Using an SAP ASE database as a source for AWS DMS
<a name="CHAP_Source.SAP"></a>

You can migrate data from an SAP Adaptive Server Enterprise (ASE) database—formerly known as Sybase—using AWS DMS. With an SAP ASE database as a source, you can migrate data to any of the other supported AWS DMS target databases. 

For information about versions of SAP ASE that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

For additional details on working with SAP ASE databases and AWS DMS, see the following sections.

**Topics**
+ [

## Prerequisites for using an SAP ASE database as a source for AWS DMS
](#CHAP_Source.SAP.Prerequisites)
+ [

## Limitations on using SAP ASE as a source for AWS DMS
](#CHAP_Source.SAP.Limitations)
+ [

## Permissions required for using SAP ASE as a source for AWS DMS
](#CHAP_Source.SAP.Security)
+ [

## Removing the truncation point
](#CHAP_Source.SAP.Truncation)
+ [

## Endpoint settings when using SAP ASE as a source for AWS DMS
](#CHAP_Source.SAP.ConnectionAttrib)
+ [

## Source data types for SAP ASE
](#CHAP_Source.SAP.DataTypes)

## Prerequisites for using an SAP ASE database as a source for AWS DMS
<a name="CHAP_Source.SAP.Prerequisites"></a>

For an SAP ASE database to be a source for AWS DMS, do the following:
+ Enable SAP ASE replication for tables by using the `sp_setreptable` command. For more information, see [Sybase Infocenter Archive]( http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.dc32410_1501/html/refman/X37830.htm). 
+ Disable `RepAgent` on the SAP ASE database. For more information, see [Stop and disable the RepAgent thread in the primary database](http://infocenter-archive.sybase.com/help/index.jsp?topic=/com.sybase.dc20096_1260/html/mra126ag/mra126ag65.htm). 
+ To replicate to SAP ASE version 15.7 on an Windows EC2 instance configured for non-Latin characters (for example, Chinese), install SAP ASE 15.7 SP121 on the target computer.

**Note**  
For ongoing change data capture (CDC) replication, DMS runs `dbcc logtransfer` and `dbcc log` to read data from the transaction log.

## Limitations on using SAP ASE as a source for AWS DMS
<a name="CHAP_Source.SAP.Limitations"></a>

The following limitations apply when using an SAP ASE database as a source for AWS DMS:
+ You can run only one AWS DMS task with ongoing replication or CDC for each SAP ASE database. You can run multiple full-load-only tasks in parallel.
+ You can't rename a table. For example, the following command fails.

  ```
  sp_rename 'Sales.SalesRegion', 'SalesReg;
  ```
+ You can't rename a column. For example, the following command fails.

  ```
  sp_rename 'Sales.Sales.Region', 'RegID', 'COLUMN';
  ```
+ Zero values located at the end of binary data type strings are truncated when replicated to the target database. For example, `0x0000000000000000000000000100000100000000` in the source table becomes `0x00000000000000000000000001000001` in the target table.
+ If the database default is set not to allow NULL values, AWS DMS creates the target table with columns that don't allow NULL values. Consequently, if a full load or CDC replication task contains empty values, AWS DMS throws an error. You can prevent these errors by allowing NULL values in the source database by using the following commands.

  ```
  sp_dboption database_name, 'allow nulls by default', 'true'
  go
  use database_name
  CHECKPOINT
  go
  ```
+ The `reorg rebuild` index command isn't supported.
+ AWS DMS does not support clusters or using MSA (Multi-Site Availability)/Warm Standby as a source.
+ When `AR_H_TIMESTAMP` transformation header expression is used in mapping rules, the milliseconds won't be captured for an added column.
+ Running Merge operations during CDC will result in a non-recoverable error. To bring the target back in sync, run a full load.
+ Rollback trigger events are not supported for tables that use a data row locking scheme.
+ AWS DMS can't resume a replication task after dropping a table within the task scope from a source SAP database. If the DMS replication task was stopped and performed any DML operation (INSERT,UPDATE,DELETE) followed by dropping the table, you must restart the replication task.

## Permissions required for using SAP ASE as a source for AWS DMS
<a name="CHAP_Source.SAP.Security"></a>

To use an SAP ASE database as a source in an AWS DMS task, you need to grant permissions. Grant the user account specified in the AWS DMS database definitions the following permissions in the SAP ASE database: 
+ sa\$1role
+ replication\$1role
+ sybase\$1ts\$1role
+ By default, where you need to have permission to run the `sp_setreptable` stored procedure, AWS DMS enables the SAP ASE replication option. If you want to run `sp_setreptable` on a table directly from the database endpoint and not through AWS DMS itself, you can use the `enableReplication` extra connection attribute. For more information, see [Endpoint settings when using SAP ASE as a source for AWS DMS](#CHAP_Source.SAP.ConnectionAttrib).

## Removing the truncation point
<a name="CHAP_Source.SAP.Truncation"></a>

When a task starts, AWS DMS establishes a `$replication_truncation_point` entry in the `syslogshold` system view, indicating that a replication process is in progress. While AWS DMS is working, it advances the replication truncation point at regular intervals, according to the amount of data that has already been copied to the target.

After the `$replication_truncation_point` entry is established, keep the AWS DMS task running to prevent the database log from becoming excessively large. If you want to stop the AWS DMS task permanently, remove the replication truncation point by issuing the following command:

```
dbcc settrunc('ltm','ignore')
```

After the truncation point is removed, you can't resume the AWS DMS task. The log continues to be truncated automatically at the checkpoints (if automatic truncation is set).

## Endpoint settings when using SAP ASE as a source for AWS DMS
<a name="CHAP_Source.SAP.ConnectionAttrib"></a>

You can use endpoint settings to configure your SAP ASE source database similar to using extra connection attributes. You specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--sybase-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

The following table shows the endpoint settings that you can use with SAP ASE as a source.

[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SAP.html)

## Source data types for SAP ASE
<a name="CHAP_Source.SAP.DataTypes"></a>

For a list of the SAP ASE source data types that are supported when using AWS DMS and the default mapping from AWS DMS data types, see the following table. AWS DMS doesn't support SAP ASE source tables with columns of the user-defined type (UDT) data type. Replicated columns with this data type are created as NULL. 

For information on how to view the data type that is mapped in the target, see the [Targets for data migration](CHAP_Target.md) section for your target endpoint.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  SAP ASE data types  |  AWS DMS data types  | 
| --- | --- | 
| BIGINT | INT8 | 
| UNSIGNED BIGINT | UINT8 | 
| INT | INT4 | 
| UNSIGNED INT | UINT4 | 
| SMALLINT | INT2 | 
| UNSIGNED SMALLINT | UINT2 | 
| TINYINT | UINT1 | 
| DECIMAL | NUMERIC | 
| NUMERIC | NUMERIC | 
| FLOAT | REAL8 | 
| DOUBLE | REAL8 | 
| REAL | REAL4 | 
| MONEY | NUMERIC | 
| SMALLMONEY | NUMERIC | 
| DATETIME | DATETIME | 
| BIGDATETIME | DATETIME(6) | 
| SMALLDATETIME | DATETIME | 
| DATE | DATE | 
| TIME | TIME | 
| BIGTIME | TIME | 
| CHAR | STRING | 
| UNICHAR | WSTRING | 
| NCHAR | WSTRING | 
| VARCHAR | STRING | 
| UNIVARCHAR | WSTRING | 
| NVARCHAR | WSTRING | 
| BINARY | BYTES | 
| VARBINARY | BYTES | 
| BIT | BOOLEAN | 
| TEXT | CLOB | 
| UNITEXT | NCLOB | 
| IMAGE | BLOB | 

# Using MongoDB as a source for AWS DMS
<a name="CHAP_Source.MongoDB"></a>

 For information about versions of MongoDB that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

Note the following about MongoDB version support:
+ Versions of AWS DMS 3.4.5 and later support MongoDB versions 4.2 and 4.4. 
+ Versions of AWS DMS 3.4.5 and later and versions of MongoDB 4.2 and later support distributed transactions. For more information on MongoDB distributed transactions, see [Transactions](https://docs.mongodb.com/manual/core/transactions/) in the [MongoDB documentation](https://www.mongodb.com/docs/).
+ Versions of AWS DMS 3.5.0 and later don't support versions of MongoDB prior to 3.6.
+ Versions of AWS DMS 3.5.1 and later support MongoDB version 5.0.
+ Versions of AWS DMS 3.5.2 and later support MongoDB version 6.0.
+ Versions of AWS DMS 3.5.4 and later support MongoDB version 7.0 and 8.0.



If you are new to MongoDB, be aware of the following important MongoDB database concepts: 
+ A record in MongoDB is a *document*, which is a data structure composed of field and value pairs. The value of a field can include other documents, arrays, and arrays of documents. A document is roughly equivalent to a row in a relational database table.
+ A *collection* in MongoDB is a group of documents, and is roughly equivalent to a relational database table.
+ A *database* in MongoDB is a set of collections, and is roughly equivalent to a schema in a relational database.
+ Internally, a MongoDB document is stored as a binary JSON (BSON) file in a compressed format that includes a type for each field in the document. Each document has a unique ID.

AWS DMS supports two migration modes when using MongoDB as a source, *Document mode* or *Table mode*. You specify which migration mode to use when you create the MongoDB endpoint or by setting the **Metadata mode** parameter from the AWS DMS console. Optionally, you can create a second column named `_id` that acts as the primary key by selecting the check mark button for **\$1id as a separate column** in the endpoint configuration panel. 

Your choice of migration mode affects the resulting format of the target data, as explained following. 

**Document mode**  
In document mode, the MongoDB document is migrated as is, meaning that the document data is consolidated into a single column named `_doc` in a target table. Document mode is the default setting when you use MongoDB as a source endpoint.  
For example, consider the following documents in a MongoDB collection called myCollection.  

```
 db.myCollection.find()
{ "_id" : ObjectId("5a94815f40bd44d1b02bdfe0"), "a" : 1, "b" : 2, "c" : 3 }
{ "_id" : ObjectId("5a94815f40bd44d1b02bdfe1"), "a" : 4, "b" : 5, "c" : 6 }
```
After migrating the data to a relational database table using document mode, the data is structured as follows. The data fields in the MongoDB document are consolidated into the` _doc` column.      
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html)
You can optionally set the extra connection attribute `extractDocID` to *true* to create a second column named `"_id"` that acts as the primary key. If you are going to use CDC, set this parameter to *true*.  
When using CDC with sources that produce [multi-document transactions](https://www.mongodb.com/docs/manual/reference/method/Session.startTransaction/#mongodb-method-Session.startTransaction), the `ExtractDocId` parameter **must be** set to *true*. If this parameter is not enabled, the AWS DMS task will fail when it encounters a multi-document transaction.  
In document mode, AWS DMS manages the creation and renaming of collections like this:  
+ If you add a new collection to the source database, AWS DMS creates a new target table for the collection and replicates any documents. 
+ If you rename an existing collection on the source database, AWS DMS doesn't rename the target table. 
If the target endpoint is Amazon DocumentDB, run the migration in **Document mode**.

**Table mode**  
In table mode, AWS DMS transforms each top-level field in a MongoDB document into a column in the target table. If a field is nested, AWS DMS flattens the nested values into a single column. AWS DMS then adds a key field and data types to the target table's column set.   
For each MongoDB document, AWS DMS adds each key and type to the target table's column set. For example, using table mode, AWS DMS migrates the previous example into the following table.      
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html)
Nested values are flattened into a column containing dot-separated key names. The column is named the concatenation of the flattened field names separated by periods. For example, AWS DMS migrates a JSON document with a field of nested values such as `{"a" : {"b" : {"c": 1}}}` into a column named `a.b.c.`  
To create the target columns, AWS DMS scans a specified number of MongoDB documents and creates a set of all the fields and their types. AWS DMS then uses this set to create the columns of the target table. If you create or modify your MongoDB source endpoint using the console, you can specify the number of documents to scan. The default value is 1000 documents. If you use the AWS CLI, you can use the extra connection attribute `docsToInvestigate`.  
In table mode, AWS DMS manages documents and collections like this:  
+ When you add a document to an existing collection, the document is replicated. If there are fields that don't exist in the target, those fields aren't replicated.
+ When you update a document, the updated document is replicated. If there are fields that don't exist in the target, those fields aren't replicated.
+ Deleting a document is fully supported.
+ Adding a new collection doesn't result in a new table on the target when done during a CDC task.
+ In the Change Data Capture(CDC) phase, AWS DMS doesn't support renaming a collection.

**Topics**
+ [

## Permissions needed when using MongoDB as a source for AWS DMS
](#CHAP_Source.MongoDB.PrerequisitesCDC)
+ [

## Configuring a MongoDB replica set for CDC
](#CHAP_Source.MongoDB.PrerequisitesCDC.ReplicaSet)
+ [

## Security requirements when using MongoDB as a source for AWS DMS
](#CHAP_Source.MongoDB.Security)
+ [

## Segmenting MongoDB collections and migrating in parallel
](#CHAP_Source.MongoDB.ParallelLoad)
+ [

## Migrating multiple databases when using MongoDB as a source for AWS DMS
](#CHAP_Source.MongoDB.Multidatabase)
+ [

## Limitations when using MongoDB as a source for AWS DMS
](#CHAP_Source.MongoDB.Limitations)
+ [

## Endpoint configuration settings when using MongoDB as a source for AWS DMS
](#CHAP_Source.MongoDB.Configuration)
+ [

## Source data types for MongoDB
](#CHAP_Source.MongoDB.DataTypes)

## Permissions needed when using MongoDB as a source for AWS DMS
<a name="CHAP_Source.MongoDB.PrerequisitesCDC"></a>

For an AWS DMS migration with a MongoDB source, you can create either a user account with root privileges, or a user with permissions only on the database to migrate. 

The following code creates a user to be the root account.

```
use admin
db.createUser(
  {
    user: "root",
    pwd: "password",
    roles: [ { role: "root", db: "admin" } ]
  }
)
```

For a MongoDB 3.x source, the following code creates a user with minimal privileges on the database to be migrated.

```
use database_to_migrate
db.createUser( 
{ 
    user: "dms-user",
    pwd: "password",
    roles: [ { role: "read", db: "local" }, "read"] 
})
```

For a MongoDB 4.x source, the following code creates a user with minimal privileges.

```
{ resource: { db: "", collection: "" }, actions: [ "find", "changeStream" ] }
```

For example, create the following role in the "admin" database.

```
use admin
db.createRole(
{
role: "changestreamrole",
privileges: [
{ resource: { db: "", collection: "" }, actions: [ "find","changeStream" ] }
],
roles: []
}
)
```

And once the role is created, create a user in the database to be migrated.

```
 use test
> db.createUser( 
{ 
user: "dms-user12345",
pwd: "password",
roles: [ { role: "changestreamrole", db: "admin" }, "read"] 
})
```

## Configuring a MongoDB replica set for CDC
<a name="CHAP_Source.MongoDB.PrerequisitesCDC.ReplicaSet"></a>

To use ongoing replication or CDC with MongoDB, AWS DMS requires access to the MongoDB operations log (oplog). To create the oplog, you need to deploy a replica set if one doesn't exist. For more information, see [ the MongoDB documentation](https://docs.mongodb.com/manual/tutorial/deploy-replica-set/).

You can use CDC with either the primary or secondary node of a MongoDB replica set as the source endpoint.

**To convert a standalone instance to a replica set**

1. Using the command line, connect to `mongo`.

   ```
   mongo localhost
   ```

1. Stop the `mongod` service.

   ```
   service mongod stop
   ```

1. Restart `mongod` using the following command:

   ```
   mongod --replSet "rs0" --auth -port port_number
   ```

1. Test the connection to the replica set using the following commands:

   ```
   mongo -u root -p password --host rs0/localhost:port_number 
     --authenticationDatabase "admin"
   ```

If you plan to perform a document mode migration, select option `_id as a separate column` when you create the MongoDB endpoint. Selecting this option creates a second column named `_id` that acts as the primary key. This second column is required by AWS DMS to support data manipulation language (DML) operations.

**Note**  
AWS DMS uses the operations log (oplog) to capture changes during ongoing replication. If MongoDB flushes out the records from the oplog before AWS DMS reads them, your tasks fail. We recommend sizing the oplog to retain changes for at least 24 hours. 

## Security requirements when using MongoDB as a source for AWS DMS
<a name="CHAP_Source.MongoDB.Security"></a>

AWS DMS supports two authentication methods for MongoDB. The two authentication methods are used to encrypt the password, so they are only used when the `authType` parameter is set to *PASSWORD*.

The MongoDB authentication methods are as follows:
+ **MONGODB-CR** – For backward compatibility
+ **SCRAM-SHA-1** – The default when using MongoDB version 3.x and 4.0

If an authentication method isn't specified, AWS DMS uses the default method for the version of the MongoDB source.

## Segmenting MongoDB collections and migrating in parallel
<a name="CHAP_Source.MongoDB.ParallelLoad"></a>

To improve performance of a migration task, MongoDB source endpoints support two options for parallel full load in table mapping. 

In other words, you can migrate a collection in parallel by using either autosegmentation or range segmentation with table mapping for a parallel full load in JSON settings. With autosegmentation, you can specify the criteria for AWS DMS to automatically segment your source for migration in each thread. With range segmentation, you can tell AWS DMS the specific range of each segment for DMS to migrate in each thread. For more information on these settings, see [Table and collection settings rules and operations](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Tablesettings.md).

### Migrating a MongoDB database in parallel using autosegmentation ranges
<a name="CHAP_Source.MongoDB.ParallelLoad.AutoPartitioned"></a>

You can migrate your documents in parallel by specifying the criteria for AWS DMS to automatically partition (segment) your data for each thread. In particular, you specify the number of documents to migrate per thread. Using this approach, AWS DMS attempts to optimize segment boundaries for maximum performance per thread.

You can specify the segmentation criteria using the table-settings options following in table mapping.


|  Table-settings option  |  Description  | 
| --- | --- | 
|  `"type"`  |  (Required) Set to `"partitions-auto"` for MongoDB as a source.  | 
|  `"number-of-partitions"`  |  (Optional) Total number of partitions (segments) used for migration. The default is 16.  | 
|  `"collection-count-from-metadata"`  |  (Optional) If this option is set to `true`, AWS DMS uses an estimated collection count for determining the number of partitions. If this option is set to `false`, AWS DMS uses the actual collection count. The default is `true`.  | 
|  `"max-records-skip-per-page"`  |  (Optional) The number of records to skip at once when determining the boundaries for each partition. AWS DMS uses a paginated skip approach to determine the minimum boundary for a partition. The default is 10,000.  Setting a relatively large value can result in cursor timeouts and task failures. Setting a relatively low value results in more operations per page and a slower full load.   | 
|  `"batch-size"`  |  (Optional) Limits the number of documents returned in one batch. Each batch requires a round trip to the server. If the batch size is zero (0), the cursor uses the server-defined maximum batch size. The default is 0.  | 

The example following shows a table mapping for autosegmentation.

```
{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "admin",
                "table-name": "departments"
            },
            "rule-action": "include",
            "filters": []
        },
        {
            "rule-type": "table-settings",
            "rule-id": "2",
            "rule-name": "2",
            "object-locator": {
                "schema-name": "admin",
                "table-name": "departments"
            },
            "parallel-load": {
                "type": "partitions-auto",
                "number-of-partitions": 5,
                "collection-count-from-metadata": "true",
                "max-records-skip-per-page": 1000000,
                "batch-size": 50000
            }
        }
    ]
}
```

Autosegmentation has the limitation following. The migration for each segment fetches the collection count and the minimum `_id` for the collection separately. It then uses a paginated skip to calculate the minimum boundary for that segment. 

Therefore, ensure that the minimum `_id` value for each collection remains constant until all the segment boundaries in the collection are calculated. If you change the minimum `_id` value for a collection during its segment boundary calculation, it can cause data loss or duplicate row errors.

### Migrating a MongoDB database in parallel using range segmentation
<a name="CHAP_Source.MongoDB.ParallelLoad.Ranges"></a>

You can migrate your documents in parallel by specifying the ranges for each segment in a thread. Using this approach, you tell AWS DMS the specific documents to migrate in each thread according to your choice of document ranges per thread.

The image following shows a MongoDB collection that has seven items, and `_id` as the primary key.

![\[MongoDB collection with seven items.\]](http://docs.aws.amazon.com/dms/latest/userguide/images/datarep-docdb-collection.png)


To split the collection into three specific segments for AWS DMS to migrate in parallel, you can add table mapping rules to your migration task. This approach is shown in the following JSON example.

```
{ // Task table mappings:
  "rules": [
    {
      "rule-type": "selection",
      "rule-id": "1",
      "rule-name": "1",
      "object-locator": {
        "schema-name": "testdatabase",
        "table-name": "testtable"
      },
      "rule-action": "include"
    }, // "selection" :"rule-type"
    {
      "rule-type": "table-settings",
      "rule-id": "2",
      "rule-name": "2",
      "object-locator": {
        "schema-name": "testdatabase",
        "table-name": "testtable"
      },
      "parallel-load": {
        "type": "ranges",
        "columns": [
           "_id",
           "num"
        ],
        "boundaries": [
          // First segment selects documents with _id less-than-or-equal-to 5f805c97873173399a278d79
          // and num less-than-or-equal-to 2.
          [
             "5f805c97873173399a278d79",
             "2"
          ],
          // Second segment selects documents with _id > 5f805c97873173399a278d79 and
          // _id less-than-or-equal-to 5f805cc5873173399a278d7c and
          // num > 2 and num less-than-or-equal-to 5.
          [
             "5f805cc5873173399a278d7c",
             "5"
          ]                                   
          // Third segment is implied and selects documents with _id > 5f805cc5873173399a278d7c.
        ] // :"boundaries"
      } // :"parallel-load"
    } // "table-settings" :"rule-type"
  ] // :"rules"
} // :Task table mappings
```

That table mapping definition splits the source collection into three segments and migrates in parallel. The following are the segmentation boundaries.

```
Data with _id less-than-or-equal-to "5f805c97873173399a278d79" and num less-than-or-equal-to 2 (2 records)
Data with _id > "5f805c97873173399a278d79" and num > 2 and _id  less-than-or-equal-to "5f805cc5873173399a278d7c" and num less-than-or-equal-to 5 (3 records)
Data with _id > "5f805cc5873173399a278d7c" and num > 5 (2 records)
```

After the migration task is complete, you can verify from the task logs that the tables loaded in parallel, as shown in the following example. You can also verify the MongoDB `find` clause used to unload each segment from the source table.

```
[TASK_MANAGER    ] I:  Start loading segment #1 of 3 of table 'testdatabase'.'testtable' (Id = 1) by subtask 1. Start load timestamp 0005B191D638FE86  (replicationtask_util.c:752)

[SOURCE_UNLOAD   ] I:  Range Segmentation filter for Segment #0 is initialized.   (mongodb_unload.c:157)

[SOURCE_UNLOAD   ] I:  Range Segmentation filter for Segment #0 is: { "_id" : { "$lte" : { "$oid" : "5f805c97873173399a278d79" } }, "num" : { "$lte" : { "$numberInt" : "2" } } }  (mongodb_unload.c:328)

[SOURCE_UNLOAD   ] I: Unload finished for segment #1 of segmented table 'testdatabase'.'testtable' (Id = 1). 2 rows sent.

[TASK_MANAGER    ] I: Start loading segment #1 of 3 of table 'testdatabase'.'testtable' (Id = 1) by subtask 1. Start load timestamp 0005B191D638FE86 (replicationtask_util.c:752)
 
[SOURCE_UNLOAD   ] I: Range Segmentation filter for Segment #0 is initialized. (mongodb_unload.c:157) 

[SOURCE_UNLOAD   ] I: Range Segmentation filter for Segment #0 is: { "_id" : { "$lte" : { "$oid" : "5f805c97873173399a278d79" } }, "num" : { "$lte" : { "$numberInt" : "2" } } } (mongodb_unload.c:328)
 
[SOURCE_UNLOAD   ] I: Unload finished for segment #1 of segmented table 'testdatabase'.'testtable' (Id = 1). 2 rows sent.

[TARGET_LOAD     ] I: Load finished for segment #1 of segmented table 'testdatabase'.'testtable' (Id = 1). 1 rows received. 0 rows skipped. Volume transfered 480.

[TASK_MANAGER    ] I: Load finished for segment #1 of table 'testdatabase'.'testtable' (Id = 1) by subtask 1. 2 records transferred.
```

Currently, AWS DMS supports the following MongoDB data types as a segment key column:
+ Double
+ String
+ ObjectId
+ 32 bit integer
+ 64 bit integer

## Migrating multiple databases when using MongoDB as a source for AWS DMS
<a name="CHAP_Source.MongoDB.Multidatabase"></a>

AWS DMS versions 3.4.5 and higher support migrating multiple databases in a single task for all supported MongoDB versions. If you want to migrate multiple databases, take these steps:

1. When you create the MongoDB source endpoint, do one of the following:
   + On the DMS console's **Create endpoint** page, make sure that **Database name** is empty under **Endpoint configuration**.
   + Using the AWS CLI `CreateEndpoint` command, assign an empty string value to the `DatabaseName` parameter in `MongoDBSettings`.

1. For each database that you want to migrate from a MongoDB source, specify the database name as a schema name in the table mapping for the task. You can do this using either the guided input in the console or directly in JSON. For more information on the guided input, see [Specifying table selection and transformations rules from the console](CHAP_Tasks.CustomizingTasks.TableMapping.Console.md). For more information on the JSON, see [Selection rules and actions](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Selections.md).

For example, you might specify the JSON following to migrate three MongoDB databases.

**Example Migrate all tables in a schema**  
The JSON following migrates all tables from the `Customers`, `Orders`, and `Suppliers` databases in your source endpoint to your target endpoint.  

```
{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "Customers",
                "table-name": "%"
            },
            "rule-action": "include",
            "filters": []
        },
        {
            "rule-type": "selection",
            "rule-id": "2",
            "rule-name": "2",
            "object-locator": {
                "schema-name": "Orders",
                "table-name": "%"
            },
            "rule-action": "include",
            "filters": []
        },
        {
            "rule-type": "selection",
            "rule-id": "3",
            "rule-name": "3",
            "object-locator": {
                "schema-name": "Inventory",
                "table-name": "%"
            },
            "rule-action": "include",
            "filters": []
        }
    ]
}
```

## Limitations when using MongoDB as a source for AWS DMS
<a name="CHAP_Source.MongoDB.Limitations"></a>

The following are limitations when using MongoDB as a source for AWS DMS:
+ In table mode, the documents in a collection must be consistent in the data type that they use for the value in the same field. For example, if a document in a collection includes `'{ a:{ b:value ... }'`, all documents in the collection that reference the `value` of the `a.b` field must use the same data type for `value`, wherever it appears in the collection.
+ When the `_id` option is set as a separate column, the ID string can't exceed 200 characters.
+ Object ID and array type keys are converted to columns that are prefixed with `oid` and `array` in table mode.

  Internally, these columns are referenced with the prefixed names. If you use transformation rules in AWS DMS that reference these columns, make sure to specify the prefixed column. For example, you specify `${oid__id}` and not `${_id}`, or `${array__addresses}` and not `${_addresses}`. 
+  Collection names and key names can't include the dollar symbol (\$1). 
+ AWS DMS doesn't support collections containing the same field with different case (upper, lower) in table mode with a RDBMS target. For example, AWS DMS does not support having two collections named `Field1` and `field1`. 
+ Table mode and document mode have the limitations described preceding.
+ Migrating in parallel using autosegmentation has the limitations described preceding.
+ Source filters aren't supported for MongoDB.
+ AWS DMS doesn't support documents where the nesting level is greater than 97.
+ AWS DMS requires UTF-8 encoded source data when migrating to non-DocumentDB targets. For sources with non-UTF-8 characters, convert them to UTF-8 before migration or migrate to Amazon DocumentDB instead.
+ AWS DMS doesn't support the following features of MongoDB version 5.0:
  + Live resharding
  + Client-Side Field Level Encryption (CSFLE)
  + Timeseries collection migration
**Note**  
A timeseries collection migrated in the full-load phase will be converted to a normal collection in Amazon DocumentDB, because DocumentDB doesn't support timeseries collections.

## Endpoint configuration settings when using MongoDB as a source for AWS DMS
<a name="CHAP_Source.MongoDB.Configuration"></a>

When you set up your MongoDB source endpoint, you can specify multiple endpoint configuration settings using the AWS DMS console. 

The following table describes the configuration settings available when using MongoDB databases as an AWS DMS source. 


| Setting (attribute) | Valid values | Default value and description | 
| --- | --- | --- | 
|  **Authentication mode**  |  `"none"` `"password"`  |  The value `"password"` prompts for a user name and password. When `"none"` is specified, user name and password parameters aren't used.  | 
|  **Authentication source**  |  A valid MongoDB database name.  |  The name of the MongoDB database that you want to use to validate your credentials for authentication. The default value is `"admin"`.   | 
|  **Authentication mechanism**  |  `"default"` `"mongodb_cr"` `"scram_sha_1"`  |  The authentication mechanism. The value` "default"` is `"scram_sha_1"`. This setting isn't used when `authType` is set to `"no"`.  | 
|  **Metadata mode**  |  Document and table  |  Chooses document mode or table mode.   | 
|  **Number of documents to scan** (`docsToInvestigate`)  |  A positive integer greater than `0`.  |  Use this option in table mode only to define the target table definition.  | 
|  **\$1id as a separate column**  |  Check mark in box  |  Optional check mark box that creates a second column named `_id` that acts as the primary key.  | 
|   `ExtractDocID`   |  `true` `false`  |  `false` – Use this attribute when `NestingLevel` is set to `"none"`.  When using CDC with sources that produce [multi-document transactions](https://www.mongodb.com/docs/manual/reference/method/Session.startTransaction/#mongodb-method-Session.startTransaction), the `ExtractDocId` parameter **must be** set to `true`. If this parameter is not enabled, the AWS DMS task will fail when it encounters a multi-document transaction.  | 
|  `socketTimeoutMS`  |  An integer greater than or equal to 0. Extra Connection Attribute (ECA) only.  |  This setting is in units of milliseconds and configures the connection timeout for MongoDB clients. If the value is less than or equal to zero, then the MongoDB client default is used.  | 
|   `UseUpdateLookUp`   |  `true` `false`  |  When true, during CDC update events, AWS DMS copies over the entire updated document to the target. When set to false, AWS DMS uses the MongoDB update command to only update modified fields in the document on the target.  | 
|   `ReplicateShardCollections`   |  `true` `false`  |  When true, AWS DMS replicates data to shard collections. AWS DMS only uses this setting if the target endpoint is a DocumentDB elastic cluster. When this setting is true, note the following: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html)  | 
|  `useTransactionVerification`  |  `true` `false`  |  When `false`, disables the verification between the change stream and oplogs.   You can miss operations if discrepancies between change streams and oplog entries occur, as the default DMS behavior is to fail the task in such scenarios. Default: `true`.   | 
|  `useOplog`  |  `true` `false`  |  When `true`, enables DMS task to read directly from 'oplog' rather than using the change stream. Default: `false`.  | 

If you choose **Document** as **Metadata mode**, different options are available. 

If the target endpoint is DocumentDB, make sure to run the migration in **Document mode** Also, modify your source endpoint and select the option **\$1id as separate column**. This is a mandatory prerequisite if your source MongoDB workload involves transactions.

## Source data types for MongoDB
<a name="CHAP_Source.MongoDB.DataTypes"></a>

Data migration that uses MongoDB as a source for AWS DMS supports most MongoDB data types. In the following table, you can find the MongoDB source data types that are supported when using AWS DMS and the default mapping from AWS DMS data types. For more information about MongoDB data types, see [ BSON types](https://docs.mongodb.com/manual/reference/bson-types) in the MongoDB documentation.

For information on how to view the data type that is mapped in the target, see the section for the target endpoint that you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  MongoDB data types  |  AWS DMS data types  | 
| --- | --- | 
| Boolean | Bool | 
| Binary | BLOB | 
| Date | Date | 
| Timestamp | Date | 
| Int | INT4 | 
| Long | INT8 | 
| Double | REAL8 | 
| String (UTF-8) | CLOB | 
| Array | CLOB | 
| OID | String | 
| REGEX | CLOB | 
| CODE | CLOB | 

# Using Amazon DocumentDB (with MongoDB compatibility) as a source for AWS DMS
<a name="CHAP_Source.DocumentDB"></a>

For information about versions of Amazon DocumentDB (with MongoDB compatibility) that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

 Using Amazon DocumentDB as a source, you can migrate data from one Amazon DocumentDB cluster to another Amazon DocumentDB cluster. You can also migrate data from an Amazon DocumentDB cluster to one of the other target endpoints supported by AWS DMS.

If you are new to Amazon DocumentDB, be aware of the following important concepts for Amazon DocumentDB databases:
+ A record in Amazon DocumentDB is a *document*, a data structure composed of field and value pairs. The value of a field can include other documents, arrays, and arrays of documents. A document is roughly equivalent to a row in a relational database table.
+ A *collection* in Amazon DocumentDB is a group of documents, and is roughly equivalent to a relational database table.
+ A *database* in Amazon DocumentDB is a set of collections, and is roughly equivalent to a schema in a relational database.

AWS DMS supports two migration modes when using Amazon DocumentDB as a source, document mode and table mode. You specify the migration mode when you create the Amazon DocumentDB source endpoint in the AWS DMS console, using either the **Metadata mode** option or the extra connection attribute `nestingLevel`. Following, you can find an explanation how the choice of migration mode affects the resulting format of the target data.

**Document mode**  
In *document mode, *the JSON document is migrated as is. That means the document data is consolidated into one of two items. When you use a relational database as a target, the data is a single column named `_doc` in a target table. When you use a nonrelational database as a target, the data is a single JSON document. Document mode is the default mode, which we recommend when migrating to an Amazon DocumentDB target.  
For example, consider the following documents in a Amazon DocumentDB collection called `myCollection`.  

```
 db.myCollection.find()
{ "_id" : ObjectId("5a94815f40bd44d1b02bdfe0"), "a" : 1, "b" : 2, "c" : 3 }
{ "_id" : ObjectId("5a94815f40bd44d1b02bdfe1"), "a" : 4, "b" : 5, "c" : 6 }
```
After migrating the data to a relational database table using document mode, the data is structured as follows. The data fields in the document are consolidated into the` _doc` column.      
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DocumentDB.html)
You can optionally set the extra connection attribute `extractDocID` to `true` to create a second column named `"_id"` that acts as the primary key. If you are going to use change data capture (CDC), set this parameter to `true` except when using Amazon DocumentDB as the target.  
When using CDC with sources that produce [multi-document transactions](https://www.mongodb.com/docs/manual/reference/method/Session.startTransaction/#mongodb-method-Session.startTransaction), the `ExtractDocId` parameter **must be** set to `true`. If this parameter is not enabled, the AWS DMS task will fail when it encounters a multi-document transaction.  
If you add a new collection to the source database, AWS DMS creates a new target table for the collection and replicates any documents. 

**Table mode**  
In *table mode, *AWS DMS transforms each top-level field in a Amazon DocumentDB document into a column in the target table. If a field is nested, AWS DMS flattens the nested values into a single column. AWS DMS then adds a key field and data types to the target table's column set.   
For each Amazon DocumentDB document, AWS DMS adds each key and type to the target table's column set. For example, using table mode, AWS DMS migrates the previous example into the following table.      
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DocumentDB.html)
Nested values are flattened into a column containing dot-separated key names. The column is named using the concatenation of the flattened field names separated by periods. For example, AWS DMS migrates a JSON document with a field of nested values such as `{"a" : {"b" : {"c": 1}}}` into a column named `a.b.c.`  
To create the target columns, AWS DMS scans a specified number of Amazon DocumentDB documents and creates a set of all the fields and their types. AWS DMS then uses this set to create the columns of the target table. If you create or modify your Amazon DocumentDB source endpoint using the console, you can specify the number of documents to scan. The default value is 1,000 documents. If you use the AWS CLI, you can use the extra connection attribute `docsToInvestigate`.  
In table mode, AWS DMS manages documents and collections like this:  
+ When you add a document to an existing collection, the document is replicated. If there are fields that don't exist in the target, those fields aren't replicated.
+ When you update a document, the updated document is replicated. If there are fields that don't exist in the target, those fields aren't replicated.
+ Deleting a document is fully supported.
+ Adding a new collection doesn't result in a new table on the target when done during a CDC task.
+ In the Change Data Capture(CDC) phase, AWS DMS doesn't support renaming a collection.

**Topics**
+ [

## Setting permissions to use Amazon DocumentDB as a source
](#CHAP_Source.DocumentDB.Permissions)
+ [

## Configuring CDC for an Amazon DocumentDB cluster
](#CHAP_Source.DocumentDB.ConfigureCDC)
+ [

## Connecting to Amazon DocumentDB using TLS
](#CHAP_Source.DocumentDB.TLS)
+ [

## Creating an Amazon DocumentDB source endpoint
](#CHAP_Source.DocumentDB.ConfigureEndpoint)
+ [

## Segmenting Amazon DocumentDB collections and migrating in parallel
](#CHAP_Source.DocumentDB.ParallelLoad)
+ [

## Migrating multiple databases when using Amazon DocumentDB as a source for AWS DMS
](#CHAP_Source.DocumentDB.Multidatabase)
+ [

## Limitations when using Amazon DocumentDB as a source for AWS DMS
](#CHAP_Source.DocumentDB.Limitations)
+ [

## Using endpoint settings with Amazon DocumentDB as a source
](#CHAP_Source.DocumentDB.ECAs)
+ [

## Source data types for Amazon DocumentDB
](#CHAP_Source.DocumentDB.DataTypes)

## Setting permissions to use Amazon DocumentDB as a source
<a name="CHAP_Source.DocumentDB.Permissions"></a>

When using Amazon DocumentDB source for an AWS DMS migration, you can create a user account with root privileges. Or you can create a user with permissions only for the database to be migrated. 

The following code creates a user as the root account.

```
use admin
db.createUser(
  {
    user: "root",
    pwd: "password",
    roles: [ { role: "root", db: "admin" } ]
  })
```

For Amazon DocumentDB 3.6, the code following creates a user with minimal privileges on the database to be migrated.

```
use db_name
db.createUser( 
    {
        user: "dms-user",
        pwd: "password",
        roles: [{ role: "read", db: "db_name" }]
    }
)
```

For Amazon DocumentDB 4.0 and higher, AWS DMS uses a deployment-wide change stream. Here, the code following creates a user with minimal privileges.

```
db.createUser( 
{ 
    user: "dms-user",
    pwd: "password",
    roles: [ { role: "readAnyDatabase", db: "admin" }] 
})
```

## Configuring CDC for an Amazon DocumentDB cluster
<a name="CHAP_Source.DocumentDB.ConfigureCDC"></a>

To use ongoing replication or CDC with Amazon DocumentDB, AWS DMS requires access to the Amazon DocumentDB cluster's change streams. For a description of the time-ordered sequence of update events in your cluster's collections and databases, see [Using change streams](https://docs.aws.amazon.com/documentdb/latest/developerguide/change_streams.html) in the *Amazon DocumentDB Developer Guide*. 

Authenticate to your Amazon DocumentDB cluster using the MongoDB shell. Then run the following command to enable change streams.

```
db.adminCommand({modifyChangeStreams: 1,
    database: "DB_NAME",
    collection: "", 
    enable: true});
```

This approach enables the change stream for all collections in your database. After change streams are enabled, you can create a migration task that migrates existing data and at the same time replicates ongoing changes. AWS DMS continues to capture and apply changes even after the bulk data is loaded. Eventually, the source and target databases synchronize, minimizing downtime for a migration.

**Note**  
AWS DMS uses the operations log (oplog) to capture changes during ongoing replication. If Amazon DocumentDB flushes out the records from the oplog before AWS DMS reads them, your tasks will fail. We recommend sizing the oplog to retain changes for at least 24 hours.

## Connecting to Amazon DocumentDB using TLS
<a name="CHAP_Source.DocumentDB.TLS"></a>

By default, a newly created Amazon DocumentDB cluster accepts secure connections only using Transport Layer Security (TLS). When TLS is enabled, every connection to Amazon DocumentDB requires a public key.

You can retrieve the public key for Amazon DocumentDB by downloading the file `rds-combined-ca-bundle.pem` from an AWS-hosted Amazon S3 bucket. For more information on downloading this file, see [Encrypting connections using TLS](https://docs.aws.amazon.com/documentdb/latest/developerguide/security.encryption.ssl.html) in the *Amazon DocumentDB Developer Guide.* 

After you download the `rds-combined-ca-bundle.pem` file, you can import the public key that it contains into AWS DMS. The following steps describe how to do so.

**To import your public key using the AWS DMS console**

1. Sign in to the AWS Management Console and choose AWS DMS.

1. In the navigation pane, choose **Certificates**.

1. Choose **Import certificate**. The **Import new CA certificate** page appears.

1. In the **Certificate configuration** section, do one of the following:
   + For **Certificate identifier**, enter a unique name for the certificate, such as `docdb-cert`.
   + Choose **Choose file**, navigate to the location where you saved the `rds-combined-ca-bundle.pem` file, and select it.

1. Choose **Add new CA certificate**.

The AWS CLI following example uses the AWS DMS `import-certificate` command to import the public key `rds-combined-ca-bundle.pem` file.

```
aws dms import-certificate \
    --certificate-identifier docdb-cert \
    --certificate-pem file://./rds-combined-ca-bundle.pem
```

## Creating an Amazon DocumentDB source endpoint
<a name="CHAP_Source.DocumentDB.ConfigureEndpoint"></a>

You can create an Amazon DocumentDB source endpoint using either the console or AWS CLI. Use the procedure following with the console.

**To configure an Amazon DocumentDB source endpoint using the AWS DMS console**

1. Sign in to the AWS Management Console and choose AWS DMS.

1. Choose **Endpoints** from the navigation pane, then choose **Create Endpoint**.

1. For **Endpoint identifier**, provide a name that helps you easily identify it, such as `docdb-source`.

1. For **Source engine**, choose **Amazon DocumentDB (with MongoDB compatibility)**.

1. For **Server name**, enter the name of the server where your Amazon DocumentDB database endpoint resides. For example, you might enter the public DNS name of your Amazon EC2 instance, such as `democluster.cluster-cjf6q8nxfefi.us-east-2.docdb.amazonaws.com`.

1. For **Port**, enter 27017.

1. For **SSL mode**, choose **verify-full**. If you have disabled SSL on your Amazon DocumentDB cluster, you can skip this step.

1. For **CA certificate**, choose the Amazon DocumentDB certificate, `rds-combined-ca-bundle.pem`. For instructions on adding this certificate, see [Connecting to Amazon DocumentDB using TLS](#CHAP_Source.DocumentDB.TLS).

1. For **Database name**, enter the name of the database to be migrated.

Use the following procedure with the CLI.

**To configure an Amazon DocumentDB source endpoint using the AWS CLI**
+ Run the following AWS DMS `create-endpoint` command to configure an Amazon DocumentDB source endpoint, replacing placeholders with your own values.

  ```
  aws dms create-endpoint \
             --endpoint-identifier a_memorable_name \
             --endpoint-type source \
             --engine-name docdb \
             --username value \
             --password value \
             --server-name servername_where_database_endpoint_resides \
             --port 27017 \
             --database-name name_of_endpoint_database
  ```

## Segmenting Amazon DocumentDB collections and migrating in parallel
<a name="CHAP_Source.DocumentDB.ParallelLoad"></a>

To improve performance of a migration task, Amazon DocumentDB source endpoints support two options of the parallel full load feature in table mapping. In other words, you can migrate a collection in parallel by using either the autosegmentation or the range segmentation options of table mapping for a parallel full load in JSON settings. The auto-segmenting options allow you to specify the criteria for AWS DMS to automatically segment your source for migration in each thread. The range segmentation options allow you to tell AWS DMS the specific range of each segment for DMS to migrate in each thread. For more information on these settings, see [Table and collection settings rules and operations](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Tablesettings.md).

### Migrating an Amazon DocumentDB database in parallel using autosegmentation ranges
<a name="CHAP_Source.DocumentDB.ParallelLoad.AutoPartitioned"></a>

You can migrate your documents in parallel by specifying the criteria for AWS DMS to automatically partition (segment) your data for each thread, especially the number of documents to migrate per thread. Using this approach, AWS DMS attempts to optimize segment boundaries for maximum performance per thread.

You can specify the segmentation criteria using the table-settings options following in table-mapping:


|  Table-settings option  |  Description  | 
| --- | --- | 
|  `"type"`  |  (Required) Set to `"partitions-auto"` for Amazon DocumentDB as a source.  | 
|  `"number-of-partitions"`  |  (Optional) Total number of partitions (segments) used for migration. The default is 16.  | 
|  `"collection-count-from-metadata"`  |  (Optional) If set to `true`, AWS DMS uses an estimated collection count for determining the number of partitions. If set to `false`, AWS DMS uses the actual collection count. The default is `true`.  | 
|  `"max-records-skip-per-page"`  |  (Optional) The number of records to skip at once when determining the boundaries for each partition. AWS DMS uses a paginated skip approach to determine the minimum boundary for a partition. The default is 10000. Setting a relatively large value might result in curser timeouts and task failures. Setting a relatively low value results in more operations per page and a slower full load.   | 
|  `"batch-size"`  |  (Optional) Limits the number of documents returned in one batch. Each batch requires a round trip to the server. If the batch size is zero (0), the cursor uses the server-defined maximum batch size. The default is 0.  | 

The example following shows a table mapping for autosegmentation.

```
{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "admin",
                "table-name": "departments"
            },
            "rule-action": "include",
            "filters": []
        },
        {
            "rule-type": "table-settings",
            "rule-id": "2",
            "rule-name": "2",
            "object-locator": {
                "schema-name": "admin",
                "table-name": "departments"
            },
            "parallel-load": {
                "type": "partitions-auto",
                "number-of-partitions": 5,
                "collection-count-from-metadata": "true",
                "max-records-skip-per-page": 1000000,
                "batch-size": 50000
            }
        }
    ]
}
```

Auto-segmentation has the limitation following. The migration for each segment fetches the collection count and the minimum `_id` for the collection separately. It then uses a paginated skip to calculate the minimum boundary for that segment. Therefore, ensure that the minimum `_id` value for each collection remains constant until all the segment boundaries in the collection are calculated. If you change the minimum `_id` value for a collection during its segment boundary calculation, this might cause data loss or duplicate row errors.

### Migrating an Amazon DocumentDB database in parallel using specific segment ranges
<a name="CHAP_Source.DocumentDB.ParallelLoad.Ranges"></a>

The following example shows an Amazon DocumentDB collection that has seven items, and `_id` as the primary key.

![\[Amazon DocumentDB collection with seven items.\]](http://docs.aws.amazon.com/dms/latest/userguide/images/datarep-docdb-collection.png)


To split the collection into three segments and migrate in parallel, you can add table mapping rules to your migration task as shown in the following JSON example.

```
{ // Task table mappings:
  "rules": [
    {
      "rule-type": "selection",
      "rule-id": "1",
      "rule-name": "1",
      "object-locator": {
        "schema-name": "testdatabase",
        "table-name": "testtable"
      },
      "rule-action": "include"
    }, // "selection" :"rule-type"
    {
      "rule-type": "table-settings",
      "rule-id": "2",
      "rule-name": "2",
      "object-locator": {
        "schema-name": "testdatabase",
        "table-name": "testtable"
      },
      "parallel-load": {
        "type": "ranges",
        "columns": [
           "_id",
           "num"
        ],
        "boundaries": [
          // First segment selects documents with _id less-than-or-equal-to 5f805c97873173399a278d79
          // and num less-than-or-equal-to 2.
          [
             "5f805c97873173399a278d79",
             "2"
          ],
          // Second segment selects documents with _id > 5f805c97873173399a278d79 and
          // _id less-than-or-equal-to 5f805cc5873173399a278d7c and
          // num > 2 and num less-than-or-equal-to 5.
          [
             "5f805cc5873173399a278d7c",
             "5"
          ]                                   
          // Third segment is implied and selects documents with _id > 5f805cc5873173399a278d7c.
        ] // :"boundaries"
      } // :"parallel-load"
    } // "table-settings" :"rule-type"
  ] // :"rules"
} // :Task table mappings
```

That table mapping definition splits the source collection into three segments and migrates in parallel. The following are the segmentation boundaries.

```
Data with _id less-than-or-equal-to "5f805c97873173399a278d79" and num less-than-or-equal-to 2 (2 records)
Data with _id less-than-or-equal-to "5f805cc5873173399a278d7c" and num less-than-or-equal-to 5 and not in (_id less-than-or-equal-to  "5f805c97873173399a278d79" and num less-than-or-equal-to 2) (3 records)
Data not in (_id less-than-or-equal-to "5f805cc5873173399a278d7c" and num less-than-or-equal-to 5) (2 records)
```

After the migration task is complete, you can verify from the task logs that the tables loaded in parallel, as shown in the following example. You can also verify the Amazon DocumentDB `find` clause used to unload each segment from the source table.

```
[TASK_MANAGER    ] I:  Start loading segment #1 of 3 of table 'testdatabase'.'testtable' (Id = 1) by subtask 1. Start load timestamp 0005B191D638FE86  (replicationtask_util.c:752)

[SOURCE_UNLOAD   ] I:  Range Segmentation filter for Segment #0 is initialized.   (mongodb_unload.c:157)

[SOURCE_UNLOAD   ] I:  Range Segmentation filter for Segment #0 is: { "_id" : { "$lte" : { "$oid" : "5f805c97873173399a278d79" } }, "num" : { "$lte" : { "$numberInt" : "2" } } }  (mongodb_unload.c:328)

[SOURCE_UNLOAD   ] I: Unload finished for segment #1 of segmented table 'testdatabase'.'testtable' (Id = 1). 2 rows sent.

[TASK_MANAGER    ] I: Start loading segment #1 of 3 of table 'testdatabase'.'testtable' (Id = 1) by subtask 1. Start load timestamp 0005B191D638FE86 (replicationtask_util.c:752)
 
[SOURCE_UNLOAD   ] I: Range Segmentation filter for Segment #0 is initialized. (mongodb_unload.c:157) 

[SOURCE_UNLOAD   ] I: Range Segmentation filter for Segment #0 is: { "_id" : { "$lte" : { "$oid" : "5f805c97873173399a278d79" } }, "num" : { "$lte" : { "$numberInt" : "2" } } } (mongodb_unload.c:328)
 
[SOURCE_UNLOAD   ] I: Unload finished for segment #1 of segmented table 'testdatabase'.'testtable' (Id = 1). 2 rows sent.

[TARGET_LOAD     ] I: Load finished for segment #1 of segmented table 'testdatabase'.'testtable' (Id = 1). 1 rows received. 0 rows skipped. Volume transfered 480.

[TASK_MANAGER    ] I: Load finished for segment #1 of table 'testdatabase'.'testtable' (Id = 1) by subtask 1. 2 records transferred.
```

Currently, AWS DMS supports the following Amazon DocumentDB data types as a segment key column:
+ Double
+ String
+ ObjectId
+ 32 bit integer
+ 64 bit integer

## Migrating multiple databases when using Amazon DocumentDB as a source for AWS DMS
<a name="CHAP_Source.DocumentDB.Multidatabase"></a>

AWS DMS versions 3.4.5 and higher support migrating multiple databases in a single task only for Amazon DocumentDB versions 4.0 and higher. If you want to migrate multiple databases, do the following:

1. When you create the Amazon DocumentDB source endpoint:
   + In the AWS Management Console for AWS DMS, leave **Database name** empty under **Endpoint configuration** on the **Create endpoint** page.
   + In the AWS Command Line Interface (AWS CLI), assign an empty string value to the **DatabaseName** parameter in **DocumentDBSettings** that you specify for the **CreateEndpoint** action.

1. For each database that you want to migrate from this Amazon DocumentDB source endpoint, specify the name of each database as the name of a schema in the table-mapping for the task using either the guided input in the console or directly in JSON. For more information on the guided input, see the description of the [Specifying table selection and transformations rules from the console](CHAP_Tasks.CustomizingTasks.TableMapping.Console.md). For more information on the JSON, see [Selection rules and actions](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Selections.md).

For example, you might specify the JSON following to migrate three Amazon DocumentDB databases.

**Example Migrate all tables in a schema**  
The JSON following migrates all tables from the `Customers`, `Orders`, and `Suppliers` databases in your source enpoint to your target endpoint.  

```
{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "Customers",
                "table-name": "%"
            },
            "object-locator": {
                "schema-name": "Orders",
                "table-name": "%"
            },
            "object-locator": {
                "schema-name": "Inventory",
                "table-name": "%"
            },
            "rule-action": "include"
        }
    ]
}
```

## Limitations when using Amazon DocumentDB as a source for AWS DMS
<a name="CHAP_Source.DocumentDB.Limitations"></a>

The following are limitations when using Amazon DocumentDB as a source for AWS DMS:
+ When the `_id` option is set as a separate column, the ID string can't exceed 200 characters.
+ Object ID and array type keys are converted to columns that are prefixed with `oid` and `array` in table mode.

  Internally, these columns are referenced with the prefixed names. If you use transformation rules in AWS DMS that reference these columns, make sure to specify the prefixed column. For example, specify `${oid__id}` and not `${_id}`, or `${array__addresses}` and not `${_addresses}`. 
+  Collection names and key names can't include the dollar symbol (\$1). 
+ Table mode and document mode have the limitations discussed preceding.
+ Migrating in parallel using autosegmentation has the limitations described preceding.
+ An Amazon DocumentDB (MongoDB compatible) source doesn’t support using a specific timestamp as a start position for change data capture (CDC). An ongoing replication task starts capturing changes regardless of the timestamp.
+ AWS DMS doesn't support documents where the nesting level is greater than 97 for AWS DMS versions lower than 3.5.2.
+ Source filters aren't supported for DocumentDB.
+ AWS DMS doesn’t support CDC (change data capture) replication for DocumentDB as a source in elastic cluster mode.

## Using endpoint settings with Amazon DocumentDB as a source
<a name="CHAP_Source.DocumentDB.ECAs"></a>

You can use endpoint settings to configure your Amazon DocumentDB source database similar to using extra connection attributes. You specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--doc-db-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

The following table shows the endpoint settings that you can use with Amazon DocumentDB as a source.


| Attribute name | Valid values | Default value and description | 
| --- | --- | --- | 
|   `NestingLevel`   |  `"none"` `"one"`  |  `"none"` – Specify `"none"` to use document mode. Specify `"one"` to use table mode.  | 
|   `ExtractDocID`   |  `true` `false`  |  `false` – Use this attribute when `NestingLevel` is set to `"none"`.  When using CDC with sources that produce [multi-document transactions](https://www.mongodb.com/docs/manual/reference/method/Session.startTransaction/#mongodb-method-Session.startTransaction), the `ExtractDocId` parameter **must be** set to `true`. If this parameter is not enabled, the AWS DMS task will fail when it encounters a multi-document transaction.  | 
|   `DocsToInvestigate`   |  A positive integer greater than `0`.  |  `1000` – Use this attribute when `NestingLevel` is set to `"one"`.   | 
|   `ReplicateShardCollections `   |  `true` `false`  |  When true, AWS DMS replicates data to shard collections. AWS DMS only uses this setting if the target endpoint is a DocumentDB elastic cluster. When this setting is true, note the following: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DocumentDB.html)  | 

## Source data types for Amazon DocumentDB
<a name="CHAP_Source.DocumentDB.DataTypes"></a>

In the following table, you can find the Amazon DocumentDB source data types that are supported when using AWS DMS. You can also find the default mapping from AWS DMS data types in this table. For more information about data types, see [BSON types](https://docs.mongodb.com/manual/reference/bson-types) in the MongoDB documentation.

For information on how to view the data type that is mapped in the target, see the section for the target endpoint that you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  Amazon DocumentDB data types  |  AWS DMS data types  | 
| --- | --- | 
| Boolean | Bool | 
| Binary | BLOB | 
| Date | Date | 
| Timestamp | Date | 
| Int | INT4 | 
| Long | INT8 | 
| Double | REAL8 | 
| String (UTF-8) | CLOB | 
| Array | CLOB | 
| OID | String | 

# Using Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3"></a>

You can migrate data from an Amazon S3 bucket using AWS DMS. To do this, provide access to an Amazon S3 bucket containing one or more data files. In that S3 bucket, include a JSON file that describes the mapping between the data and the database tables of the data in those files.

The source data files must be present in the Amazon S3 bucket before the full load starts. You specify the bucket name using the `bucketName` parameter. 

The source data files can be in the following formats:
+ Comma-separated value (.csv)
+ Parquet (DMS version 3.5.3 and later). For information about using Parquet-format files, see [Using Parquet-format files in Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.Parquet).

For source data files in comma-separated value (.csv) format, name them using the following naming convention. In this convention, *`schemaName`* is the source schema and *`tableName`* is the name of a table within that schema.

```
/schemaName/tableName/LOAD001.csv
/schemaName/tableName/LOAD002.csv
/schemaName/tableName/LOAD003.csv
...
```

 For example, suppose that your data files are in `amzn-s3-demo-bucket`, at the following Amazon S3 path.

```
s3://amzn-s3-demo-bucket/hr/employee
```

At load time, AWS DMS assumes that the source schema name is `hr`, and that the source table name is `employee`.

In addition to `bucketName` (which is required), you can optionally provide a `bucketFolder` parameter to specify where AWS DMS should look for data files in the Amazon S3 bucket. Continuing the previous example, if you set `bucketFolder` to `sourcedata`, then AWS DMS reads the data files at the following path.

```
s3://amzn-s3-demo-bucket/sourcedata/hr/employee
```

You can specify the column delimiter, row delimiter, null value indicator, and other parameters using extra connection attributes. For more information, see [Endpoint settings for Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.Configuring).

You can specify a bucket owner and prevent sniping by using the `ExpectedBucketOwner` Amazon S3 endpoint setting, as shown following. Then, when you make a request to test a connection or perform a migration, S3 checks the account ID of the bucket owner against the specified parameter.

```
--s3-settings='{"ExpectedBucketOwner": "AWS_Account_ID"}'
```

**Topics**
+ [

## Defining external tables for Amazon S3 as a source for AWS DMS
](#CHAP_Source.S3.ExternalTableDef)
+ [

## Using CDC with Amazon S3 as a source for AWS DMS
](#CHAP_Source.S3.CDC)
+ [

## Prerequisites when using Amazon S3 as a source for AWS DMS
](#CHAP_Source.S3.Prerequisites)
+ [

## Limitations when using Amazon S3 as a source for AWS DMS
](#CHAP_Source.S3.Limitations)
+ [

## Endpoint settings for Amazon S3 as a source for AWS DMS
](#CHAP_Source.S3.Configuring)
+ [

## Source data types for Amazon S3
](#CHAP_Source.S3.DataTypes)
+ [

## Using Parquet-format files in Amazon S3 as a source for AWS DMS
](#CHAP_Source.S3.Parquet)

## Defining external tables for Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3.ExternalTableDef"></a>

In addition to the data files, you must also provide an external table definition. An *external table definition* is a JSON document that describes how AWS DMS should interpret the data from Amazon S3. The maximum size of this document is 2 MB. If you create a source endpoint using the AWS DMS Management Console, you can enter the JSON directly into the table-mapping box. If you use the AWS Command Line Interface (AWS CLI) or AWS DMS API to perform migrations, you can create a JSON file to specify the external table definition.

Suppose that you have a data file that includes the following.

```
101,Smith,Bob,2014-06-04,New York
102,Smith,Bob,2015-10-08,Los Angeles
103,Smith,Bob,2017-03-13,Dallas
104,Smith,Bob,2017-03-13,Dallas
```

Following is an example external table definition for this data.

```
{
    "TableCount": "1",
    "Tables": [
        {
            "TableName": "employee",
            "TablePath": "hr/employee/",
            "TableOwner": "hr",
            "TableColumns": [
                {
                    "ColumnName": "Id",
                    "ColumnType": "INT8",
                    "ColumnNullable": "false",
                    "ColumnIsPk": "true"
                },
                {
                    "ColumnName": "LastName",
                    "ColumnType": "STRING",
                    "ColumnLength": "20"
                },
                {
                    "ColumnName": "FirstName",
                    "ColumnType": "STRING",
                    "ColumnLength": "30"
                },
                {
                    "ColumnName": "HireDate",
                    "ColumnType": "DATETIME"
                },
                {
                    "ColumnName": "OfficeLocation",
                    "ColumnType": "STRING",
                    "ColumnLength": "20"
                }
            ],
            "TableColumnsTotal": "5"
        }
    ]
}
```

The elements in this JSON document are as follows:

`TableCount` – the number of source tables. In this example, there is only one table.

`Tables` – an array consisting of one JSON map per source table. In this example, there is only one map. Each map consists of the following elements:
+ `TableName` – the name of the source table.
+ `TablePath` – the path in your Amazon S3 bucket where AWS DMS can find the full data load file. If a `bucketFolder` value is specified, its value is prepended to the path.
+ `TableOwner` – the schema name for this table.
+ `TableColumns` – an array of one or more maps, each of which describes a column in the source table:
  + `ColumnName` – the name of a column in the source table.
  + `ColumnType` – the data type for the column. For valid data types, see [Source data types for Amazon S3](#CHAP_Source.S3.DataTypes).
  + `ColumnLength` – the number of bytes in this column. Maximum column length is limited to2147483647 Bytes (2,047 MegaBytes) since an S3 source doesn't support FULL LOB mode. `ColumnLength` is valid for the following data types:
    + BYTE
    + STRING
  + `ColumnNullable` – a Boolean value that is `true` if this column can contain NULL values (default=`false`).
  + `ColumnIsPk` – a Boolean value that is `true` if this column is part of the primary key (default=`false`).
  + `ColumnDateFormat` – the input date format for a column with DATE, TIME, and DATETIME types, and used to parse a data string into a date object. Possible values include:

    ```
    - YYYY-MM-dd HH:mm:ss
    - YYYY-MM-dd HH:mm:ss.F
    - YYYY/MM/dd HH:mm:ss
    - YYYY/MM/dd HH:mm:ss.F
    - MM/dd/YYYY HH:mm:ss
    - MM/dd/YYYY HH:mm:ss.F
    - YYYYMMdd HH:mm:ss
    - YYYYMMdd HH:mm:ss.F
    ```
+ `TableColumnsTotal` – the total number of columns. This number must match the number of elements in the `TableColumns` array.

If you don't specify otherwise, AWS DMS assumes that `ColumnLength` is zero.

**Note**  
In supported versions of AWS DMS, the S3 source data can also contain an optional operation column as the first column before the `TableName` column value. This operation column identifies the operation (`INSERT`) used to migrate the data to an S3 target endpoint during a full load.   
If present, the value of this column is the initial character of the `INSERT` operation keyword (`I`). If specified, this column generally indicates that the S3 source was created by DMS as an S3 target during a previous migration.   
In DMS versions prior to 3.4.2, this column wasn't present in S3 source data created from a previous DMS full load. Adding this column to S3 target data allows the format of all rows written to the S3 target to be consistent whether they are written during a full load or during a CDC load. For more information on the options for formatting S3 target data, see [Indicating source DB operations in migrated S3 data](CHAP_Target.S3.md#CHAP_Target.S3.Configuring.InsertOps).

For a column of the NUMERIC type, specify the precision and scale. *Precision* is the total number of digits in a number, and *scale* is the number of digits to the right of the decimal point. You use the `ColumnPrecision` and `ColumnScale` elements for this, as shown following.

```
...
    {
        "ColumnName": "HourlyRate",
        "ColumnType": "NUMERIC",
        "ColumnPrecision": "5"
        "ColumnScale": "2"
    }
...
```

For a column of the DATETIME type with data that contains fractional seconds, specify the scale. *Scale* is the number of digits for the fractional seconds, and can range from 0 to 9. You use the `ColumnScale` element for this, as shown following.

```
...
{
      "ColumnName": "HireDate",
      "ColumnType": "DATETIME",
      "ColumnScale": "3"
}
...
```

If you don't specify otherwise, AWS DMS assumes `ColumnScale` is zero and truncates the fractional seconds.

## Using CDC with Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3.CDC"></a>

After AWS DMS performs a full data load, it can optionally replicate data changes to the target endpoint. To do this, you upload change data capture files (CDC files) to your Amazon S3 bucket. AWS DMS reads these CDC files when you upload them, and then applies the changes at the target endpoint. 

The CDC files are named as follows:

```
CDC00001.csv
CDC00002.csv
CDC00003.csv
...
```

**Note**  
To replicate CDC files in the change data folder successfully upload them in a lexical (sequential) order. For example, upload the file CDC00002.csv before the file CDC00003.csv. Otherwise, CDC00002.csv is skipped and isn't replicated if you load it after CDC00003.csv. But the file CDC00004.csv replicates successfully if loaded after CDC00003.csv.

To indicate where AWS DMS can find the files, specify the `cdcPath` parameter. Continuing the previous example, if you set `cdcPath` to `changedata`, then AWS DMS reads the CDC files at the following path.

```
s3://amzn-s3-demo-bucket/changedata
```

If you set `cdcPath` to `changedata` and `bucketFolder` to `myFolder`, then AWS DMS reads the CDC files at the following path.

```
s3://amzn-s3-demo-bucket/myFolder/changedata
```

The records in a CDC file are formatted as follows:
+ Operation – the change operation to be performed: `INSERT` or `I`, `UPDATE` or `U`, or `DELETE` or `D`. These keyword and character values are case-insensitive.
**Note**  
In supported AWS DMS versions, AWS DMS can identify the operation to perform for each load record in two ways. AWS DMS can do this from the record's keyword value (for example, `INSERT`) or from its keyword initial character (for example, `I`). In prior versions, AWS DMS recognized the load operation only from the full keyword value.   
In prior versions of AWS DMS, the full keyword value was written to log the CDC data. Also, prior versions wrote the operation value to any S3 target using only the keyword initial.   
Recognizing both formats allows AWS DMS to handle the operation regardless of how the operation column is written to create the S3 source data. This approach supports using S3 target data as the source for a later migration. With this approach, you don't need to change the format of any keyword initial value that appears in the operation column of the later S3 source.
+ Table name – the name of the source table.
+ Schema name – the name of the source schema.
+ Data – one or more columns that represent the data to be changed.

Following is an example CDC file for a table named `employee`.

```
INSERT,employee,hr,101,Smith,Bob,2014-06-04,New York
UPDATE,employee,hr,101,Smith,Bob,2015-10-08,Los Angeles
UPDATE,employee,hr,101,Smith,Bob,2017-03-13,Dallas
DELETE,employee,hr,101,Smith,Bob,2017-03-13,Dallas
```

## Prerequisites when using Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3.Prerequisites"></a>

To use Amazon S3 as a source for AWS DMS, your source S3 bucket must be in the same AWS Region as the DMS replication instance that migrates your data. In addition, the AWS account you use for the migration must have read access to the source bucket. For AWS DMS version 3.4.7 and higher, DMS must access the source bucket through a VPC endpoint or a public route. For information about VPC endpoints, see [Configuring VPC endpoints for AWS DMS](CHAP_VPC_Endpoints.md).

The AWS Identity and Access Management (IAM) role assigned to the user account used to create the migration task must have the following set of permissions.

------
#### [ JSON ]

****  

```
{
    "Version":"2012-10-17",		 	 	 
    "Statement": [
       {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::amzn-s3-demo-bucket*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::amzn-s3-demo-bucket*"
            ]
        }
    ]
}
```

------

The AWS Identity and Access Management (IAM) role assigned to the user account used to create the migration task must have the following set of permissions if versioning is enabled on the Amazon S3 bucket.

------
#### [ JSON ]

****  

```
{
    "Version":"2012-10-17",		 	 	 
    "Statement": [
       {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::amzn-s3-demo-bucket*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::amzn-s3-demo-bucket*"
            ]
        }
    ]
}
```

------

## Limitations when using Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3.Limitations"></a>

The following limitations apply when using Amazon S3 as a source:
+ Don’t enable versioning for S3. If you need S3 versioning, use lifecycle policies to actively delete old versions. Otherwise, you might encounter endpoint test connection failures because of an S3 `list-object` call timeout. To create a lifecycle policy for an S3 bucket, see [ Managing your storage lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html). To delete a version of an S3 object, see [ Deleting object versions from a versioning-enabled bucket](https://docs.aws.amazon.com/AmazonS3/latest/dev/DeletingObjectVersions.html).
+ A VPC-enabled (gateway VPC) S3 bucket is supported in versions 3.4.7 and higher.
+ MySQL converts the `time` datatype to `string`. To see `time` data type values in MySQL, define the column in the target table as `string`, and set the task's **Target table preparation mode** setting to **Truncate**.
+ AWS DMS uses the `BYTE` data type internally for data in both `BYTE` and `BYTES` data types.
+ S3 source endpoints do not support the DMS table reload feature.
+ AWS DMS doesn't support Full LOB mode with Amazon S3 as a Source.

The following limitations apply when using Parquet-format files in Amazon S3 as a source:
+ Dates in `MMYYYYDD`, or `DDMMYYYY` are not supported for the S3 Parquet Source date-partitioning feature.

## Endpoint settings for Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3.Configuring"></a>

You can use endpoint settings to configure your Amazon S3 source database similar to using extra connection attributes. You specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/index.html), with the `--s3-settings '{"EndpointSetting": "value", ...}'` JSON syntax.

**Note**  
AWS DMS defaults to a secure connection to the Amazon S3 endpoint without requiring to specify SSL mode or certificate.

The following table shows the endpoint settings that you can use with Amazon S3 as a source.


| **Option** | **Description** | 
| --- | --- | 
| BucketFolder |  (Optional) A folder name in the S3 bucket. If this attribute is provided, source data files and CDC files are read from the path `s3://amzn-s3-demo-bucket/bucketFolder/schemaName/tableName/` and `s3://amzn-s3-demo-bucket/bucketFolder/` respectively. If this attribute isn't specified, then the path used is `schemaName/tableName/`.  `'{"BucketFolder": "sourceData"}'`  | 
| BucketName |  The name of the S3 bucket. `'{"BucketName": "amzn-s3-demo-bucket"}'`  | 
| CdcPath | The location of CDC files. This attribute is required if a task captures change data; otherwise, it's optional. If CdcPath is present, then AWS DMS reads CDC files from this path and replicates the data changes to the target endpoint. For more information, see [Using CDC with Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.CDC). `'{"CdcPath": "changeData"}'`  | 
| CsvDelimiter |  The delimiter used to separate columns in the source files. The default is a comma. An example follows. `'{"CsvDelimiter": ","}'`  | 
| CsvNullValue |  A user-defined string that AWS DMS treats as null when reading from the source. The default is an empty string. If you do not set this parameter, AWS DMS treats an empty string as a null value. If you set this parameter to a string such as "\$1N", AWS DMS treats this string as the null value, and treats empty strings as an empty string value.  | 
| CsvRowDelimiter |  The delimiter used to separate rows in the source files. The default is a newline (`\n`). `'{"CsvRowDelimiter": "\n"}'`  | 
| DataFormat |  Set this value to `Parquet` to read data in Parquet format. `'{"DataFormat": "Parquet"}'`  | 
| IgnoreHeaderRows |  When this value is set to 1, AWS DMS ignores the first row header in a .csv file. A value of 1 enables the feature, a value of 0 disables the feature. The default is 0. `'{"IgnoreHeaderRows": 1}'`  | 
| Rfc4180 |  When this value is set to `true` or `y`, each leading double quotation mark has to be followed by an ending double quotation mark. This formatting complies with RFC 4180. When this value is set to `false` or `n`, string literals are copied to the target as is. In this case, a delimiter (row or column) signals the end of the field. Thus, you can't use a delimiter as part of the string, because it signals the end of the value. The default is `true`. Valid values: `true`, `false`, `y`, `n` `'{"Rfc4180": false}'`  | 

## Source data types for Amazon S3
<a name="CHAP_Source.S3.DataTypes"></a>

Data migration that uses Amazon S3 as a source for AWS DMS needs to map data from Amazon S3 to AWS DMS data types. For more information, see [Defining external tables for Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.ExternalTableDef).

For information on how to view the data type that is mapped in the target, see the section for the target endpoint you are using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).

The following AWS DMS data types are used with Amazon S3 as a source:
+ BYTE – Requires `ColumnLength`. For more information, see [Defining external tables for Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.ExternalTableDef).
+ DATE
+ TIME
+ DATETIME – For more information and an example, see the DATETIME type example in [Defining external tables for Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.ExternalTableDef).
+ INT1
+ INT2
+ INT4
+ INT8
+ NUMERIC – Requires `ColumnPrecision` and `ColumnScale`. AWS DMS supports the following maximum values:
  + **ColumnPrecision: 38**
  + **ColumnScale: 31**

  For more information and an example, see the NUMERIC type example in [Defining external tables for Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.ExternalTableDef).
+ REAL4
+ REAL8
+ STRING – Requires `ColumnLength`. For more information, see [Defining external tables for Amazon S3 as a source for AWS DMS](#CHAP_Source.S3.ExternalTableDef).
+ UINT1
+ UINT2
+ UINT4
+ UINT8
+ BLOB
+ CLOB
+ BOOLEAN

## Using Parquet-format files in Amazon S3 as a source for AWS DMS
<a name="CHAP_Source.S3.Parquet"></a>

In AWS DMS version 3.5.3 and later, you can use Parquet-format files in an S3 bucket as a source for both Full-Load or CDC replication. 

DMS only supports Parquet format files as a source that DMS generates by migrating data to an S3 target endpoint. File names must be in the supported format, or DMS won't include them in the migration.

For source data files in Parquet format, they must be in the following folder and naming convention.

```
schema/table1/LOAD00001.parquet
schema/table2/LOAD00002.parquet
schema/table2/LOAD00003.parquet
```

For source data files for CDC data in Parquet format, name and store them using the following folder and naming convention.

```
schema/table/20230405-094615814.parquet
schema/table/20230405-094615853.parquet
schema/table/20230405-094615922.parquet
```

To access files in Parquet format, set the following endpoint settings:
+ Set `DataFormat` to `Parquet`. 
+ Do not set the `cdcPath` setting. Make sure that you create your Parquet-format files in the specified schema/ table folders. 

For more information about settings for S3 endpoints, see [S3Settings](https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html) in the *AWS Database Migration Service API Reference*.

### Supported datatypes for Parquet-format files
<a name="CHAP_Source.S3.Parquet.Datatypes"></a>

AWS DMS supports the following source and target data types when migrating data from Parquet-format files. Ensure that your target table has columns of the correct data types before migrating.


| Source data type | Target data type | 
| --- | --- | 
| BYTE | BINARY | 
| DATE | DATE32 | 
| TIME | TIME32 | 
| DATETIME | TIMESTAMP | 
| INT1 | INT8 | 
| INT2 | INT16 | 
| INT4 | INT32 | 
| INT8 | INT64 | 
| NUMERIC | DECIMAL | 
| REAL4 | FLOAT | 
| REAL8 | DOUBLE | 
| STRING | STRING | 
| UINT1 | UINT8 | 
| UINT2 | UINT16 | 
| UINT4 | UINT32 | 
| UINT8 | UINT | 
| WSTRING | STRING | 
| BLOB | BINARY | 
| NCLOB | STRING | 
| CLOB | STRING | 
| BOOLEAN | BOOL | 

# Using IBM Db2 for Linux, Unix, Windows, and Amazon RDS database (Db2 LUW) as a source for AWS DMS
<a name="CHAP_Source.DB2"></a>

You can migrate data from an IBM Db2 for Linux, Unix, Windows, and Amazon RDS (Db2 LUW) database to any supported target database using AWS Database Migration Service (AWS DMS). 

For information about versions of Db2 on Linux, Unix, Windows, and RDS that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md). 

You can use Secure Sockets Layer (SSL) to encrypt connections between your Db2 LUW endpoint and the replication instance. For more information on using SSL with a Db2 LUW endpoint, see [Using SSL with AWS Database Migration Service](CHAP_Security.SSL.md).

When AWS DMS reads data from an IBM Db2 source database, it uses the default isolation level CURSOR STABILITY (CS) for Db2 version 9.7 and above. For more information, see [IBM Db2 for Linux, UNIX and Windows](https://www.ibm.com/docs/en/db2/12.1.0) documentation.

## Prerequisites when using Db2 LUW as a source for AWS DMS
<a name="CHAP_Source.DB2.Prerequisites"></a>

The following prerequisites are required before you can use an Db2 LUW database as a source.

To enable ongoing replication, also called change data capture (CDC), do the following:
+ Set the database to be recoverable, which AWS DMS requires to capture changes. A database is recoverable if either or both of the database configuration parameters `LOGARCHMETH1` and `LOGARCHMETH2` are set to `ON`.

  If your database is recoverable, then AWS DMS can access the Db2 `ARCHIVE LOG` if needed.
+ Ensure that the DB2 transaction logs are available, with a sufficient retention period to be processed by AWS DMS. 
+ DB2 requires `SYSADM` or `DBADM` authorization to extract transaction log records. Grant the user account the following permissions:
  + `SYSADM` or `DBADM`
  + `DATAACCESS`
**Note**  
For full-load only tasks, the DMS user account needs DATAACCESS permission.
+ When using IBM DB2 for LUW version 9.7 as a source, set the extra connection attribute (ECA), `CurrentLsn` as follows:

  `CurrentLsn=LSN` where `LSN` specifies a log sequence number (LSN) where you want the replication to start. Or, `CurrentLsn=scan`.
+ When using Amazon RDS for Db2 LUW as a source, ensure that the archive logs are available to AWS DMS. Because AWS-managed Db2 databases purge the archive logs as soon as possible, you should increase the length of time that the logs remain available. For example, to increase log retention to 24 hours, run the following command:

  ```
  db2 "call rdsadmin.set_archive_log_retention( ?, 'TESTDB', '24')"
  ```

  For more information about Amazon RDS for Db2 LUW procedures, see the [ Amazon RDS for Db2 stored procedure reference ](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/db2-stored-procedures.html) in the *Amazon Relational Database Service User Guide*.
+ Grant the following privileges if you use DB2 specific premigration assessments:

  ```
  GRANT CONNECT ON DATABASE TO USER <DMS_USER>;
  GRANT SELECT ON SYSIBM.SYSDUMMY1 TO USER <DMS_USER>;
  GRANT SELECT ON SYSIBMADM.ENV_INST_INFO TO USER <DMS_USER>;
  GRANT SELECT ON SYSIBMADM.DBCFG TO USER <DMS_USER>;
  GRANT SELECT ON SYSCAT.SCHEMATA TO USER <DMS_USER>;
  GRANT SELECT ON SYSCAT.COLUMNS TO USER <DMS_USER>;
  GRANT SELECT ON SYSCAT.TABLES TO USER <DMS_USER>;
  GRANT EXECUTE ON FUNCTION SYSPROC.AUTH_LIST_AUTHORITIES_FOR_AUTHID TO <DMS_USER>;
  GRANT EXECUTE ON PACKAGE NULLID.SYSSH200 TO USER <DMS_USER>;
  ```

## Limitations when using Db2 LUW as a source for AWS DMS
<a name="CHAP_Source.DB2.Limitations"></a>

AWS DMS doesn't support clustered databases. However, you can define a separate Db2 LUW for each of the endpoints of a cluster. For example, you can create a Full Load migration task with any one of the nodes in the cluster, then create separate tasks from each node.

AWS DMS doesn't support the `BOOLEAN` data type in your source Db2 LUW database.

When using ongoing replication (CDC), the following limitations apply:
+ When a table with multiple partitions is truncated, the number of DDL events shown in the AWS DMS console is equal to the number of partitions. This is because Db2 LUW records a separate DDL for each partition.
+ The following DDL actions aren't supported on partitioned tables:
  + ALTER TABLE ADD PARTITION
  + ALTER TABLE DETACH PARTITION
  + ALTER TABLE ATTACH PARTITION
+ AWS DMS doesn't support an ongoing replication migration from a DB2 high availability disaster recovery (HADR) standby instance. The standby is inaccessible.
+ The DECFLOAT data type isn't supported. Consequently, changes to DECFLOAT columns are ignored during ongoing replication.
+ The RENAME COLUMN statement isn't supported.
+ When performing updates to Multi-Dimensional Clustering (MDC) tables, each update is shown in the AWS DMS console as INSERT \$1 DELETE.
+ When the task setting **Include LOB columns in replication** isn't enabled, any table that has LOB columns is suspended during ongoing replication.
+ For Db2 LUW versions 10.5 and higher, variable-length string columns with data that is stored out-of-row are ignored. This limitation only applies to tables created with extended row size for columns with data types like VARCHAR and VARGRAPHIC. To work around this limitation, move the table to a table space with a higher page size. For more information, see [ What can I do if I want to change the pagesize of DB2 tablespaces]( https://www.ibm.com/support/pages/what-can-i-do-if-i-want-change-pagesize-db2-tablespaces ).
+ For ongoing replication, DMS doesn't support migrating data loaded at the page level by the DB2 LOAD utility. Instead, use the IMPORT utility which uses SQL inserts. For more information, see [ differences between the import and load utilities]( https://www.ibm.com/docs/en/db2/11.1?topic=utilities-differences-between-import-load-utility). 
+ While a replication task is running, DMS captures CREATE TABLE DDLs only if the tables were created with the DATA CAPTURE CHANGE attribute.
+ DMS has the following limitations when using the Db2 Database Partition Feature (DPF):
  + DMS can't coordinate transactions across Db2 nodes in a DPF environment. This is due to constraints within the IBM DB2READLOG API interface. In DPF, transactions may span multiple Db2 nodes, depending upon how DB2 partitions the data. As a result, your DMS solution must capture transactions from each Db2 node independently.
  + DMS can capture local transactions from each Db2 node in the DPF cluster by setting `connectNode` to `1` on multiple DMS source endpoints. This configuration corresponds to logical node numbers defined in the DB2 server configuration file `db2nodes.cfg`.
  + Local transactions on individual Db2 nodes may be parts of a larger, global transaction. DMS applies each local transaction independently on the target, without coordination with transactions on other Db2 nodes. This independent processing can lead to complications, especially when rows are moved between partitions.
  + When DMS replicates from multiple Db2 nodes, there is no assurance of the correct order of operations on the target, because DMS applies operations independently for each Db2 node. You must ensure that capturing local transactions independently from each Db2 node works for your specific use case.
  + When migrating from a DPF environment, we recommend first running a Full Load task without cached events, and then running CDC-only tasks. We recommend running one task per Db2 node, starting from the Full Load start timestamp or LRI (log record identifier) you set using the `StartFromContext` endpoint extra connection attribute. For information about determining your replication start point, see [ Finding the LSN or LRI value for replication start ](https://www.ibm.com/support/pages/db2-finding-lsn-or-lri-value-replication-start) in the *IBM Support documentation*. 
+ For ongoing replication (CDC), if you plan to start replication from a specific timestamp, you must set the `StartFromContext` extra connection attribute to the required timestamp.
+ Currently, DMS doesn't support the Db2 pureScale Feature, an extension of DB2 LUW that you can use to scale your database solution.
+ The `DATA CAPTURE CHANGES` table option is a crucial prerequisite for DB2 data replication processes. Neglecting to enable this option when creating tables can cause missing data, especially for CDC (Change Data Capture) only replication tasks initiated from an earlier starting point. AWS DMS will enable this attribute by default when restarting a CDC or FULL\$1CDC task. However, any changes made in the source database before the task restart may be missed.

  ```
  ALTER TABLE TABLE_SCHEMA.TABLE_NAME DATA CAPTURE CHANGES INCLUDE LONGVAR COLUMNS;
  ```

## Endpoint settings when using Db2 LUW as a source for AWS DMS
<a name="CHAP_Source.DB2.ConnectionSettings"></a>

You can specify the settings when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/create-endpoint.html), with the

`--ibm-db2-settings '{"EndpointSetting1": "value1","EndpointSetting2": "value2"}'`

JSON syntax.

The following table shows the endpoint settings that you can use with Db2 LUW as a source.


| Setting Name | Description | 
| --- | --- | 
|  `CurrentLsn`  |  For ongoing replication (CDC), use `CurrentLsn` to specify a log sequence number (LSN) where you want the replication to start.   | 
|  `MaxKBytesPerRead`  |  Maximum number of bytes per read, as a NUMBER value. The default is 64 KB.  | 
|  `SetDataCaptureChanges`  |  Enables ongoing replication (CDC) as a BOOLEAN value. The default is true.  | 

## Extra Connection Attributes (ECAs) when using Db2 LUW as a source for AWS DMS
<a name="CHAP_Source.DB2.ConnectionAttrib"></a>

You can specify the Extra Connection Attributes (ECAs) when you create the source endpoint using the AWS DMS console, or by using the `create-endpoint` command in the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/dms/create-endpoint.html), with the

`--extra-connection-attributes 'ECAname1=value1;ECAname2=value2;'`

The following table shows the ECAs that you can use with Db2 LUW as a source.


| Attribute Name | Description | 
| --- | --- | 
|  `ConnectionTimeout`  |  Use this ECA to set the endpoint connection timeout for the Db2 LUW endpoint, in seconds. The default value is 10 seconds. Example: `ConnectionTimeout=30;`  | 
|  `executeTimeout`  |  Extra connection attribute which sets the statement (query) timeout for the DB2 LUW endpoint, in seconds. The default value is 60 seconds. Example: `executeTimeout=120;`  | 
|  `StartFromContext`  |  For ongoing replication (CDC), use `StartFromContext` to specify a log's lower limit from where to start the replication. `StartFromContext` accepts different forms of values. Valid values include: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DB2.html) [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DB2.html) [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.DB2.html) To determine the LRI/LSN range of a log file, run the `db2flsn` command as shown in the example following. <pre>db2flsn -db SAMPLE -lrirange 2</pre> The output from that example is similar to the following.  <pre><br />S0000002.LOG: has LRI range 00000000000000010000000000002254000000000004F9A6 to <br />000000000000000100000000000022CC000000000004FB13</pre> In that output, the log file is S0000002.LOG and the **StartFromContext** LRI value is the 34 bytes at the end of the range. <pre>0100000000000022CC000000000004FB13</pre>  | 

## Source data types for IBM Db2 LUW
<a name="CHAP_Source.DB2.DataTypes"></a>

Data migration that uses Db2 LUW as a source for AWS DMS supports most Db2 LUW data types. The following table shows the Db2 LUW source data types that are supported when using AWS DMS and the default mapping from AWS DMS data types. For more information about Db2 LUW data types, see the [Db2 LUW documentation](https://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.sql.ref.doc/doc/r0008483.html).

For information on how to view the data type that is mapped in the target, see the section for the target endpoint that you're using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  Db2 LUW data types  |  AWS DMS data types  | 
| --- | --- | 
|  INTEGER  |  INT4  | 
|  SMALLINT  |  INT2  | 
|  BIGINT  |  INT8  | 
|  DECIMAL (p,s)  |  NUMERIC (p,s)  | 
|  FLOAT  |  REAL8  | 
|  DOUBLE  |  REAL8  | 
|  REAL  |  REAL4  | 
|  DECFLOAT (p)  |  If precision is 16, then REAL8; if precision is 34, then STRING  | 
|  GRAPHIC (n)  |  WSTRING, for fixed-length graphic strings of double byte chars with a length greater than 0 and less than or equal to 127  | 
|  VARGRAPHIC (n)  |  WSTRING, for varying-length graphic strings with a length greater than 0 and less than or equal to16,352 double byte chars  | 
|  LONG VARGRAPHIC (n)  |  CLOB, for varying-length graphic strings with a length greater than 0 and less than or equal to16,352 double byte chars  | 
|  CHARACTER (n)  |  STRING, for fixed-length strings of double byte chars with a length greater than 0 and less than or equal to 255  | 
|  VARCHAR (n)  |  STRING, for varying-length strings of double byte chars with a length greater than 0 and less than or equal to 32,704  | 
|  LONG VARCHAR (n)  |  CLOB, for varying-length strings of double byte chars with a length greater than 0 and less than or equal to 32,704  | 
|  CHAR (n) FOR BIT DATA  |  BYTES  | 
|  VARCHAR (n) FOR BIT DATA  |  BYTES  | 
|  LONG VARCHAR FOR BIT DATA  |  BYTES  | 
|  DATE  |  DATE  | 
|  TIME  |  TIME  | 
|  TIMESTAMP  |  DATETIME  | 
|  BLOB (n)  |  BLOB Maximum length is 2,147,483,647 bytes  | 
|  CLOB (n)  |  CLOB Maximum length is 2,147,483,647 bytes  | 
|  DBCLOB (n)  |  CLOB Maximum length is 1,073,741,824 double byte chars  | 
|  XML  |  CLOB  | 

# Using IBM Db2 for z/OS databases as a source for AWS DMS
<a name="CHAP_Source.DB2zOS"></a>

You can migrate data from an IBM for z/OS database to any supported target database using AWS Database Migration Service (AWS DMS). 

For information about versions of Db2 for z/OS that AWS DMS supports as a source, see [Sources for AWS DMS](CHAP_Introduction.Sources.md).

## Prerequisites when using Db2 for z/OS as a source for AWS DMS
<a name="CHAP_Source.DB2zOS.Prerequisites"></a>

To use an IBM Db2 for z/OS database as a source in AWS DMS, grant the following privileges to the Db2 for z/OS user specified in the source endpoint connection settings.

```
GRANT SELECT ON SYSIBM.SYSTABLES TO Db2USER;
GRANT SELECT ON SYSIBM.SYSTABLESPACE TO Db2USER;
GRANT SELECT ON SYSIBM.SYSTABLEPART TO Db2USER;                    
GRANT SELECT ON SYSIBM.SYSCOLUMNS TO Db2USER;
GRANT SELECT ON SYSIBM.SYSDATABASE TO Db2USER;
GRANT SELECT ON SYSIBM.SYSDUMMY1 TO Db2USER
```

Also grant SELECT ON `user defined` source tables.

An AWS DMS IBM Db2 for z/OS source endpoint relies on the IBM Data Server Driver for ODBC to access data. The database server must have a valid IBM ODBC Connect license for DMS to connect to this endpoint.

## Limitations when using Db2 for z/OS as a source for AWS DMS
<a name="CHAP_Source.DB2zOS.Limitations"></a>

The following limitations apply when using an IBM Db2 for z/OS database as a source for AWS DMS:
+ Only Full Load replication tasks are supported. Change data capture (CDC) isn't supported.
+ Parallel load isn't supported.
+ Data validation of views are not supported.
+ Schema, table, and columns names must be specified in UPPER case in table mappings for Column/table level transformations and row level selection filters.

## Source data types for IBM Db2 for z/OS
<a name="CHAP_Source.DB2zOS.DataTypes"></a>

Data migrations that use Db2 for z/OS as a source for AWS DMS support most Db2 for z/OS data types. The following table shows the Db2 for z/OS source data types that are supported when using AWS DMS, and the default mapping from AWS DMS data types.

For more information about Db2 for z/OS data types, see the [IBM Db2 for z/OS documentation](https://www.ibm.com/docs/en/db2-for-zos/12?topic=elements-data-types).

For information on how to view the data type that is mapped in the target, see the section for the target endpoint that you're using.

For additional information about AWS DMS data types, see [Data types for AWS Database Migration Service](CHAP_Reference.DataTypes.md).


|  Db2 for z/OS data types  |  AWS DMS data types  | 
| --- | --- | 
|  INTEGER  |  INT4  | 
|  SMALLINT  |  INT2  | 
|  BIGINT  |  INT8  | 
|  DECIMAL (p,s)  |  NUMERIC (p,s) If a decimal point is set to a comma (,) in the DB2 configuration, configure Replicate to support the DB2 setting.   | 
|  FLOAT  |  REAL8  | 
|  DOUBLE  |  REAL8  | 
|  REAL  |  REAL4  | 
|  DECFLOAT (p)  |  If precision is 16, then REAL8; if precision is 34, then STRING  | 
|  GRAPHIC (n)  |  If n>=127 then WSTRING, for fixed-length graphic strings of double byte chars with a length greater than 0 and less than or equal to 127  | 
|  VARGRAPHIC (n)  |  WSTRING, for varying-length graphic strings with a length greater than 0 and less than or equal to16,352 double byte chars  | 
|  LONG VARGRAPHIC (n)  |  CLOB, for varying-length graphic strings with a length greater than 0 and less than or equal to16,352 double byte chars  | 
|  CHARACTER (n)  |  STRING, for fixed-length strings of double byte chars with a length greater than 0 and less than or equal to 255  | 
|  VARCHAR (n)  |  STRING, for varying-length strings of double byte chars with a length greater than 0 and less than or equal to 32,704  | 
|  LONG VARCHAR (n)  |  CLOB, for varying-length strings of double byte chars with a length greater than 0 and less than or equal to 32,704  | 
|  CHAR (n) FOR BIT DATA  |  BYTES  | 
|  VARCHAR (n) FOR BIT DATA  |  BYTES  | 
|  LONG VARCHAR FOR BIT DATA  |  BYTES  | 
|  DATE  |  DATE  | 
|  TIME  |  TIME  | 
|  TIMESTAMP  |  DATETIME  | 
|  BLOB (n)  |  BLOB Maximum length is 2,147,483,647 bytes  | 
|  CLOB (n)  |  CLOB Maximum length is 2,147,483,647 bytes  | 
|  DBCLOB (n)  |  CLOB Maximum length is 1,073,741,824 double byte chars  | 
|  XML  |  CLOB  | 
|  BINARY  |  BYTES  | 
|  VARBINARY  |  BYTES  | 
|  ROWID  |  BYTES. For more information about working with ROWID, see following.   | 
|  TIMESTAMP WITH TIME ZONE  |  Not supported.  | 

ROWID columns are migrated by default when the target table prep mode for the task is set to DROP\$1AND\$1CREATE (the default). Data validation ignores these columns because the rows are meaningless outside the specific database and table. To turn off migration of these columns, you can do one of the following preparatory steps: 
+ Precreate the target table without these columns. Then, set the target table prep mode of the task to either DO\$1NOTHING or TRUNCATE\$1BEFORE\$1LOAD. You can use AWS Schema Conversion Tool (AWS SCT) to precreate the target table without the columns.
+ Add a table mapping rule to a task that filters out these columns so that they're ignored. For more information, see [Transformation rules and actions](CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.md).

## EBCDIC collations in PostgreSQL for AWS Mainframe Modernization service
<a name="CHAP_Source.DB2zOS.EBCDIC"></a>

AWS Mainframe Modernization program helps you modernize your mainframe applications to AWS managed runtime environments. It provides tools and resources that help you plan and implement your migration and modernization projects. For more information about mainframe modernization and migration, see [Mainframe Modernization with AWS](https://aws.amazon.com/mainframe/).

Some IBM Db2 for z/OS data sets are encoded in the Extended Binary Coded Decimal Interchange (EBCDIC) character set. This is a character set that was developed before ASCII (American Standard Code for Information Interchange) became commonly used. A *code page* maps each character of text to the characters in a character set. A traditional code page contains the mapping information between a code point and a character ID. A *character ID *is an 8-byte character data string. A *code point* is an 8-bit binary number that represents a character. Code points are usually shown as hexadecimal representations of their binary values.

If you currently use either the Micro Focus or BluAge component of the Mainframe Modernization service, you must tell AWS DMS to *shift* (translate) certain code points. You can use AWS DMS task settings to perform the shifts. The following example shows how to use the AWS DMS `CharacterSetSettings` operation to map the shifts in a DMS task setting.

```
"CharacterSetSettings": {
        "CharacterSetSupport": null,
        "CharacterReplacements": [
{"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0178"}
            }
        ]
    }
```

Some EBCDIC collations already exist for PostgreSQL that understand the shifting that's needed. Several different code pages are supported. The sections following provide JSON samples of what you must shift for all the supported code pages. You can simply copy-and-past the necessary JSON that you need in your DMS task.

### Micro Focus specific EBCDIC collations
<a name="CHAP_Source.DB2zOS.EBCDIC.MicroFocus"></a>

For Micro Focus, shift a subset of characters as needed for the following collations.

```
 da-DK-cp1142m-x-icu
 de-DE-cp1141m-x-icu
 en-GB-cp1146m-x-icu
 en-US-cp1140m-x-icu
 es-ES-cp1145m-x-icu
 fi-FI-cp1143m-x-icu
 fr-FR-cp1147m-x-icu
 it-IT-cp1144m-x-icu
 nl-BE-cp1148m-x-icu
```

**Example Micro Focus data shifts per collation:**  
**en\$1us\$1cp1140m**  
Code Shift:  

```
0000    0180
00A6    0160
00B8    0161
00BC    017D
00BD    017E
00BE    0152
00A8    0153
00B4    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1141m**  
Code Shift:  

```
0000    0180
00B8    0160
00BC    0161
00BD    017D
00BE    017E
00A8    0152
00B4    0153
00A6    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1142m**  
Code Shift:  

```
0000    0180
00A6    0160
00B8    0161
00BC    017D
00BD    017E
00BE    0152
00A8    0153
00B4    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1143m**  
Code Shift:  

```
0000    0180
00B8    0160
00BC    0161
00BD    017D
00BE    017E
00A8    0152
00B4    0153
00A6    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1144m**  
Code Shift:  

```
0000    0180
00B8    0160
00BC    0161
00BD    017D
00BE    017E
00A8    0152
00B4    0153
00A6    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1145m**  
Code Shift:  

```
0000    0180
00A6    0160
00B8    0161
00A8    017D
00BC    017E
00BD    0152
00BE    0153
00B4    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1146m**  
Code Shift:  

```
0000    0180
00A6    0160
00B8    0161
00BC    017D
00BD    017E
00BE    0152
00A8    0153
00B4    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1147m**  
Code Shift:  

```
0000    0180
00B8    0160
00A8    0161
00BC    017D
00BD    017E
00BE    0152
00B4    0153
00A6    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0178"}
```
**en\$1us\$1cp1148m**  
Code Shift:  

```
0000    0180
00A6    0160
00B8    0161
00BC    017D
00BD    017E
00BE    0152
00A8    0153
00B4    0178
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0000","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "00A6","TargetCharacterCodePoint": "0160"}
,{"SourceCharacterCodePoint": "00B8","TargetCharacterCodePoint": "0161"}
,{"SourceCharacterCodePoint": "00BC","TargetCharacterCodePoint": "017D"}
,{"SourceCharacterCodePoint": "00BD","TargetCharacterCodePoint": "017E"}
,{"SourceCharacterCodePoint": "00BE","TargetCharacterCodePoint": "0152"}
,{"SourceCharacterCodePoint": "00A8","TargetCharacterCodePoint": "0153"}
,{"SourceCharacterCodePoint": "00B4","TargetCharacterCodePoint": "0178"}
```

### BluAge specific EBCDIC collations
<a name="CHAP_Source.DB2zOS.EBCDIC.BluAge"></a>

For BluAge, shift all of the following *low values* and *high values* as needed. These collations should only be used to support the Mainframe Migration BluAge service.

```
da-DK-cp1142b-x-icu
 da-DK-cp277b-x-icu
 de-DE-cp1141b-x-icu
 de-DE-cp273b-x-icu
 en-GB-cp1146b-x-icu
 en-GB-cp285b-x-icu
 en-US-cp037b-x-icu
 en-US-cp1140b-x-icu
 es-ES-cp1145b-x-icu
 es-ES-cp284b-x-icu
 fi-FI-cp1143b-x-icu
 fi-FI-cp278b-x-icu 
 fr-FR-cp1147b-x-icu
 fr-FR-cp297b-x-icu
 it-IT-cp1144b-x-icu
 it-IT-cp280b-x-icu
 nl-BE-cp1148b-x-icu
 nl-BE-cp500b-x-icu
```

**Example BluAge Data Shifts:**  
**da-DK-cp277b** and **da-DK-cp1142b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**de-DE-273b** and **de-DE-1141b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**en-GB-285b** and **en-GB-1146b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
{"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**en-us-037b** and **en-us-1140b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
{"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**es-ES-284b** and **es-ES-1145b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**fi\$1FI-278b** and **fi-FI-1143b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**fr-FR-297b** and **fr-FR-1147b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
{"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**it-IT-280b** and **it-IT-1144b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```
**nl-BE-500b** and **nl-BE-1148b**  
Code Shift:  

```
0180    0180
0001    0181
0002    0182
0003    0183
009C    0184
0009    0185
0086    0186
007F    0187
0097    0188
008D    0189
008E    018A
000B    018B
000C    018C
000D    018D
000E    018E
000F    018F
0010    0190
0011    0191
0012    0192
0013    0193
009D    0194
0085    0195
0008    0196
0087    0197
0018    0198
0019    0199
0092    019A
008F    019B
001C    019C
001D    019D
001E    019E
001F    019F
0080    01A0
0081    01A1
0082    01A2
0083    01A3
0084    01A4
000A    01A5
0017    01A6
001B    01A7
0088    01A8
0089    01A9
008A    01AA
008B    01AB
008C    01AC
0005    01AD
0006    01AE
0007    01AF
0090    01B0
0091    01B1
0016    01B2
0093    01B3
0094    01B4
0095    01B5
0096    01B6
0004    01B7
0098    01B8
0099    01B9
009A    01BA
009B    01BB
0014    01BC
0015    01BD
009E    01BE
001A    01BF
009F    027F
```
Corresponding input mapping for an AWS DMS task:  

```
 {"SourceCharacterCodePoint": "0180","TargetCharacterCodePoint": "0180"}
,{"SourceCharacterCodePoint": "0001","TargetCharacterCodePoint": "0181"}
,{"SourceCharacterCodePoint": "0002","TargetCharacterCodePoint": "0182"}
,{"SourceCharacterCodePoint": "0003","TargetCharacterCodePoint": "0183"}
,{"SourceCharacterCodePoint": "009C","TargetCharacterCodePoint": "0184"}
,{"SourceCharacterCodePoint": "0009","TargetCharacterCodePoint": "0185"}
,{"SourceCharacterCodePoint": "0086","TargetCharacterCodePoint": "0186"}
,{"SourceCharacterCodePoint": "007F","TargetCharacterCodePoint": "0187"}
,{"SourceCharacterCodePoint": "0097","TargetCharacterCodePoint": "0188"}
,{"SourceCharacterCodePoint": "008D","TargetCharacterCodePoint": "0189"}
,{"SourceCharacterCodePoint": "008E","TargetCharacterCodePoint": "018A"}
,{"SourceCharacterCodePoint": "000B","TargetCharacterCodePoint": "018B"}
,{"SourceCharacterCodePoint": "000C","TargetCharacterCodePoint": "018C"}
,{"SourceCharacterCodePoint": "000D","TargetCharacterCodePoint": "018D"}
,{"SourceCharacterCodePoint": "000E","TargetCharacterCodePoint": "018E"}
,{"SourceCharacterCodePoint": "000F","TargetCharacterCodePoint": "018F"}
,{"SourceCharacterCodePoint": "0010","TargetCharacterCodePoint": "0190"}
,{"SourceCharacterCodePoint": "0011","TargetCharacterCodePoint": "0191"}
,{"SourceCharacterCodePoint": "0012","TargetCharacterCodePoint": "0192"}
,{"SourceCharacterCodePoint": "0013","TargetCharacterCodePoint": "0193"}
,{"SourceCharacterCodePoint": "009D","TargetCharacterCodePoint": "0194"}
,{"SourceCharacterCodePoint": "0085","TargetCharacterCodePoint": "0195"}
,{"SourceCharacterCodePoint": "0008","TargetCharacterCodePoint": "0196"}
,{"SourceCharacterCodePoint": "0087","TargetCharacterCodePoint": "0197"}
,{"SourceCharacterCodePoint": "0018","TargetCharacterCodePoint": "0198"}
,{"SourceCharacterCodePoint": "0019","TargetCharacterCodePoint": "0199"}
,{"SourceCharacterCodePoint": "0092","TargetCharacterCodePoint": "019A"}
,{"SourceCharacterCodePoint": "008F","TargetCharacterCodePoint": "019B"}
,{"SourceCharacterCodePoint": "001C","TargetCharacterCodePoint": "019C"}
,{"SourceCharacterCodePoint": "001D","TargetCharacterCodePoint": "019D"}
,{"SourceCharacterCodePoint": "001E","TargetCharacterCodePoint": "019E"}
,{"SourceCharacterCodePoint": "001F","TargetCharacterCodePoint": "019F"}
,{"SourceCharacterCodePoint": "0080","TargetCharacterCodePoint": "01A0"}
,{"SourceCharacterCodePoint": "0081","TargetCharacterCodePoint": "01A1"}
,{"SourceCharacterCodePoint": "0082","TargetCharacterCodePoint": "01A2"}
,{"SourceCharacterCodePoint": "0083","TargetCharacterCodePoint": "01A3"}
,{"SourceCharacterCodePoint": "0084","TargetCharacterCodePoint": "01A4"}
,{"SourceCharacterCodePoint": "000A","TargetCharacterCodePoint": "01A5"}
,{"SourceCharacterCodePoint": "0017","TargetCharacterCodePoint": "01A6"}
,{"SourceCharacterCodePoint": "001B","TargetCharacterCodePoint": "01A7"}
,{"SourceCharacterCodePoint": "0088","TargetCharacterCodePoint": "01A8"}
,{"SourceCharacterCodePoint": "0089","TargetCharacterCodePoint": "01A9"}
,{"SourceCharacterCodePoint": "008A","TargetCharacterCodePoint": "01AA"}
,{"SourceCharacterCodePoint": "008B","TargetCharacterCodePoint": "01AB"}
,{"SourceCharacterCodePoint": "008C","TargetCharacterCodePoint": "01AC"}
,{"SourceCharacterCodePoint": "0005","TargetCharacterCodePoint": "01AD"}
,{"SourceCharacterCodePoint": "0006","TargetCharacterCodePoint": "01AE"}
,{"SourceCharacterCodePoint": "0007","TargetCharacterCodePoint": "01AF"}
,{"SourceCharacterCodePoint": "0090","TargetCharacterCodePoint": "01B0"}
,{"SourceCharacterCodePoint": "0091","TargetCharacterCodePoint": "01B1"}
,{"SourceCharacterCodePoint": "0016","TargetCharacterCodePoint": "01B2"}
,{"SourceCharacterCodePoint": "0093","TargetCharacterCodePoint": "01B3"}
,{"SourceCharacterCodePoint": "0094","TargetCharacterCodePoint": "01B4"}
,{"SourceCharacterCodePoint": "0095","TargetCharacterCodePoint": "01B5"}
,{"SourceCharacterCodePoint": "0096","TargetCharacterCodePoint": "01B6"}
,{"SourceCharacterCodePoint": "0004","TargetCharacterCodePoint": "01B7"}
,{"SourceCharacterCodePoint": "0098","TargetCharacterCodePoint": "01B8"}
,{"SourceCharacterCodePoint": "0099","TargetCharacterCodePoint": "01B9"}
,{"SourceCharacterCodePoint": "009A","TargetCharacterCodePoint": "01BA"}
,{"SourceCharacterCodePoint": "009B","TargetCharacterCodePoint": "01BB"}
,{"SourceCharacterCodePoint": "0014","TargetCharacterCodePoint": "01BC"}
,{"SourceCharacterCodePoint": "0015","TargetCharacterCodePoint": "01BD"}
,{"SourceCharacterCodePoint": "009E","TargetCharacterCodePoint": "01BE"}
,{"SourceCharacterCodePoint": "001A","TargetCharacterCodePoint": "01BF"}
,{"SourceCharacterCodePoint": "009F","TargetCharacterCodePoint": "027F"}
```