<a id="cli-aws-s3tables-start-table-migration"></a>

# start-table-migration

## Description

Starts migrating an Amazon Web Services Glue Data Catalog table into a new table in a table bucket. The migration runs asynchronously: this operation returns the migration ID and its initial state, and you track progress with `GetTableMigration` .

The namespace must already exist, and a table with the specified name must not. The migration creates the table during planning and commits data to it as it runs. Until you call `CompleteTableMigration` , you can read the table but you can’t write to it. If the migration fails or is cancelled, the table and the data already committed to it remain.

The migration first checks the source and the destination, and writes the table compatibility and object compatibility reports to the bucket that you specify in `migrationOutput` . If you set `reviewMigrationPlan` to `true` , the migration waits in the `pendingReview` state so that you can review the reports, and continues only when you call `UpdateTableMigration` with `requestedState` set to `queued` . If you don’t continue or cancel the migration within 30 days, the migration fails. Otherwise it continues to migrate the data, and writes the object migration report when the migration ends. If planning fails, the migration writes no object migration report.

> Permissions
* You must have the `s3tables:StartTableMigration` permission to use this operation.
* You must have the `iam:PassRole` permission on the role that you specify in `roleARN` , with a condition that allows passing it to `migration.s3tables.amazonaws.com` . The associated resource is the table bucket.
* If `destinationTable` specifies SSE-KMS encryption, you must have the `kms:DescribeKey` and `kms:GenerateDataKey` permissions on the KMS key. You must also grant the S3 Tables maintenance principal access to the key. For more information, see [Permissions requirements for S3 Tables SSE-KMS encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-kms-permissions.html) .

See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/s3tables-2018-05-10/StartTableMigration)

`start-table-migration` uses document type values. Document types follow the JSON data model where valid values are: strings, numbers, booleans, null, arrays, and objects. For command input, options and nested parameters that are labeled with the type `document` must be provided as JSON. Shorthand syntax does not support document types.

## Synopsis

```
  start-table-migration
--role-arn <value>
--destination-table <value>
--source-type <value>
--source <value>
[--data-migration <value>]
[--review-migration-plan | --no-review-migration-plan]
--migration-output <value>
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]
[--cli-error-format <value>]
[--help]
```

## Options

`--role-arn` (string) [required]

> The Amazon Resource Name (ARN) of the IAM role that the migration uses to read the source table and its data and to write the destination table and the reports. The role must be in your account, and its trust policy must allow `migration.s3tables.amazonaws.com` to assume it. You can limit the trust policy to the destination table bucket with the `aws:SourceArn` and `aws:SourceAccount` condition keys.

> Constraints:

> * min: `20`
> * max: `2048`
> * pattern: `arn:aws[-a-z0-9]*:iam::[0-9]{12}:role/.+`

`--destination-table` (structure) [required]

> The table to create and migrate the source into.

> tableBucketARN -> (string) [required]

> > The Amazon Resource Name (ARN) of the table bucket to create the table in.

> > Constraints:

> > * pattern: `(arn:aws[-a-z0-9]*:[a-z0-9]+:[-a-z0-9]*:[0-9]{12}:bucket/[a-z0-9_-]{3,63})`

> namespace -> (string) [required]

> > The namespace to create the table in.

> > Constraints:

> > * min: `1`
> > * max: `255`
> > * pattern: `[0-9a-z_]*`

> name -> (string) [required]

> > The name of the table to create.

> > Constraints:

> > * min: `1`
> > * max: `255`
> > * pattern: `[0-9a-z_]*`

> encryptionConfiguration -> (structure)

> > The encryption configuration for the table. If you don’t specify one, the table uses the default encryption configuration of its table bucket. If you specify an encryption configuration, the migration role must have the `s3tables:PutTableEncryption` permission. If you choose SSE-KMS, the migration role must have the `kms:DescribeKey` , `kms:GenerateDataKey` and `kms:Decrypt` permissions on the KMS key, and the S3 Tables maintenance principal must have access to it. For more information, see [Permissions requirements for S3 Tables SSE-KMS encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-kms-permissions.html) .

> > sseAlgorithm -> (string) [required]

> > > The server-side encryption algorithm to use. Valid values are `AES256` for S3-managed encryption keys, or `aws:kms` for Amazon Web Services KMS-managed encryption keys. If you choose SSE-KMS encryption you must grant the S3 Tables maintenance principal access to your KMS key. For more information, see [Permissions requirements for S3 Tables SSE-KMS encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-kms-permissions.html) .

> > > Possible values:

> > > * `AES256`
> > > * `aws:kms`

> > kmsKeyArn -> (string)

> > > The Amazon Resource Name (ARN) of the KMS key to use for encryption. This field is required only when `sseAlgorithm` is set to `aws:kms` .

> > > Constraints:

> > > * min: `1`
> > > * max: `2048`
> > > * pattern: `(arn:aws[-a-z0-9]*:kms:[-a-z0-9]*:[0-9]{12}:key/.+)`

> storageClassConfiguration -> (structure)

> > The storage class configuration for the table. If you don’t specify one, the table inherits the storage class configuration of its table bucket. If you specify a storage class configuration, the migration role must have the `s3tables:PutTableStorageClass` permission.

> > storageClass -> (string) [required]

> > > The storage class for the table or table bucket. Valid values include storage classes optimized for different access patterns and cost profiles.

> > > Possible values:

> > > * `STANDARD`
> > > * `INTELLIGENT_TIERING`

> tags -> (map)

> > The tags to add to the table. A tag is a key-value pair that you apply to your resources. For more information, see [Tagging for cost allocation or attribute-based access control (ABAC)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/tagging.html) . If you specify tags, the migration role must have the `s3tables:TagResource` permission.

> > key -> (string)

> > > Constraints:

> > > * min: `1`
> > > * max: `128`

> > value -> (string)

> > > Constraints:

> > > * min: `0`
> > > * max: `256`

> properties -> (structure)

> > The properties of the destination table, such as its Iceberg format version and the format of its data files.

> > writeFormat -> (string)

> > > The file format of the data files. The only supported format is `parquet` , the default.

> > > Possible values:

> > > * `parquet`

> > icebergFormatVersion -> (integer)

> > > The Iceberg format version of the destination table. Valid values are `2` and `3` . The default is `2` .

> > compressionCodec -> (string)

> > > The compression codec of the data files. The default is `zstd` .

> > > Possible values:

> > > * `zstd`
> > > * `snappy`
> > > * `gzip`
> > > * `uncompressed`

> > dictionaryEncoding -> (boolean)

> > > Specifies whether the data files use dictionary encoding. The default is `true` .

> schema -> (structure)

> > The schema of the table to create. If you don’t specify a schema, the migration infers it from the source table, with every column optional. A schema that you specify must include every partition column of the source table. It can leave out other columns only if `unmatchedSourceFieldsPolicy` is `drop` . A struct column must have the same fields as the source column. Column default values aren’t supported. If you mark a column as required, every source object must have a value for it: an object with a null value in a required column fails to migrate. The destination table is partitioned in the same way as the source table. To specify fields in `icebergSortOrder` , you must specify a schema.

> > type -> (string) [required]

> > > The type of the top-level schema, which is always a `struct` type as defined in the [Apache Iceberg specification](https://iceberg.apache.org/spec/#schemas-and-data-types) . This value must be `struct` .

> > > Possible values:

> > > * `struct`

> > fields -> (list) [required]

> > > The schema fields for the table. Each field defines a column in the table, including its name, type, and whether it is required.

> > > (structure)

> > > > Contains details about a schema field in the V2 format. This field format supports nested and complex data types such as `struct` , `list` , and `map` , in addition to primitive types.

> > > > id -> (integer) [required]

> > > > > The unique identifier for the schema field. Field IDs are used by Apache Iceberg to track schema evolution and maintain compatibility across schema changes.

> > > > name -> (string) [required]

> > > > > The name of the field.

> > > > type -> (document) [required]

> > > > > The data type of the field. This can be a primitive type string such as `boolean` , `int` , `long` , `float` , `double` , `string` , `binary` , `date` , `timestamp` , or `timestamptz` , or a complex type represented as a JSON object for nested types such as `struct` , `list` , or `map` . For more information, see the [Apache Iceberg schemas and data types documentation](https://iceberg.apache.org/spec/#schemas-and-data-types) .

> > > > required -> (boolean) [required]

> > > > > A Boolean value that specifies whether values are required for each row in this field. If this is `true` , the field does not allow null values.

> > > > doc -> (string)

> > > > > An optional description of the field.

> > schemaId -> (integer)

> > > An optional unique identifier for the schema. Schema IDs are used by Apache Iceberg to track schema evolution.

> > identifierFieldIds -> (list)

> > > A list of field IDs that are used as the identifier fields for the table. Identifier fields uniquely identify a row in the table.

> > > (integer)

> icebergSortOrder -> (structure)

> > The sort order of the rows within each data file that the migration writes. A sort order can have up to five fields, each naming a column of the schema that you specify by its field ID, so to use a sort order you must specify a schema. `StartTableMigration` rejects a field that names no column of the schema, or whose transform doesn’t fit the column’s type.

> > orderId -> (integer) [required]

> > > The unique identifier for this sort order. If not specified, defaults to `1` . The order ID is used by Apache Iceberg to track sort order evolution.

> > fields -> (list) [required]

> > > The list of sort fields that define how data is sorted within files. Each field specifies a source field, sort direction, and null ordering. This field is required if `writeOrder` is provided.

> > > (structure)

> > > > Defines a single sort field in an Iceberg sort order specification.

> > > > sourceId -> (integer) [required]

> > > > > The ID of the source schema field to sort by. This must reference a valid field ID from the table schema.

> > > > transform -> (string) [required]

> > > > > The transform to apply to the source field before sorting. Use `identity` to sort by the field value directly, or specify other transforms as needed.

> > > > direction -> (string) [required]

> > > > > The sort direction. Valid values are `asc` for ascending order or `desc` for descending order.

> > > > > Possible values:

> > > > > * `asc`
> > > > > * `desc`

> > > > nullOrder -> (string) [required]

> > > > > Specifies how null values are ordered. Valid values are `nulls-first` to place nulls before non-null values, or `nulls-last` to place nulls after non-null values.

> > > > > Possible values:

> > > > > * `nulls-first`
> > > > > * `nulls-last`

JSON Syntax:

```json
{
  "tableBucketARN": "string",
  "namespace": "string",
  "name": "string",
  "encryptionConfiguration": {
    "sseAlgorithm": "AES256"|"aws:kms",
    "kmsKeyArn": "string"
  },
  "storageClassConfiguration": {
    "storageClass": "STANDARD"|"INTELLIGENT_TIERING"
  },
  "tags": {"string": "string"
    ...},
  "properties": {
    "writeFormat": "parquet",
    "icebergFormatVersion": integer,
    "compressionCodec": "zstd"|"snappy"|"gzip"|"uncompressed",
    "dictionaryEncoding": true|false
  },
  "schema": {
    "type": "struct",
    "fields": [
      {
        "id": integer,
        "name": "string",
        "type": {...},
        "required": true|false,
        "doc": "string"
      }
      ...
    ],
    "schemaId": integer,
    "identifierFieldIds": [integer, ...]
  },
  "icebergSortOrder": {
    "orderId": integer,
    "fields": [
      {
        "sourceId": integer,
        "transform": "string",
        "direction": "asc"|"desc",
        "nullOrder": "nulls-first"|"nulls-last"
      }
      ...
    ]
  }
}
```

`--source-type` (string) [required]

> The type of the source. It must match the member that you set in `source` .

> Possible values:

> * `glueTable`

`--source` (tagged union structure) [required]

> The source table to migrate.

> #### NOTE
> This is a Tagged Union structure. Only one of the following top level keys can be set: `glueTable`.

> glueTable -> (structure)

> > An Amazon Web Services Glue Data Catalog table to migrate. The table must be an external Apache Hive table whose data is stored as Parquet in a general purpose bucket in the same account and Region as the table bucket. Views, transactional (ACID) tables, tables in an open table format such as Apache Iceberg, and tables registered with Amazon Web Services Lake Formation can’t be migrated. If the table can’t be migrated, the migration fails, and the table compatibility report lists the reason.

> > catalogId -> (string) [required]

> > > The ID of the Data Catalog that contains the table. It must be your account ID.

> > > Constraints:

> > > * min: `1`
> > > * max: `255`

> > databaseName -> (string) [required]

> > > The name of the database that contains the table.

> > > Constraints:

> > > * min: `1`
> > > * max: `255`

> > tableName -> (string) [required]

> > > The name of the table.

> > > Constraints:

> > > * min: `1`
> > > * max: `255`

Shorthand Syntax:

```
glueTable={catalogId=string,databaseName=string,tableName=string}
```

JSON Syntax:

```json
{
  "glueTable": {
    "catalogId": "string",
    "databaseName": "string",
    "tableName": "string"
  }
}
```

`--data-migration` (structure)

> The maintenance settings and migration policies for the destination table.

> maintenanceSettings -> (structure)

> > The maintenance settings for the destination table. The migration applies them when it creates the table: snapshot management starts then, and compaction starts after you complete the migration. If you don’t specify a setting, the migration uses its default: a target file size of 512 MB, the binpack compaction strategy, at least 3 snapshots, and a maximum snapshot age of 120 hours. Unreferenced file removal follows the table bucket’s configuration after you complete the migration.

> > icebergCompaction -> (structure)

> > > The compaction settings.

> > > targetFileSizeMB -> (integer)

> > > > The target size, from 64 to 512 MB, of the data files that the migration writes, and of the files that compaction writes after you complete the migration. The default is 512.

> > > > Constraints:

> > > > * min: `64`
> > > > * max: `512`

> > > strategy -> (string)

> > > > The compaction strategy. The only valid value is `binpack` , which is the default.

> > > > Possible values:

> > > > * `auto`
> > > > * `binpack`
> > > > * `sort`
> > > > * `z-order`

> > icebergSnapshotManagement -> (structure)

> > > The snapshot management settings.

> > > minSnapshotsToKeep -> (integer)

> > > > The minimum number of snapshots to keep. The value must be 3 or more. The default is 3.

> > > > Constraints:

> > > > * min: `3`

> > > maxSnapshotAgeHours -> (integer)

> > > > The maximum age, in hours, of a snapshot before it can be expired. The default is 120.

> > > > Constraints:

> > > > * min: `1`
> > > > * max: `2147483647`

> migrationPolicies -> (structure)

> > The policies for source data that doesn’t fit the destination schema.

> > unmatchedSourceFieldsPolicy -> (string)

> > > The migration policy for a source field that has no matching column in the destination schema. Valid values:

> > > * `fail` - Fails the migration. This is the default.
> > > * `drop` - Leaves source columns that the destination schema doesn’t have out of the destination table, and doesn’t migrate source objects outside the Amazon Web Services Glue table’s partition locations.

> > > A destination schema that leaves out a partition column or a field of a struct column fails the migration under either value.

> > > Possible values:

> > > * `fail`
> > > * `drop`

> > incompatibleValuesPolicy -> (string)

> > > The migration policy for a source value that the destination column’s type can’t hold. Valid values:

> > > * `fail` - Fails the migration. This is the default.
> > > * `setNull` - Writes null for a value that the destination column’s type can’t hold. If a column’s type in a source object is incompatible with the destination column’s type, the column is null for that whole object.

> > > A required column, or a type that migration doesn’t support, fails the migration under either value.

> > > Possible values:

> > > * `fail`
> > > * `setNull`

Shorthand Syntax:

```
maintenanceSettings={icebergCompaction={targetFileSizeMB=integer,strategy=string},icebergSnapshotManagement={minSnapshotsToKeep=integer,maxSnapshotAgeHours=integer}},migrationPolicies={unmatchedSourceFieldsPolicy=string,incompatibleValuesPolicy=string}
```

JSON Syntax:

```json
{
  "maintenanceSettings": {
    "icebergCompaction": {
      "targetFileSizeMB": integer,
      "strategy": "auto"|"binpack"|"sort"|"z-order"
    },
    "icebergSnapshotManagement": {
      "minSnapshotsToKeep": integer,
      "maxSnapshotAgeHours": integer
    }
  },
  "migrationPolicies": {
    "unmatchedSourceFieldsPolicy": "fail"|"drop",
    "incompatibleValuesPolicy": "fail"|"setNull"
  }
}
```

`--review-migration-plan` | `--no-review-migration-plan` (boolean)

> Specifies whether the migration waits in the `pendingReview` state, so that you can review the compatibility reports before it migrates any data. If `true` , the migration continues when you call `UpdateTableMigration` with `requestedState` set to `queued` . If you don’t continue or cancel the migration within 30 days, the migration fails. The default is `false` .

`--migration-output` (structure) [required]

> The bucket and prefixes that the migration writes its reports to. The bucket must be a general purpose bucket in your account and in the same Region as the table bucket, and the migration role must be able to write objects to it. Otherwise, the migration fails when it writes its first report.

> bucket -> (string) [required]

> > The general purpose bucket that the migration writes every report to.

> > Constraints:

> > * min: `12`
> > * max: `1024`
> > * pattern: `arn:aws[-a-z0-9]*:s3[a-z]*:[-a-z0-9]*:([0-9]{12})?:.+`

> prefixes -> (structure)

> > The key prefix for each report.

> > tableCompatibilityReport -> (string)

> > > The key prefix for the table compatibility report, up to 512 bytes. If not specified, the report’s keys start at the root of the bucket.

> > > Constraints:

> > > * min: `0`
> > > * max: `512`

> > objectCompatibilityReport -> (string)

> > > The key prefix for the object compatibility report, up to 512 bytes. If not specified, the report’s keys start at the root of the bucket.

> > > Constraints:

> > > * min: `0`
> > > * max: `512`

> > objectMigrationReport -> (string)

> > > The key prefix for the object migration report, up to 512 bytes. If not specified, the report’s keys start at the root of the bucket.

> > > Constraints:

> > > * min: `0`
> > > * max: `512`

Shorthand Syntax:

```
bucket=string,prefixes={tableCompatibilityReport=string,objectCompatibilityReport=string,objectMigrationReport=string}
```

JSON Syntax:

```json
{
  "bucket": "string",
  "prefixes": {
    "tableCompatibilityReport": "string",
    "objectCompatibilityReport": "string",
    "objectMigrationReport": "string"
  }
}
```

`--cli-input-json` | `--cli-input-yaml` (string)
Reads arguments from the JSON string provided. The JSON string follows the format provided by `--generate-cli-skeleton`. If other arguments are provided on the command line, those values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally. This may not be specified along with `--cli-input-yaml`.

`--generate-cli-skeleton` (string)
Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value `input`, prints a sample input JSON that can be used as an argument for `--cli-input-json`. Similarly, if provided `yaml-input` it will print a sample input YAML that can be used with `--cli-input-yaml`. If provided with the value `output`, it validates the command inputs and returns a sample output JSON for that command. The generated JSON skeleton is not stable between versions of the AWS CLI and there are no backwards compatibility guarantees in the JSON skeleton generated.

## Global Options

`--debug` (boolean)

Turn on debug logging.

`--endpoint-url` (string)

Override command’s default URL with the given URL.

`--no-verify-ssl` (boolean)

By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.

`--no-paginate` (boolean)

Disable automatic pagination. If automatic pagination is disabled, the AWS CLI will only make one call, for the first page of results.

`--output` (string)

The formatting style for command output.

* json
* text
* table
* yaml
* yaml-stream
* off

`--query` (string)

A JMESPath query to use in filtering the response data.

`--profile` (string)

Use a specific profile from your credential file.

`--region` (string)

The region to use. Overrides config/env settings.

`--version` (string)

Display the version of this tool.

`--color` (string)

Turn on/off color output.

* on
* off
* auto

`--no-sign-request` (boolean)

Do not sign requests. Credentials will not be loaded if this argument is provided.

`--ca-bundle` (string)

The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.

`--cli-read-timeout` (int)

The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.

`--cli-connect-timeout` (int)

The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.

`--cli-binary-format` (string)

The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string. The raw-in-base64-out format preserves compatibility with AWS CLI V1 behavior and binary values must be passed literally. When providing contents from a file that map to a binary blob `fileb://` will always be treated as binary and use the file contents directly regardless of the `cli-binary-format` setting. When using `file://` the file contents will need to properly formatted for the configured `cli-binary-format`.

* base64
* raw-in-base64-out

`--no-cli-pager` (boolean)

Disable cli pager for output.

`--cli-auto-prompt` (boolean)

Automatically prompt for CLI input parameters.

`--no-cli-auto-prompt` (boolean)

Disable automatically prompt for CLI input parameters.

`--cli-error-format` (string)

The formatting style for error output. By default, errors are displayed in enhanced format.

* legacy
* json
* yaml
* text
* table
* enhanced

`--help` (boolean)

Display help for the command/subcommand.

## Output

migrationId -> (string)

> The ID of the migration.

> Constraints:

> * min: `1`
> * max: `64`
> * pattern: `[a-z0-9-]+`

state -> (string)

> The state of the migration when the request is accepted, such as `planning` .

> Possible values:

> * `new`
> * `planning`
> * `pendingReview`
> * `queued`
> * `migrating`
> * `succeeded`
> * `failed`
> * `cancelling`
> * `cancelled`