[ aws . s3tables ]

get-table-migration

Description

Gets the details of a migration, including its state, the reason it failed, its statistics and the location of its reports. Identify the migration by its migrationId , or specify the tableARN of a destination table to get that table’s most recent migration.

Permissions

• You must have the s3tables:GetTableMigration permission to use this operation.

See also: AWS API Documentation

Synopsis

  get-table-migration
[--migration-id <value>]
[--table-arn <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]
[--cli-error-format <value>]
[--help]

Options

--migration-id (string)

The ID of the migration to get. Specify either migrationId or tableARN , not both.

Constraints:

• min: 1
• max: 64
• pattern: [a-z0-9-]+

--table-arn (string)

The Amazon Resource Name (ARN) of the destination table. Gets the table’s most recent migration. Specify either tableARN or migrationId , not both.

Constraints:

• min: 1
• max: 2048
• pattern: (arn:aws[-a-z0-9]*:[a-z0-9]+:[-a-z0-9]*:[0-9]{12}:bucket/[a-z0-9_-]{3,63}/table/[a-zA-Z0-9-_]{1,255})

--cli-input-json | --cli-input-yaml (string) Reads arguments from the JSON string provided. The JSON string follows the format provided by --generate-cli-skeleton. If other arguments are provided on the command line, those values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally. This may not be specified along with --cli-input-yaml.

--generate-cli-skeleton (string) Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value input, prints a sample input JSON that can be used as an argument for --cli-input-json. Similarly, if provided yaml-input it will print a sample input YAML that can be used with --cli-input-yaml. If provided with the value output, it validates the command inputs and returns a sample output JSON for that command. The generated JSON skeleton is not stable between versions of the AWS CLI and there are no backwards compatibility guarantees in the JSON skeleton generated.

Global Options

--debug (boolean)

Turn on debug logging.

--endpoint-url (string)

Override command’s default URL with the given URL.

--no-verify-ssl (boolean)

By default, the AWS CLI uses SSL when communicating with AWS services. For each SSL connection, the AWS CLI will verify SSL certificates. This option overrides the default behavior of verifying SSL certificates.

--no-paginate (boolean)

Disable automatic pagination. If automatic pagination is disabled, the AWS CLI will only make one call, for the first page of results.

--output (string)

The formatting style for command output.

• json
• text
• table
• yaml
• yaml-stream
• off

--query (string)

A JMESPath query to use in filtering the response data.

--profile (string)

Use a specific profile from your credential file.

--region (string)

The region to use. Overrides config/env settings.

--version (string)

Display the version of this tool.

--color (string)

Turn on/off color output.

• on
• off
• auto

--no-sign-request (boolean)

Do not sign requests. Credentials will not be loaded if this argument is provided.

--ca-bundle (string)

The CA certificate bundle to use when verifying SSL certificates. Overrides config/env settings.

--cli-read-timeout (int)

The maximum socket read time in seconds. If the value is set to 0, the socket read will be blocking and not timeout. The default value is 60 seconds.

--cli-connect-timeout (int)

The maximum socket connect time in seconds. If the value is set to 0, the socket connect will be blocking and not timeout. The default value is 60 seconds.

--cli-binary-format (string)

The formatting style to be used for binary blobs. The default format is base64. The base64 format expects binary blobs to be provided as a base64 encoded string. The raw-in-base64-out format preserves compatibility with AWS CLI V1 behavior and binary values must be passed literally. When providing contents from a file that map to a binary blob fileb:// will always be treated as binary and use the file contents directly regardless of the cli-binary-format setting. When using file:// the file contents will need to properly formatted for the configured cli-binary-format.

• base64
• raw-in-base64-out

--no-cli-pager (boolean)

Disable cli pager for output.

--cli-auto-prompt (boolean)

Automatically prompt for CLI input parameters.

--no-cli-auto-prompt (boolean)

Disable automatically prompt for CLI input parameters.

--cli-error-format (string)

The formatting style for error output. By default, errors are displayed in enhanced format.

• legacy
• json
• yaml
• text
• table
• enhanced

--help (boolean)

Display help for the command/subcommand.

Output

migrationId -> (string)

The ID of the migration.

Constraints:

• min: 1
• max: 64
• pattern: [a-z0-9-]+

tableBucketARN -> (string)

The Amazon Resource Name (ARN) of the table bucket that contains the destination table.

Constraints:

• pattern: (arn:aws[-a-z0-9]*:[a-z0-9]+:[-a-z0-9]*:[0-9]{12}:bucket/[a-z0-9_-]{3,63})

namespace -> (string)

The namespace of the destination table.

Constraints:

• min: 1
• max: 255
• pattern: [0-9a-z_]*

tableARN -> (string)

The Amazon Resource Name (ARN) of the destination table, once the migration creates it.

Constraints:

• min: 1
• max: 2048
• pattern: (arn:aws[-a-z0-9]*:[a-z0-9]+:[-a-z0-9]*:[0-9]{12}:bucket/[a-z0-9_-]{3,63}/table/[a-zA-Z0-9-_]{1,255})

name -> (string)

The name of the destination table.

Constraints:

• min: 1
• max: 255
• pattern: [0-9a-z_]*

type -> (string)

The type of the migration. Valid values:

• start - A migration that StartTableMigration started.
• sync - A migration that SyncTableMigration started.

Possible values:

• start
• sync

state -> (string)

The state of the migration. Valid values:

• new - S3 Tables accepted the request.
• planning - The migration is checking the source table, its objects and the destination for compatibility, and writing the compatibility reports.
• pendingReview - Planning finished, and the migration waits for you to continue or cancel it with UpdateTableMigration .
• queued - The migration is waiting to start migrating data.
• migrating - The migration is running the plan from the planning state: it copies data and commits it to the destination table. A migration reaches this state only if hasValidPlan is true .
• succeeded - Every planned object migrated.
• failed - The migration failed. Check failureReason for the cause.
• cancelling - You cancelled the migration, and it is finishing the work in progress.
• cancelled - The migration is cancelled.

A migration ends in the succeeded , failed or cancelled state. You can cancel it until it ends.

Possible values:

• new
• planning
• pendingReview
• queued
• migrating
• succeeded
• failed
• cancelling
• cancelled

roleARN -> (string)

The Amazon Resource Name (ARN) of the IAM role that the migration uses.

Constraints:

• min: 20
• max: 2048
• pattern: arn:aws[-a-z0-9]*:iam::[0-9]{12}:role/.+

reviewMigrationPlan -> (boolean)

A Boolean value that specifies whether the migration waits in the pendingReview state, so that you can review the compatibility reports before it migrates any data.

hasValidPlan -> (boolean)

A Boolean value that specifies whether planning produced a plan that the migration can run. If planning hasn’t finished, this value isn’t returned.

createdAt -> (timestamp)

The date and time that the migration was created.

stateUpdatedAt -> (timestamp)

The date and time that the migration’s state last changed.

failureReason -> (structure)

The reason that the migration failed. The code names the cause, such as a migration role that S3 Tables can’t assume or an Amazon Web Services Glue table that doesn’t exist, or the report that lists the findings. This value is returned only when the migration is in the failed state.

code -> (string) [required]

The failure code. It names the cause or the report that lists the findings. Valid values:

• rolePermissions - S3 Tables couldn’t assume the migration role, or the role doesn’t have a permission that the migration needs. detail is the role ARN.
• sourcePermissions - The migration role couldn’t read the Amazon Web Services Glue table or the source objects. detail names the table (database.table ) or the source bucket.
• destinationPermissions - The migration role couldn’t write to the table bucket or the report bucket. detail names the bucket.
• resourceNotFound - A resource that you specified doesn’t exist, such as the Amazon Web Services Glue table or the source bucket. detail names it.
• sourceNotSupported - The source table can’t be migrated, for example because it is empty, has too many objects, or has columns or partitions that can’t be migrated. The table compatibility report lists the findings.
• tableCompatibilityReport - The migration couldn’t create the destination table, for example because a table with that name already exists or the KMS key can’t be used. For a sync, the destination table changed after the previous migration. The table compatibility report lists the finding.
• objectCompatibilityReport - Some source objects can’t be migrated, for example because they are archived, or because they were overwritten after an earlier migration migrated them. The object compatibility report lists them.
• objectMigrationReport - Some source objects failed to migrate. The object migration report lists them and the cause for each.
• reportBucketNotSupported - The report bucket can’t receive the migration’s reports, for example because it is in a different Region than the table bucket.
• internalError - S3 Tables encountered an internal error. Retry the migration, and contact Amazon Web Services Support if the error persists.

Possible values:

• rolePermissions
• sourcePermissions
• destinationPermissions
• resourceNotFound
• sourceNotSupported
• objectMigrationReport
• objectCompatibilityReport
• tableCompatibilityReport
• internalError
• reportBucketNotSupported

detail -> (string)

The resource that the failure applies to, such as the role ARN, the Amazon Web Services Glue table (database.table ) or a bucket. For sourceNotSupported , the reason that the source isn’t supported, such as EMPTY_TABLE . This value isn’t returned for the report codes or for internalError .

Constraints:

• min: 1
• max: 1024

planningStats -> (structure)

Statistics about the source and the partitions in the migration. This value is returned after planning finishes.

objectsScanned -> (long)

The number of source objects that the migration found under the source table’s location, whether or not it migrates them. The object compatibility report has one row for each.

objectsPlanned -> (long)

The number of source objects in the migration. For a start, these are the data files in the Amazon Web Services Glue table’s partitions, or under the table’s location if the table isn’t partitioned. For a sync, these are all the data files of the table after the sync, including files that earlier migrations migrated.

partitionCount -> (long)

The number of partitions in the destination table after this migration, or 0 if the source table isn’t partitioned. This value is returned when planning produces a valid plan.

migrationStats -> (structure)

The number of objects that migrated, failed or are pending. These counts are set when planning finishes and updated when the migration ends.

objectsMigrated -> (long)

The number of source objects whose data the migration wrote to the destination table. Migrated data can be queried after the migration commits it. If the migration fails or is cancelled, some migrated objects might not be committed.

objectsFailedMigration -> (long)

The number of source objects that didn’t migrate, because they failed or because another object, such as an object in the same partition, failed. The object migration report lists the cause for each object.

objectsPendingMigration -> (long)

The number of source objects that haven’t migrated or failed yet. When a migration fails or is cancelled, these are the objects that it didn’t migrate.

sourceType -> (string)

The type of the migration’s source.

Possible values:

• glueTable

source -> (tagged union structure)

The source table of the migration.

Note

This is a Tagged Union structure. Only one of the following top level keys can be set: glueTable.

glueTable -> (structure)

An Amazon Web Services Glue Data Catalog table to migrate. The table must be an external Apache Hive table whose data is stored as Parquet in a general purpose bucket in the same account and Region as the table bucket. Views, transactional (ACID) tables, tables in an open table format such as Apache Iceberg, and tables registered with Amazon Web Services Lake Formation can’t be migrated. If the table can’t be migrated, the migration fails, and the table compatibility report lists the reason.

catalogId -> (string) [required]

The ID of the Data Catalog that contains the table. It must be your account ID.

Constraints:

• min: 1
• max: 255

databaseName -> (string) [required]

The name of the database that contains the table.

Constraints:

• min: 1
• max: 255

tableName -> (string) [required]

The name of the table.

Constraints:

• min: 1
• max: 255

dataMigration -> (structure)

The maintenance settings and migration policies for the destination table.

maintenanceSettings -> (structure) [required]

The maintenance settings for the destination table. The migration applies them when it creates the table: snapshot management starts then, and compaction starts after you complete the migration. If you don’t specify a setting, the migration uses its default: a target file size of 512 MB, the binpack compaction strategy, at least 3 snapshots, and a maximum snapshot age of 120 hours. Unreferenced file removal follows the table bucket’s configuration after you complete the migration.

icebergCompaction -> (structure)

The compaction settings.

targetFileSizeMB -> (integer)

The target size, from 64 to 512 MB, of the data files that the migration writes, and of the files that compaction writes after you complete the migration. The default is 512.

Constraints:

• min: 64
• max: 512

strategy -> (string)

The compaction strategy. The only valid value is binpack , which is the default.

Possible values:

• auto
• binpack
• sort
• z-order

icebergSnapshotManagement -> (structure)

The snapshot management settings.

minSnapshotsToKeep -> (integer)

The minimum number of snapshots to keep. The value must be 3 or more. The default is 3.

Constraints:

• min: 3

maxSnapshotAgeHours -> (integer)

The maximum age, in hours, of a snapshot before it can be expired. The default is 120.

Constraints:

• min: 1
• max: 2147483647

migrationPolicies -> (structure) [required]

The policies for source data that doesn’t fit the destination schema.

unmatchedSourceFieldsPolicy -> (string)

The migration policy for a source field that has no matching column in the destination schema. Valid values:

• fail - Fails the migration. This is the default.
• drop - Leaves source columns that the destination schema doesn’t have out of the destination table, and doesn’t migrate source objects outside the Amazon Web Services Glue table’s partition locations.

A destination schema that leaves out a partition column or a field of a struct column fails the migration under either value.

Possible values:

• fail
• drop

incompatibleValuesPolicy -> (string)

The migration policy for a source value that the destination column’s type can’t hold. Valid values:

• fail - Fails the migration. This is the default.
• setNull - Writes null for a value that the destination column’s type can’t hold. If a column’s type in a source object is incompatible with the destination column’s type, the column is null for that whole object.

A required column, or a type that migration doesn’t support, fails the migration under either value.

Possible values:

• fail
• setNull

migrationOutput -> (structure)

The bucket and prefixes for the migration’s reports, and the location of each report once it is written.

bucket -> (string) [required]

The general purpose bucket that the migration writes every report to.

Constraints:

• min: 12
• max: 1024
• pattern: arn:aws[-a-z0-9]*:s3[a-z]*:[-a-z0-9]*:([0-9]{12})?:.+

tableCompatibilityReport -> (structure) [required]

The table compatibility report: the sample object that the schema is inferred from, the table-level findings, and one row for each destination column with its type change and how the migration policies apply to it. The migration writes it when planning finishes, before any data migrates. If planning fails, the report might not be written.

prefix -> (string)

The key prefix that the report is written under.

Constraints:

• min: 0
• max: 512

key -> (string)

The key of the report object in the bucket. For the object compatibility and object migration reports, this is the report’s manifest; for the table compatibility report, it is the report itself. This value is returned after the migration writes the report. A migration doesn’t always write every report. For example, a migration that fails during planning doesn’t write the object migration report.

Constraints:

• min: 1
• max: 1024

checksumCRC64NVME -> (string)

The Base64-encoded CRC64NVME checksum of the object that key names. For a report that has a manifest, the manifest lists the checksum of each report file. This value is returned after the migration writes the report.

Constraints:

• min: 12
• max: 12
• pattern: [A-Za-z0-9+/]{11}=

invalidFindings -> (map)

For the compatibility reports, the number of blocking findings in each category. The object compatibility report counts each blocked object once, under its most severe finding. Categories with no blocking findings are left out.

key -> (string)

The category of a compatibility report finding, by what you change to fix it. Valid values:

• access - A permission or a request setting. You don’t need to change the source.
• metadata - The Amazon Web Services Glue table definition, or properties of a source object, such as its size, storage class, encryption or Parquet types.
• data - The contents of a source object, such as a partition value that can’t be read, an object overwritten after an earlier migration, or a corrupt file.
• internal - An internal error in S3 Tables. Retry the migration, and contact Amazon Web Services Support if the error persists.

Possible values:

• access
• metadata
• data
• internal

value -> (long)

objectCompatibilityReport -> (structure) [required]

The object compatibility report: one row for each source object under the source table’s location, with whether the migration includes the object and its most severe finding. The migration writes it when planning finishes. If planning fails, the report might not be written.

prefix -> (string)

The key prefix that the report is written under.

Constraints:

• min: 0
• max: 512

key -> (string)

The key of the report object in the bucket. For the object compatibility and object migration reports, this is the report’s manifest; for the table compatibility report, it is the report itself. This value is returned after the migration writes the report. A migration doesn’t always write every report. For example, a migration that fails during planning doesn’t write the object migration report.

Constraints:

• min: 1
• max: 1024

checksumCRC64NVME -> (string)

The Base64-encoded CRC64NVME checksum of the object that key names. For a report that has a manifest, the manifest lists the checksum of each report file. This value is returned after the migration writes the report.

Constraints:

• min: 12
• max: 12
• pattern: [A-Za-z0-9+/]{11}=

invalidFindings -> (map)

For the compatibility reports, the number of blocking findings in each category. The object compatibility report counts each blocked object once, under its most severe finding. Categories with no blocking findings are left out.

key -> (string)

The category of a compatibility report finding, by what you change to fix it. Valid values:

• access - A permission or a request setting. You don’t need to change the source.
• metadata - The Amazon Web Services Glue table definition, or properties of a source object, such as its size, storage class, encryption or Parquet types.
• data - The contents of a source object, such as a partition value that can’t be read, an object overwritten after an earlier migration, or a corrupt file.
• internal - An internal error in S3 Tables. Retry the migration, and contact Amazon Web Services Support if the error persists.

Possible values:

• access
• metadata
• data
• internal

value -> (long)

objectMigrationReport -> (structure) [required]

The object migration report: one row for each source object that the migration tried to migrate, with whether it migrated, how it was transformed, and why it failed. The migration writes it once, when the migration ends. It isn’t written if planning fails or if a sync has nothing to migrate.

prefix -> (string)

The key prefix that the report is written under.

Constraints:

• min: 0
• max: 512

key -> (string)

The key of the report object in the bucket. For the object compatibility and object migration reports, this is the report’s manifest; for the table compatibility report, it is the report itself. This value is returned after the migration writes the report. A migration doesn’t always write every report. For example, a migration that fails during planning doesn’t write the object migration report.

Constraints:

• min: 1
• max: 1024

checksumCRC64NVME -> (string)

The Base64-encoded CRC64NVME checksum of the object that key names. For a report that has a manifest, the manifest lists the checksum of each report file. This value is returned after the migration writes the report.

Constraints:

• min: 12
• max: 12
• pattern: [A-Za-z0-9+/]{11}=

invalidFindings -> (map)

For the compatibility reports, the number of blocking findings in each category. The object compatibility report counts each blocked object once, under its most severe finding. Categories with no blocking findings are left out.

key -> (string)

The category of a compatibility report finding, by what you change to fix it. Valid values:

• access - A permission or a request setting. You don’t need to change the source.
• metadata - The Amazon Web Services Glue table definition, or properties of a source object, such as its size, storage class, encryption or Parquet types.
• data - The contents of a source object, such as a partition value that can’t be read, an object overwritten after an earlier migration, or a corrupt file.
• internal - An internal error in S3 Tables. Retry the migration, and contact Amazon Web Services Support if the error persists.

Possible values:

• access
• metadata
• data
• internal

value -> (long)