Integration APIs in AWS Glue

• Integration structure

• IntegrationPartition structure

• IntegrationError structure

• IntegrationFilter structure

• InboundIntegration structure

• SourceProcessingProperties structure

• TargetProcessingProperties structure

• SourceTableConfig structure

• TargetTableConfig structure

Fields

• SourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN for the source of the integration.

• TargetArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN for the target of the integration.

• Description – UTF-8 string, not more than 1000 bytes long, matching the Custom string pattern #12.

  A description for the integration.

• IntegrationName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  A unique name for the integration.

• IntegrationArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the integration.

• KmsKeyId – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of a KMS key used for encrypting the channel.

• AdditionalEncryptionContext – A map array of key-value pairs.

  Each key is a UTF-8 string.

  Each value is a UTF-8 string.

  An optional set of non-secret key–value pairs that contains additional contextual information for encryption. This can only be provided if KMSKeyId is provided.

• Tags – An array of Tag objects.

  Metadata assigned to the resource consisting of a list of key-value pairs.

• Status – Required: UTF-8 string (valid values: CREATING | ACTIVE | MODIFYING | FAILED | DELETING | SYNCING | NEEDS_ATTENTION).

  The possible statuses are:

  • CREATING: The integration is being created.

  • ACTIVE: The integration creation succeeds.

  • MODIFYING: The integration is being modified.

  • FAILED: The integration creation fails.

  • DELETING: The integration is deleted.

  • SYNCING: The integration is synchronizing.

  • NEEDS_ATTENTION: The integration needs attention, such as synchronization.

• CreateTime – Required: Timestamp.

  The time that the integration was created, in UTC.

• Errors – An array of IntegrationError objects.

  A list of errors associated with the integration.

• DataFilter – UTF-8 string, not less than 1 or more than 2048 bytes long.

  Selects source tables for the integration using Maxwell filter syntax.

Fields

• FieldName – UTF-8 string, not less than 1 or more than 128 bytes long.

  The field name used to partition data on the target.

• FunctionSpec – UTF-8 string, not less than 1 or more than 128 bytes long.

  Specifies a function used to partition data on the target.

Fields

• ErrorCode – UTF-8 string, not less than 1 or more than 128 bytes long.

  The code associated with this error.

• ErrorMessage – UTF-8 string, not less than 1 or more than 2048 bytes long.

  A message describing the error.

Fields

• Name – UTF-8 string, not less than 1 or more than 128 bytes long.

  The name of the filter.

• Values – An array of UTF-8 strings.

  A list of filter values.

Fields

• SourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the source resource for the integration.

• TargetArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the target resource for the integration.

• IntegrationArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the zero-ETL integration.

• Status – Required: UTF-8 string (valid values: CREATING | ACTIVE | MODIFYING | FAILED | DELETING | SYNCING | NEEDS_ATTENTION).

  The possible statuses are:

  • CREATING: The integration is being created.

  • ACTIVE: The integration creation succeeds.

  • MODIFYING: The integration is being modified.

  • FAILED: The integration creation fails.

  • DELETING: The integration is deleted.

  • SYNCING: The integration is synchronizing.

  • NEEDS_ATTENTION: The integration needs attention, such as synchronization.

• CreateTime – Required: Timestamp.

  The time that the integration was created, in UTC.

• Errors – An array of IntegrationError objects.

  A list of errors associated with the integration.

Fields

• RoleArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The IAM role to access the AWS Glue connection.

Fields

• RoleArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The IAM role to access the AWS Glue database.

• KmsArn – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of the KMS key used for encryption.

• ConnectionName – UTF-8 string, not less than 1 or more than 128 bytes long.

  The AWS Glue network connection to configure the AWS Glue job running in the customer VPC.

• EventBusArn – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of an Eventbridge event bus to receive the integration status notification.

Fields

• Fields – An array of UTF-8 strings.

  A list of fields used for column-level filtering.

• FilterPredicate – UTF-8 string, not less than 1 or more than 128 bytes long.

  A condition clause used for row-level filtering.

• PrimaryKey – An array of UTF-8 strings.

  Unique identifier of a record.

• RecordUpdateField – UTF-8 string, not less than 1 or more than 128 bytes long.

  Incremental pull timestamp-based field.

Fields

• UnnestSpec – UTF-8 string (valid values: TOPLEVEL | FULL | NOUNNEST).

  Specifies how nested objects are flattened to top-level elements. Valid values are: "TOPLEVEL", "FULL", or "NOUNNEST".

• PartitionSpec – An array of IntegrationPartition objects.

  Determines the file layout on the target.

• TargetTableName – UTF-8 string, not less than 1 or more than 128 bytes long.

  The optional name of a target table.

• CreateIntegration action (Python: create_integration)

• ModifyIntegration action (Python: modify_integration)

• DescribeIntegrations action (Python: describe_integrations)

• DeleteIntegration action (Python: delete_integration)

• DescribeInboundIntegrations action (Python: describe_inbound_integrations)

• CreateIntegrationTableProperties action (Python: create_integration_table_properties)

• UpdateIntegrationTableProperties action (Python: update_integration_table_properties)

• GetIntegrationTableProperties action (Python: get_integration_table_properties)

• DeleteIntegrationTableProperties action (Python: delete_integration_table_properties)

• CreateIntegrationResourceProperty action (Python: create_integration_resource_property)

• UpdateIntegrationResourceProperty action (Python: update_integration_resource_property)

• GetIntegrationResourceProperty action (Python: get_integration_resource_property)

• UntagResource action (Python: untag_resource)

• ListTagsForResource action (Python: list_tags_for_resource)

Request

• IntegrationName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  A unique name for an integration in AWS Glue.

• SourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the source resource for the integration.

• TargetArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the target resource for the integration.

• Description – UTF-8 string, not more than 1000 bytes long, matching the Custom string pattern #12.

  A description of the integration.

• DataFilter – UTF-8 string, not less than 1 or more than 2048 bytes long.

  Selects source tables for the integration using Maxwell filter syntax.

• KmsKeyId – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of a KMS key used for encrypting the channel.

• AdditionalEncryptionContext – A map array of key-value pairs.

  Each key is a UTF-8 string.

  Each value is a UTF-8 string.

  An optional set of non-secret key–value pairs that contains additional contextual information for encryption. This can only be provided if KMSKeyId is provided.

• Tags – An array of Tag objects.

  Metadata assigned to the resource consisting of a list of key-value pairs.

Response

• SourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the source resource for the integration.

• TargetArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the target resource for the integration.

• IntegrationName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  A unique name for an integration in AWS Glue.

• Description – UTF-8 string, not more than 1000 bytes long, matching the Custom string pattern #12.

  A description of the integration.

• IntegrationArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the created integration.

• KmsKeyId – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of a KMS key used for encrypting the channel.

• AdditionalEncryptionContext – A map array of key-value pairs.

  Each key is a UTF-8 string.

  Each value is a UTF-8 string.

  An optional set of non-secret key–value pairs that contains additional contextual information for encryption.

• Tags – An array of Tag objects.

  Metadata assigned to the resource consisting of a list of key-value pairs.

• Status – Required: UTF-8 string (valid values: CREATING | ACTIVE | MODIFYING | FAILED | DELETING | SYNCING | NEEDS_ATTENTION).

  The status of the integration being created.

  The possible statuses are:

  • CREATING: The integration is being created.

  • ACTIVE: The integration creation succeeds.

  • MODIFYING: The integration is being modified.

  • FAILED: The integration creation fails.

  • DELETING: The integration is deleted.

  • SYNCING: The integration is synchronizing.

  • NEEDS_ATTENTION: The integration needs attention, such as synchronization.

• CreateTime – Required: Timestamp.

  The time when the integration was created, in UTC.

• Errors – An array of IntegrationError objects.

  A list of errors associated with the integration creation.

• DataFilter – UTF-8 string, not less than 1 or more than 2048 bytes long.

  Selects source tables for the integration using Maxwell filter syntax.

Errors

• ValidationException

• AccessDeniedException

• ResourceNotFoundException

• InternalServerException

• IntegrationConflictOperationFault

• IntegrationQuotaExceededFault

• KMSKeyNotAccessibleFault

• EntityNotFoundException

• InternalServiceException

• ConflictException

• ResourceNumberLimitExceededException

• InvalidInputException

Request

• IntegrationIdentifier – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the integration.

• Description – UTF-8 string, not more than 1000 bytes long, matching the Custom string pattern #12.

  A description of the integration.

• DataFilter – UTF-8 string, not less than 1 or more than 2048 bytes long.

  Selects source tables for the integration using Maxwell filter syntax.

• IntegrationName – UTF-8 string, not less than 1 or more than 128 bytes long.

  A unique name for an integration in AWS Glue.

Response

• SourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the source for the integration.

• TargetArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the target for the integration.

• IntegrationName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  A unique name for an integration in AWS Glue.

• Description – UTF-8 string, not more than 1000 bytes long, matching the Custom string pattern #12.

  A description of the integration.

• IntegrationArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the integration.

• KmsKeyId – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of a KMS key used for encrypting the channel.

• AdditionalEncryptionContext – A map array of key-value pairs.

  Each key is a UTF-8 string.

  Each value is a UTF-8 string.

  An optional set of non-secret key–value pairs that contains additional contextual information for encryption.

• Tags – An array of Tag objects.

  Metadata assigned to the resource consisting of a list of key-value pairs.

• Status – Required: UTF-8 string (valid values: CREATING | ACTIVE | MODIFYING | FAILED | DELETING | SYNCING | NEEDS_ATTENTION).

  The status of the integration being modified.

  The possible statuses are:

  • CREATING: The integration is being created.

  • ACTIVE: The integration creation succeeds.

  • MODIFYING: The integration is being modified.

  • FAILED: The integration creation fails.

  • DELETING: The integration is deleted.

  • SYNCING: The integration is synchronizing.

  • NEEDS_ATTENTION: The integration needs attention, such as synchronization.

• CreateTime – Required: Timestamp.

  The time when the integration was created, in UTC.

• Errors – An array of IntegrationError objects.

  A list of errors associated with the integration modification.

• DataFilter – UTF-8 string, not less than 1 or more than 2048 bytes long.

  Selects source tables for the integration using Maxwell filter syntax.

Errors

• ValidationException

• AccessDeniedException

• InternalServerException

• IntegrationNotFoundFault

• IntegrationConflictOperationFault

• InvalidIntegrationStateFault

• EntityNotFoundException

• InternalServiceException

• ConflictException

• InvalidStateException

• InvalidInputException

Request

• IntegrationIdentifier – UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the integration.

• Marker – UTF-8 string, not less than 1 or more than 128 bytes long.

  A value that indicates the starting point for the next set of response records in a subsequent request.

• MaxRecords – Number (integer).

  The total number of items to return in the output.

• Filters – An array of IntegrationFilter objects.

  A list of key and values, to filter down the results. Supported keys are "Status", "IntegrationName", and "SourceArn". IntegrationName is limited to only one value.

Response

• Integrations – An array of Integration objects.

  A list of zero-ETL integrations.

• Marker – UTF-8 string, not less than 1 or more than 128 bytes long.

  A value that indicates the starting point for the next set of response records in a subsequent request.

Errors

• ValidationException

• AccessDeniedException

• InternalServerException

• IntegrationNotFoundFault

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• IntegrationIdentifier – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the integration.

Response

• SourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the source for the integration.

• TargetArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The ARN of the target for the integration.

• IntegrationName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  A unique name for an integration in AWS Glue.

• Description – UTF-8 string, not more than 1000 bytes long, matching the Custom string pattern #12.

  A description of the integration.

• IntegrationArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) for the integration.

• KmsKeyId – UTF-8 string, not less than 1 or more than 2048 bytes long.

  The ARN of a KMS key used for encrypting the channel.

• AdditionalEncryptionContext – A map array of key-value pairs.

  Each key is a UTF-8 string.

  Each value is a UTF-8 string.

  An optional set of non-secret key–value pairs that contains additional contextual information for encryption.

• Tags – An array of Tag objects.

  Metadata assigned to the resource consisting of a list of key-value pairs.

• Status – Required: UTF-8 string (valid values: CREATING | ACTIVE | MODIFYING | FAILED | DELETING | SYNCING | NEEDS_ATTENTION).

  The status of the integration being deleted.

  The possible statuses are:

  • CREATING: The integration is being created.

  • ACTIVE: The integration creation succeeds.

  • MODIFYING: The integration is being modified.

  • FAILED: The integration creation fails.

  • DELETING: The integration is deleted.

  • SYNCING: The integration is synchronizing.

  • NEEDS_ATTENTION: The integration needs attention, such as synchronization.

• CreateTime – Required: Timestamp.

  The time when the integration was created, in UTC.

• Errors – An array of IntegrationError objects.

  A list of errors associated with the integration.

• DataFilter – UTF-8 string, not less than 1 or more than 2048 bytes long.

  Selects source tables for the integration using Maxwell filter syntax.

Errors

• ValidationException

• AccessDeniedException

• InternalServerException

• IntegrationNotFoundFault

• IntegrationConflictOperationFault

• InvalidIntegrationStateFault

• EntityNotFoundException

• InternalServiceException

• ConflictException

• InvalidStateException

• InvalidInputException

Request

• IntegrationArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) of the integration.

• Marker – UTF-8 string, not less than 1 or more than 128 bytes long.

  A token to specify where to start paginating. This is the marker from a previously truncated response.

• MaxRecords – Number (integer).

  The total number of items to return in the output.

• TargetArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The Amazon Resource Name (ARN) of the target resource in the integration.

Response

• InboundIntegrations – An array of InboundIntegration objects.

  A list of inbound integrations.

• Marker – UTF-8 string, not less than 1 or more than 128 bytes long.

  A value that indicates the starting point for the next set of response records in a subsequent request.

Errors

• ValidationException

• AccessDeniedException

• InternalServerException

• IntegrationNotFoundFault

• TargetResourceNotFound

• OperationNotSupportedException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• TableName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The name of the table to be replicated.

• SourceTableConfig – A SourceTableConfig object.

  A structure for the source table configuration.

• TargetTableConfig – A TargetTableConfig object.

  A structure for the target table configuration.

Response

• No Response parameters.

Errors

• ValidationException

• AccessDeniedException

• ResourceNotFoundException

• InternalServerException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• TableName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The name of the table to be replicated.

• SourceTableConfig – A SourceTableConfig object.

  A structure for the source table configuration.

• TargetTableConfig – A TargetTableConfig object.

  A structure for the target table configuration.

Response

• No Response parameters.

Errors

• ValidationException

• AccessDeniedException

• ResourceNotFoundException

• InternalServerException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• TableName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The name of the table to be replicated.

Response

• ResourceArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• TableName – UTF-8 string, not less than 1 or more than 128 bytes long.

  The name of the table to be replicated.

• SourceTableConfig – A SourceTableConfig object.

  A structure for the source table configuration.

• TargetTableConfig – A TargetTableConfig object.

  A structure for the target table configuration.

Errors

• ValidationException

• AccessDeniedException

• ResourceNotFoundException

• InternalServerException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• TableName – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The name of the table to be replicated.

Response

• No Response parameters.

Errors

• ValidationException

• AccessDeniedException

• ResourceNotFoundException

• InternalServerException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• SourceProcessingProperties – A SourceProcessingProperties object.

  The resource properties associated with the integration source.

• TargetProcessingProperties – A TargetProcessingProperties object.

  The resource properties associated with the integration target.

Response

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• SourceProcessingProperties – A SourceProcessingProperties object.

  The resource properties associated with the integration source.

• TargetProcessingProperties – A TargetProcessingProperties object.

  The resource properties associated with the integration target.

Errors

• ValidationException

• AccessDeniedException

• ConflictException

• InternalServerException

• ResourceNotFoundException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• SourceProcessingProperties – A SourceProcessingProperties object.

  The resource properties associated with the integration source.

• TargetProcessingProperties – A TargetProcessingProperties object.

  The resource properties associated with the integration target.

Response

• ResourceArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• SourceProcessingProperties – A SourceProcessingProperties object.

  The resource properties associated with the integration source.

• TargetProcessingProperties – A TargetProcessingProperties object.

  The resource properties associated with the integration target.

Errors

• ValidationException

• AccessDeniedException

• InternalServerException

• ResourceNotFoundException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

Response

• ResourceArn – UTF-8 string, not less than 1 or more than 128 bytes long.

  The connection ARN of the source, or the database ARN of the target.

• SourceProcessingProperties – A SourceProcessingProperties object.

  The resource properties associated with the integration source.

• TargetProcessingProperties – A TargetProcessingProperties object.

  The resource properties associated with the integration target.

Errors

• ValidationException

• AccessDeniedException

• InternalServerException

• ResourceNotFoundException

• EntityNotFoundException

• InternalServiceException

• InvalidInputException

Request

• ResourceArn – Required: UTF-8 string, not less than 1 or more than 10240 bytes long, matching the Custom string pattern #49.

  The Amazon Resource Name (ARN) for the integration resource.

• TagsToRemove – Required: An array of UTF-8 strings, not more than 50 strings.

  A list of metadata tags to be removed from the resource.

Response

• No Response parameters.

Errors

• ResourceNotFoundException

Request

• ResourceARN – Required: UTF-8 string, not less than 1 or more than 10240 bytes long, matching the Custom string pattern #49.

  The resource ARN for the resource.

Response

• Tags – An array of Tag objects, not more than 10 structures.

  A list of tags.

Errors

• ResourceNotFoundException

• ResourceNotFoundException structure

• InternalServerException structure

• IntegrationAlreadyExistsFault structure

• IntegrationConflictOperationFault structure

• IntegrationQuotaExceededFault structure

• KMSKeyNotAccessibleFault structure

• IntegrationNotFoundFault structure

• TargetResourceNotFound structure

• InvalidIntegrationStateFault structure

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.

Fields

• Message – UTF-8 string.

  A message describing the problem.