CreateSchema - AWS Glue

CreateSchema

Creates a new schema set and registers the schema definition. Returns an error if the schema set already exists without actually registering the version.

When the schema set is created, a version checkpoint will be set to the first version. Compatibility mode "DISABLED" restricts any additional schema versions from being added after the first schema version. For all other compatibility modes, validation of compatibility settings will be applied only from the second version onwards when the RegisterSchemaVersion API is used.

When this API is called without a RegistryId, this will create an entry for a "default-registry" in the registry database tables, if it is not already present.

Request Syntax

{ "Compatibility": "string", "DataFormat": "string", "Description": "string", "RegistryId": { "RegistryArn": "string", "RegistryName": "string" }, "SchemaDefinition": "string", "SchemaName": "string", "Tags": { "string" : "string" } }

Request Parameters

For information about the parameters that are common to all actions, see Common Parameters.

The request accepts the following data in JSON format.

Compatibility

The compatibility mode of the schema. The possible values are:

  • NONE: No compatibility mode applies. You can use this choice in development scenarios or if you do not know the compatibility mode that you want to apply to schemas. Any new version added will be accepted without undergoing a compatibility check.

  • DISABLED: This compatibility choice prevents versioning for a particular schema. You can use this choice to prevent future versioning of a schema.

  • BACKWARD: This compatibility choice is recommended as it allows data receivers to read both the current and one previous schema version. This means that for instance, a new schema version cannot drop data fields or change the type of these fields, so they can't be read by readers using the previous version.

  • BACKWARD_ALL: This compatibility choice allows data receivers to read both the current and all previous schema versions. You can use this choice when you need to delete fields or add optional fields, and check compatibility against all previous schema versions.

  • FORWARD: This compatibility choice allows data receivers to read both the current and one next schema version, but not necessarily later versions. You can use this choice when you need to add fields or delete optional fields, but only check compatibility against the last schema version.

  • FORWARD_ALL: This compatibility choice allows data receivers to read written by producers of any new registered schema. You can use this choice when you need to add fields or delete optional fields, and check compatibility against all previous schema versions.

  • FULL: This compatibility choice allows data receivers to read data written by producers using the previous or next version of the schema, but not necessarily earlier or later versions. You can use this choice when you need to add or remove optional fields, but only check compatibility against the last schema version.

  • FULL_ALL: This compatibility choice allows data receivers to read data written by producers using all previous schema versions. You can use this choice when you need to add or remove optional fields, and check compatibility against all previous schema versions.

Type: String

Valid Values: NONE | DISABLED | BACKWARD | BACKWARD_ALL | FORWARD | FORWARD_ALL | FULL | FULL_ALL

Required: No

DataFormat

The data format of the schema definition. Currently AVRO, JSON and PROTOBUF are supported.

Type: String

Valid Values: AVRO | JSON | PROTOBUF

Required: Yes

Description

An optional description of the schema. If description is not provided, there will not be any automatic default value for this.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 2048.

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Required: No

RegistryId

This is a wrapper shape to contain the registry identity fields. If this is not provided, the default registry will be used. The ARN format for the same will be: arn:aws:glue:us-east-2:<customer id>:registry/default-registry:random-5-letter-id.

Type: RegistryId object

Required: No

SchemaDefinition

The schema definition using the DataFormat setting for SchemaName.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 170000.

Pattern: .*\S.*

Required: No

SchemaName

Name of the schema to be created of max length of 255, and may only contain letters, numbers, hyphen, underscore, dollar sign, or hash mark. No whitespace.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: [a-zA-Z0-9-_$#.]+

Required: Yes

Tags

AWS tags that contain a key value pair and may be searched by console, command line, or API. If specified, follows the AWS tags-on-create pattern.

Type: String to string map

Map Entries: Minimum number of 0 items. Maximum number of 50 items.

Key Length Constraints: Minimum length of 1. Maximum length of 128.

Value Length Constraints: Minimum length of 0. Maximum length of 256.

Required: No

Response Syntax

{ "Compatibility": "string", "DataFormat": "string", "Description": "string", "LatestSchemaVersion": number, "NextSchemaVersion": number, "RegistryArn": "string", "RegistryName": "string", "SchemaArn": "string", "SchemaCheckpoint": number, "SchemaName": "string", "SchemaStatus": "string", "SchemaVersionId": "string", "SchemaVersionStatus": "string", "Tags": { "string" : "string" } }

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

Compatibility

The schema compatibility mode.

Type: String

Valid Values: NONE | DISABLED | BACKWARD | BACKWARD_ALL | FORWARD | FORWARD_ALL | FULL | FULL_ALL

DataFormat

The data format of the schema definition. Currently AVRO, JSON and PROTOBUF are supported.

Type: String

Valid Values: AVRO | JSON | PROTOBUF

Description

A description of the schema if specified when created.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 2048.

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

LatestSchemaVersion

The latest version of the schema associated with the returned schema definition.

Type: Long

Valid Range: Minimum value of 1. Maximum value of 100000.

NextSchemaVersion

The next version of the schema associated with the returned schema definition.

Type: Long

Valid Range: Minimum value of 1. Maximum value of 100000.

RegistryArn

The Amazon Resource Name (ARN) of the registry.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 10240.

Pattern: arn:aws(-(cn|us-gov|iso(-[bef])?))?:glue:.*

RegistryName

The name of the registry.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: [a-zA-Z0-9-_$#.]+

SchemaArn

The Amazon Resource Name (ARN) of the schema.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 10240.

Pattern: arn:aws(-(cn|us-gov|iso(-[bef])?))?:glue:.*

SchemaCheckpoint

The version number of the checkpoint (the last time the compatibility mode was changed).

Type: Long

Valid Range: Minimum value of 1. Maximum value of 100000.

SchemaName

The name of the schema.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: [a-zA-Z0-9-_$#.]+

SchemaStatus

The status of the schema.

Type: String

Valid Values: AVAILABLE | PENDING | DELETING

SchemaVersionId

The unique identifier of the first schema version.

Type: String

Length Constraints: Fixed length of 36.

Pattern: [a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}

SchemaVersionStatus

The status of the first schema version created.

Type: String

Valid Values: AVAILABLE | PENDING | FAILURE | DELETING

Tags

The tags for the schema.

Type: String to string map

Map Entries: Minimum number of 0 items. Maximum number of 50 items.

Key Length Constraints: Minimum length of 1. Maximum length of 128.

Value Length Constraints: Minimum length of 0. Maximum length of 256.

Errors

For information about the errors that are common to all actions, see Common Errors.

AccessDeniedException

Access to a resource was denied.

HTTP Status Code: 400

AlreadyExistsException

A resource to be created or added already exists.

HTTP Status Code: 400

ConcurrentModificationException

Two processes are trying to modify a resource simultaneously.

HTTP Status Code: 400

EntityNotFoundException

A specified entity does not exist

HTTP Status Code: 400

InternalServiceException

An internal service error occurred.

HTTP Status Code: 500

InvalidInputException

The input provided was not valid.

HTTP Status Code: 400

ResourceNumberLimitExceededException

A resource numerical limit was exceeded.

HTTP Status Code: 400

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following: