GlueVersion

class aws_cdk.aws_glue_alpha.GlueVersion(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

(experimental) AWS Glue version determines the versions of Apache Spark and Python that are available to the job.

See:

https://docs.aws.amazon.com/glue/latest/dg/add-job.html.

Stability:

experimental

ExampleMetadata:

infused

Example:

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkEtlJob(stack, "PySparkETLJob",
    job_name="PySparkETLJobCustomName",
    description="This is a description",
    role=role,
    script=script,
    glue_version=glue.GlueVersion.V3_0,
    continuous_logging=glue.ContinuousLoggingProps(enabled=False),
    worker_type=glue.WorkerType.G_2X,
    max_concurrent_runs=100,
    timeout=cdk.Duration.hours(2),
    connections=[glue.Connection.from_connection_name(stack, "Connection", "connectionName")],
    security_configuration=glue.SecurityConfiguration.from_security_configuration_name(stack, "SecurityConfig", "securityConfigName"),
    tags={
        "FirstTagName": "FirstTagValue",
        "SecondTagName": "SecondTagValue",
        "XTagName": "XTagValue"
    },
    number_of_workers=2,
    max_retries=2
)

Attributes

V0_9

(experimental) Glue version using Spark 2.2.1 and Python 2.7.

Stability:

experimental

V1_0

(experimental) Glue version using Spark 2.4.3, Python 2.7 and Python 3.6.

Stability:

experimental

V2_0

(experimental) Glue version using Spark 2.4.3 and Python 3.7.

Stability:

experimental

V3_0

(experimental) Glue version using Spark 3.1.1 and Python 3.7.

Stability:

experimental

V4_0

(experimental) Glue version using Spark 3.3.0 and Python 3.10.

Stability:

experimental

V5_0

(experimental) Glue version using Spark 3.5.2 and Python 3.11.

Stability:

experimental