ContinuousLoggingProps

class aws_cdk.aws_glue.ContinuousLoggingProps(*, enabled, conversion_pattern=None, log_group=None, log_stream_prefix=None, quiet=None)

Bases: object

Properties for enabling Continuous Logging for Glue Jobs.

Parameters:

enabled (bool) – Enable continuous logging.
conversion_pattern (Optional[str]) – Apply the provided conversion pattern. This is a Log4j Conversion Pattern to customize driver and executor logs. Default: %d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
log_group (Optional[ILogGroup]) – Specify a custom CloudWatch log group name. Default: - a log group is created with name /aws-glue/jobs/logs-v2/.
log_stream_prefix (Optional[str]) – Specify a custom CloudWatch log stream prefix. Default: - the job run ID.
quiet (Optional[bool]) – Filter out non-useful Apache Spark driver/executor and Apache Hadoop YARN heartbeat log messages. Default: true

See:

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

ExampleMetadata:

infused

Example:

import aws_cdk as cdk
import aws_cdk.aws_iam as iam
# stack: cdk.Stack
# role: iam.IRole
# script: glue.Code

glue.PySparkEtlJob(stack, "PySparkETLJob",
    job_name="PySparkETLJobCustomName",
    description="This is a description",
    role=role,
    script=script,
    glue_version=glue.GlueVersion.V5_1,
    continuous_logging=glue.ContinuousLoggingProps(enabled=False),
    worker_configuration=glue.WorkerConfiguration(
        worker_type=glue.WorkerType.G_2X,
        number_of_workers=2
    ),
    max_concurrent_runs=100,
    timeout=cdk.Duration.hours(2),
    connections=[glue.Connection.from_connection_name(stack, "Connection", "connectionName")],
    security_configuration=glue.SecurityConfiguration.from_security_configuration_name(stack, "SecurityConfig", "securityConfigName"),
    tags={
        "FirstTagName": "FirstTagValue",
        "SecondTagName": "SecondTagValue",
        "XTagName": "XTagValue"
    },
    max_retries=2
)

Attributes

conversion_pattern

Apply the provided conversion pattern.

This is a Log4j Conversion Pattern to customize driver and executor logs.

Default:: %d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

enabled: Enable continuous logging.

log_group

Specify a custom CloudWatch log group name.

Default:

a log group is created with name /aws-glue/jobs/logs-v2/.

log_stream_prefix

Specify a custom CloudWatch log stream prefix.

Default:

the job run ID.

quiet

Filter out non-useful Apache Spark driver/executor and Apache Hadoop YARN heartbeat log messages.

Default:: true