enum WorkerType
Language | Type name |
---|---|
![]() | Amazon.CDK.AWS.Glue.Alpha.WorkerType |
![]() | github.com/aws/aws-cdk-go/awscdkgluealpha/v2#WorkerType |
![]() | software.amazon.awscdk.services.glue.alpha.WorkerType |
![]() | aws_cdk.aws_glue_alpha.WorkerType |
![]() | @aws-cdk/aws-glue-alpha ยป WorkerType |
The type of predefined worker that is allocated when a job runs.
If you need to use a WorkerType that doesn't exist as a static member, you
can instantiate a WorkerType
object, e.g: WorkerType.of('other type')
Example
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
jobName: 'PySparkETLJobCustomName',
description: 'This is a description',
role,
script,
glueVersion: glue.GlueVersion.V3_0,
continuousLogging: { enabled: false },
workerType: glue.WorkerType.G_2X,
maxConcurrentRuns: 100,
timeout: cdk.Duration.hours(2),
connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
tags: {
FirstTagName: 'FirstTagValue',
SecondTagName: 'SecondTagValue',
XTagName: 'XTagValue',
},
numberOfWorkers: 2,
maxRetries: 2,
});
Members
Name | Description |
---|---|
STANDARD | Standard Worker Type 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker. |
G_1X | G.1X Worker Type 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. Suitable for memory-intensive jobs. |
G_2X | G.2X Worker Type 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. Suitable for memory-intensive jobs. |
G_4X | G.4X Worker Type 4 DPU (16 vCPU, 64 GB of memory, 256 GB disk), and provides 1 executor per worker. We recommend this worker type for jobs whose workloads contain your most demanding transforms, aggregations, joins, and queries. This worker type is available only for AWS Glue version 3.0 or later jobs. |
G_8X | G.8X Worker Type 8 DPU (32 vCPU, 128 GB of memory, 512 GB disk), and provides 1 executor per worker. We recommend this worker type for jobs whose workloads contain your most demanding transforms, aggregations, joins, and queries. This worker type is available only for AWS Glue version 3.0 or later jobs. |
G_025X | G.025X Worker Type 0.25 DPU (2 vCPU, 4 GB of memory, 64 GB disk), and provides 1 executor per worker. Suitable for low volume streaming jobs. |
Z_2X | Z.2X Worker Type. |
STANDARD
Standard Worker Type 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
G_1X
G.1X Worker Type 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. Suitable for memory-intensive jobs.
G_2X
G.2X Worker Type 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. Suitable for memory-intensive jobs.
G_4X
G.4X Worker Type 4 DPU (16 vCPU, 64 GB of memory, 256 GB disk), and provides 1 executor per worker. We recommend this worker type for jobs whose workloads contain your most demanding transforms, aggregations, joins, and queries. This worker type is available only for AWS Glue version 3.0 or later jobs.
G_8X
G.8X Worker Type 8 DPU (32 vCPU, 128 GB of memory, 512 GB disk), and provides 1 executor per worker. We recommend this worker type for jobs whose workloads contain your most demanding transforms, aggregations, joins, and queries. This worker type is available only for AWS Glue version 3.0 or later jobs.
G_025X
G.025X Worker Type 0.25 DPU (2 vCPU, 4 GB of memory, 64 GB disk), and provides 1 executor per worker. Suitable for low volume streaming jobs.
Z_2X
Z.2X Worker Type.