aws-cdk-lib.aws_stepfunctions_tasks.JobDriver

JobDriver Struct

Specify the driver that the EMR Containers job runs on.

The job driver is used to provide an input for the job that will be run.

Source

aws-stepfunctions-tasks/lib/emrcontainers/start-job-run.ts

Type last changed

v2.1.0 · 8 Dec 2021

Example

new tasks.EmrContainersStartJobRun(this, 'EMR Containers Start Job Run', {
  virtualCluster: tasks.VirtualClusterInput.fromVirtualClusterId('de92jdei2910fwedz'),
  releaseLabel: tasks.ReleaseLabel.EMR_6_2_0,
  jobName: 'EMR-Containers-Job',
  jobDriver: {
    sparkSubmitJobDriver: {
      entryPoint: sfn.TaskInput.fromText('local:///usr/lib/spark/examples/src/main/python/pi.py'),
    },
  },
  applicationConfig: [{
    classification: tasks.Classification.SPARK_DEFAULTS,
    properties: {
      'spark.executor.instances': '1',
      'spark.executor.memory': '512M',
    },
  }],
});

Properties

sparkSubmitJobDriverSparkSubmitJobDriverspark_submit_job_driverSparkSubmitJobDriversparkSubmitJobDriverSparkSubmitJobDriverSparkSubmitJobDriverSparkSubmitJobDriverSparkSubmitJobDriverSparkSubmitJobDriverrequired#

The job driver parameters specified for spark submit.