Package software.amazon.awscdk.services.glue.alpha

Class ScalaSparkFlexEtlJobProps.Builder

java.lang.Object
software.amazon.awscdk.services.glue.alpha.ScalaSparkFlexEtlJobProps.Builder
All Implemented Interfaces:
software.amazon.jsii.Builder<ScalaSparkFlexEtlJobProps>
Enclosing interface:
ScalaSparkFlexEtlJobProps
@Stability(Experimental) public static final class ScalaSparkFlexEtlJobProps.Builder extends Object implements software.amazon.jsii.Builder<ScalaSparkFlexEtlJobProps>
A builder for ScalaSparkFlexEtlJobProps

  • Constructor Details

    • Builder

      public Builder()

  • Method Details

    • className

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder className(String className)
      Sets the value of ScalaSparkFlexEtlJobProps.getClassName()
      Parameters:
      className - The fully qualified Scala class name that serves as the entry point for the job. This parameter is required.
      Returns:
      this

    • extraFiles

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder extraFiles(List<? extends Code> extraFiles)
      Sets the value of ScalaSparkFlexEtlJobProps.getExtraFiles()
      Parameters:
      extraFiles - Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.
      Returns:
      this

    • extraJars

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder extraJars(List<? extends Code> extraJars)
      Sets the value of ScalaSparkFlexEtlJobProps.getExtraJars()
      Parameters:
      extraJars - Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located.
      Returns:
      this

    • extraJarsFirst

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder extraJarsFirst(Boolean extraJarsFirst)
      Sets the value of ScalaSparkFlexEtlJobProps.getExtraJarsFirst()
      Parameters:
      extraJarsFirst - Setting this value to true prioritizes the customer's extra JAR files in the classpath.
      Returns:
      this

    • notifyDelayAfter

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder notifyDelayAfter(Duration notifyDelayAfter)
      Sets the value of ScalaSparkFlexEtlJobProps.getNotifyDelayAfter()
      Parameters:
      notifyDelayAfter - Specifies configuration properties of a notification (optional). After a job run starts, the number of minutes to wait before sending a job run delay notification.
      Returns:
      this

    • enableMetrics

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder enableMetrics(Boolean enableMetrics)
      Sets the value of SparkJobProps.getEnableMetrics()
      Parameters:
      enableMetrics - Enable profiling metrics for the Glue job. When enabled, adds '--enable-metrics' to job arguments.
      Returns:
      this

    • enableObservabilityMetrics

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder enableObservabilityMetrics(Boolean enableObservabilityMetrics)
      Sets the value of SparkJobProps.getEnableObservabilityMetrics()
      Parameters:
      enableObservabilityMetrics - Enable observability metrics for the Glue job. When enabled, adds '--enable-observability-metrics': 'true' to job arguments.
      Returns:
      this

    • sparkUi

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder sparkUi(SparkUIProps sparkUi)
      Sets the value of
      invalid reference 
      ScalaSparkFlexEtlJobProps#getSparkUi
      Parameters:
      sparkUi - Enables the Spark UI debugging and monitoring with the specified props.
      Returns:
      this

    • workerConfiguration

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder workerConfiguration(WorkerConfiguration workerConfiguration)
      Sets the value of SparkJobProps.getWorkerConfiguration()
      Parameters:
      workerConfiguration - The worker type and the number of workers allocated when a job runs.
      Returns:
      this

    • role

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder role(IRole role)
      Sets the value of JobProps.getRole()
      Parameters:
      role - IAM Role (required) IAM Role to use for Glue job execution Must be specified by the developer because the L2 doesn't have visibility into the actions the script(s) takes during the job execution The role must trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions. This parameter is required.
      Returns:
      this

    • script

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder script(Code script)
      Sets the value of JobProps.getScript()
      Parameters:
      script - Script Code Location (required) Script to run when the Glue job executes. This parameter is required. Can be uploaded from the local directory structure using fromAsset or referenced via S3 location using fromBucket
      Returns:
      this

    • connections

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder connections(List<? extends IConnection> connections)
      Sets the value of JobProps.getConnections()
      Parameters:
      connections - Connections (optional) List of connections to use for this Glue job Connections are used to connect to other AWS Service or resources within a VPC.
      Returns:
      this

    • continuousLogging

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder continuousLogging(ContinuousLoggingProps continuousLogging)
      Sets the value of JobProps.getContinuousLogging()
      Parameters:
      continuousLogging - Enables continuous logging with the specified props.
      Returns:
      this

    • defaultArguments

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder defaultArguments(Map<String,String> defaultArguments)
      Sets the value of JobProps.getDefaultArguments()
      Parameters:
      defaultArguments - Default Arguments (optional) The default arguments for every run of this Glue job, specified as name-value pairs. This map is the escape hatch for Glue job arguments that this construct does not model. It MUST NOT be used to set arguments that already have a dedicated prop — configure those through the corresponding prop instead (continuousLogging, enableMetrics, enableObservabilityMetrics, sparkUI, className, extraJars, extraJarsFirst, extraPythonFiles, extraFiles). Passing a construct-managed argument (e.g. --enable-continuous-cloudwatch-log, --enable-metrics, --enable-spark-ui, --job-language) or a Glue-reserved argument (--debug, --mode, --JOB_NAME, --endpoint) here throws at synthesis time, so there is exactly one way to express each intent.

      Also note that these are emitted verbatim into the CloudFormation template, so avoid placing secrets here in plaintext. Pass secrets to the job at runtime through AWS Secrets Manager instead. A synthesis-time warning is emitted when an argument key looks like a credential and holds a plaintext literal.

      Returns:
      this

    • description

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder description(String description)
      Sets the value of JobProps.getDescription()
      Parameters:
      description - Description (optional) Developer-specified description of the Glue job.
      Returns:
      this

    • glueVersion

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder glueVersion(GlueVersion glueVersion)
      Sets the value of JobProps.getGlueVersion()
      Parameters:
      glueVersion - Glue Version The version of Glue to use to execute this job.
      Returns:
      this

    • jobName

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder jobName(String jobName)
      Sets the value of JobProps.getJobName()
      Parameters:
      jobName - Name of the Glue job (optional) Developer-specified name of the Glue job.
      Returns:
      this

    • maxConcurrentRuns

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder maxConcurrentRuns(Number maxConcurrentRuns)
      Sets the value of JobProps.getMaxConcurrentRuns()
      Parameters:
      maxConcurrentRuns - Max Concurrent Runs (optional) The maximum number of runs this Glue job can concurrently run. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
      Returns:
      this

    • maxRetries

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder maxRetries(Number maxRetries)
      Sets the value of JobProps.getMaxRetries()
      Parameters:
      maxRetries - Max Retries (optional) Maximum number of retry attempts Glue performs if the job fails.
      Returns:
      this

    • securityConfiguration

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder securityConfiguration(ISecurityConfiguration securityConfiguration)
      Sets the value of JobProps.getSecurityConfiguration()
      Parameters:
      securityConfiguration - Security Configuration (optional) Defines the encryption options for the Glue job.
      Returns:
      this

    • tags

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder tags(Map<String,String> tags)
      Sets the value of JobProps.getTags()
      Parameters:
      tags - Tags (optional) A list of key:value pairs of tags to apply to this Glue job resources.
      Returns:
      this

    • timeout

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps.Builder timeout(Duration timeout)
      Sets the value of JobProps.getTimeout()
      Parameters:
      timeout - Timeout (optional) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. Specified in minutes.
      Returns:
      this

    • build

      @Stability(Experimental) public ScalaSparkFlexEtlJobProps build()
      Builds the configured instance.
      Specified by:
      build in interface software.amazon.jsii.Builder<ScalaSparkFlexEtlJobProps>
      Returns:
      a new instance of ScalaSparkFlexEtlJobProps
      Throws:
      NullPointerException - if any required attribute was not provided