CreateDevEndpointCommand

Creates a new development endpoint.

Example Syntax

Use a bare-bones client and the command you need to make an API call.

import { GlueClient, CreateDevEndpointCommand } from "@aws-sdk/client-glue"; // ES Modules import
// const { GlueClient, CreateDevEndpointCommand } = require("@aws-sdk/client-glue"); // CommonJS import
const client = new GlueClient(config);
const input = { // CreateDevEndpointRequest
  EndpointName: "STRING_VALUE", // required
  RoleArn: "STRING_VALUE", // required
  SecurityGroupIds: [ // StringList
    "STRING_VALUE",
  ],
  SubnetId: "STRING_VALUE",
  PublicKey: "STRING_VALUE",
  PublicKeys: [ // PublicKeysList
    "STRING_VALUE",
  ],
  NumberOfNodes: Number("int"),
  WorkerType: "Standard" || "G.1X" || "G.2X" || "G.025X" || "G.4X" || "G.8X" || "Z.2X",
  GlueVersion: "STRING_VALUE",
  NumberOfWorkers: Number("int"),
  ExtraPythonLibsS3Path: "STRING_VALUE",
  ExtraJarsS3Path: "STRING_VALUE",
  SecurityConfiguration: "STRING_VALUE",
  Tags: { // TagsMap
    "<keys>": "STRING_VALUE",
  },
  Arguments: { // MapValue
    "<keys>": "STRING_VALUE",
  },
};
const command = new CreateDevEndpointCommand(input);
const response = await client.send(command);
// { // CreateDevEndpointResponse
//   EndpointName: "STRING_VALUE",
//   Status: "STRING_VALUE",
//   SecurityGroupIds: [ // StringList
//     "STRING_VALUE",
//   ],
//   SubnetId: "STRING_VALUE",
//   RoleArn: "STRING_VALUE",
//   YarnEndpointAddress: "STRING_VALUE",
//   ZeppelinRemoteSparkInterpreterPort: Number("int"),
//   NumberOfNodes: Number("int"),
//   WorkerType: "Standard" || "G.1X" || "G.2X" || "G.025X" || "G.4X" || "G.8X" || "Z.2X",
//   GlueVersion: "STRING_VALUE",
//   NumberOfWorkers: Number("int"),
//   AvailabilityZone: "STRING_VALUE",
//   VpcId: "STRING_VALUE",
//   ExtraPythonLibsS3Path: "STRING_VALUE",
//   ExtraJarsS3Path: "STRING_VALUE",
//   FailureReason: "STRING_VALUE",
//   SecurityConfiguration: "STRING_VALUE",
//   CreatedTimestamp: new Date("TIMESTAMP"),
//   Arguments: { // MapValue
//     "<keys>": "STRING_VALUE",
//   },
// };

CreateDevEndpointCommand Input

See CreateDevEndpointCommandInput for more details

Parameter
Type
Description
EndpointName
Required
string | undefined

The name to be assigned to the new DevEndpoint.

RoleArn
Required
string | undefined

The IAM role for the DevEndpoint.

Arguments
Record<string, string> | undefined

A map of arguments used to configure the DevEndpoint.

ExtraJarsS3Path
string | undefined

The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint.

ExtraPythonLibsS3Path
string | undefined

The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

You can only use pure Python libraries with a DevEndpoint. Libraries that rely on C extensions, such as the pandas  Python data analysis library, are not yet supported.

GlueVersion
string | undefined

Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints.

For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version  in the developer guide.

Development endpoints that are created without specifying a Glue version default to Glue 0.9.

You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

NumberOfNodes
number | undefined

The number of Glue Data Processing Units (DPUs) to allocate to this DevEndpoint.

NumberOfWorkers
number | undefined

The number of workers of a defined workerType that are allocated to the development endpoint.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

PublicKey
string | undefined

The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility because the recommended attribute to use is public keys.

PublicKeys
string[] | undefined

A list of public keys to be used by the development endpoints for authentication. The use of this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the UpdateDevEndpoint API with the public key content in the deletePublicKeys attribute, and the list of new keys in the addPublicKeys attribute.

SecurityConfiguration
string | undefined

The name of the SecurityConfiguration structure to be used with this DevEndpoint.

SecurityGroupIds
string[] | undefined

Security group IDs for the security groups to be used by the new DevEndpoint.

SubnetId
string | undefined

The subnet ID for the new DevEndpoint to use.

Tags
Record<string, string> | undefined

The tags to use with this DevEndpoint. You may use tags to limit access to the DevEndpoint. For more information about tags in Glue, see Amazon Web Services Tags in Glue  in the developer guide.

WorkerType
WorkerType | undefined

The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

  • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

  • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

  • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

CreateDevEndpointCommand Output

Parameter
Type
Description
$metadata
Required
ResponseMetadata
Metadata pertaining to this request.
Arguments
Record<string, string> | undefined

The map of arguments used to configure this DevEndpoint.

Valid arguments are:

  • "--enable-glue-datacatalog": ""

You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

AvailabilityZone
string | undefined

The Amazon Web Services Availability Zone where this DevEndpoint is located.

CreatedTimestamp
Date | undefined

The point in time at which this DevEndpoint was created.

EndpointName
string | undefined

The name assigned to the new DevEndpoint.

ExtraJarsS3Path
string | undefined

Path to one or more Java .jar files in an S3 bucket that will be loaded in your DevEndpoint.

ExtraPythonLibsS3Path
string | undefined

The paths to one or more Python libraries in an S3 bucket that will be loaded in your DevEndpoint.

FailureReason
string | undefined

The reason for a current failure in this DevEndpoint.

GlueVersion
string | undefined

Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints.

For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version  in the developer guide.

NumberOfNodes
number | undefined

The number of Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

NumberOfWorkers
number | undefined

The number of workers of a defined workerType that are allocated to the development endpoint.

RoleArn
string | undefined

The Amazon Resource Name (ARN) of the role assigned to the new DevEndpoint.

SecurityConfiguration
string | undefined

The name of the SecurityConfiguration structure being used with this DevEndpoint.

SecurityGroupIds
string[] | undefined

The security groups assigned to the new DevEndpoint.

Status
string | undefined

The current status of the new DevEndpoint.

SubnetId
string | undefined

The subnet ID assigned to the new DevEndpoint.

VpcId
string | undefined

The ID of the virtual private cloud (VPC) used by this DevEndpoint.

WorkerType
WorkerType | undefined

The type of predefined worker that is allocated to the development endpoint. May be a value of Standard, G.1X, or G.2X.

YarnEndpointAddress
string | undefined

The address of the YARN endpoint used by this DevEndpoint.

ZeppelinRemoteSparkInterpreterPort
number | undefined

The Apache Zeppelin port for the remote Apache Spark interpreter.

Throws

Name
Fault
Details
AccessDeniedException
client

Access to a resource was denied.

AlreadyExistsException
client

A resource to be created or added already exists.

IdempotentParameterMismatchException
client

The same unique identifier was associated with two different records.

InternalServiceException
server

An internal service error occurred.

InvalidInputException
client

The input provided was not valid.

OperationTimeoutException
client

The operation timed out.

ResourceNumberLimitExceededException
client

A resource numerical limit was exceeded.

ValidationException
client

A value could not be validated.

GlueServiceException
Base exception class for all service exceptions from Glue service.