选择您的 Cookie 首选项

我们使用必要 Cookie 和类似工具提供我们的网站和服务。我们使用性能 Cookie 收集匿名统计数据,以便我们可以了解客户如何使用我们的网站并进行改进。必要 Cookie 无法停用,但您可以单击“自定义”或“拒绝”来拒绝性能 Cookie。

如果您同意,AWS 和经批准的第三方还将使用 Cookie 提供有用的网站功能、记住您的首选项并显示相关内容,包括相关广告。要接受或拒绝所有非必要 Cookie,请单击“接受”或“拒绝”。要做出更详细的选择,请单击“自定义”。

KinesisStreamingSourceOptions - AWS Glue
此页面尚未翻译为您的语言。 请求翻译

KinesisStreamingSourceOptions

Additional options for the Amazon Kinesis streaming data source.

Contents

AddIdleTimeBetweenReads

Adds a time delay between two consecutive getRecords operations. The default value is "False". This option is only configurable for Glue version 2.0 and above.

Type: Boolean

Required: No

AddRecordTimestamp

When this option is set to 'true', the data output will contain an additional column named "__src_timestamp" that indicates the time when the corresponding record received by the stream. The default value is 'false'. This option is supported in AWS Glue version 4.0 or later.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

AvoidEmptyBatches

Avoids creating an empty microbatch job by checking for unread data in the Kinesis data stream before the batch is started. The default value is "False".

Type: Boolean

Required: No

Classification

An optional classification.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

Delimiter

Specifies the delimiter character.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

DescribeShardInterval

The minimum time interval between two ListShards API calls for your script to consider resharding. The default value is 1s.

Type: Long

Valid Range: Minimum value of 0.

Required: No

EmitConsumerLagMetrics

When this option is set to 'true', for each batch, it will emit the metrics for the duration between the oldest record received by the stream and the time it arrives in AWS Glue to CloudWatch. The metric's name is "glue.driver.streaming.maxConsumerLagInMs". The default value is 'false'. This option is supported in AWS Glue version 4.0 or later.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

EndpointUrl

The URL of the Kinesis endpoint.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

IdleTimeBetweenReadsInMs

The minimum time delay between two consecutive getRecords operations, specified in ms. The default value is 1000. This option is only configurable for Glue version 2.0 and above.

Type: Long

Valid Range: Minimum value of 0.

Required: No

MaxFetchRecordsPerShard

The maximum number of records to fetch per shard in the Kinesis data stream per microbatch. Note: The client can exceed this limit if the streaming job has already read extra records from Kinesis (in the same get-records call). If MaxFetchRecordsPerShard needs to be strict then it needs to be a multiple of MaxRecordPerRead. The default value is 100000.

Type: Long

Valid Range: Minimum value of 0.

Required: No

MaxFetchTimeInMs

The maximum time spent for the job executor to read records for the current batch from the Kinesis data stream, specified in milliseconds (ms). Multiple GetRecords API calls may be made within this time. The default value is 1000.

Type: Long

Valid Range: Minimum value of 0.

Required: No

MaxRecordPerRead

The maximum number of records to fetch from the Kinesis data stream in each getRecords operation. The default value is 10000.

Type: Long

Valid Range: Minimum value of 0.

Required: No

MaxRetryIntervalMs

The maximum cool-off time period (specified in ms) between two retries of a Kinesis Data Streams API call. The default value is 10000.

Type: Long

Valid Range: Minimum value of 0.

Required: No

NumRetries

The maximum number of retries for Kinesis Data Streams API requests. The default value is 3.

Type: Integer

Valid Range: Minimum value of 0.

Required: No

RetryIntervalMs

The cool-off time period (specified in ms) before retrying the Kinesis Data Streams API call. The default value is 1000.

Type: Long

Valid Range: Minimum value of 0.

Required: No

RoleArn

The Amazon Resource Name (ARN) of the role to assume using AWS Security Token Service (AWS STS). This role must have permissions for describe or read record operations for the Kinesis data stream. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSSessionName".

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

RoleSessionName

An identifier for the session assuming the role using AWS STS. You must use this parameter when accessing a data stream in a different account. Used in conjunction with "awsSTSRoleARN".

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

StartingPosition

The starting position in the Kinesis data stream to read data from. The possible values are "latest", "trim_horizon", "earliest", or a timestamp string in UTC format in the pattern yyyy-mm-ddTHH:MM:SSZ (where Z represents a UTC timezone offset with a +/-. For example: "2023-04-04T08:00:00-04:00"). The default value is "latest".

Note: Using a value that is a timestamp string in UTC format for "startingPosition" is supported only for AWS Glue version 4.0 or later.

Type: String

Valid Values: latest | trim_horizon | earliest | timestamp

Required: No

StartingTimestamp

The timestamp of the record in the Kinesis data stream to start reading data from. The possible values are a timestamp string in UTC format of the pattern yyyy-mm-ddTHH:MM:SSZ (where Z represents a UTC timezone offset with a +/-. For example: "2023-04-04T08:00:00+08:00").

Type: Timestamp

Required: No

StreamArn

The Amazon Resource Name (ARN) of the Kinesis data stream.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

StreamName

The name of the Kinesis data stream.

Type: String

Pattern: ([\u0009\u000B\u000C\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF])*

Required: No

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following:

隐私网站条款Cookie 首选项
© 2025, Amazon Web Services, Inc. 或其附属公司。保留所有权利。