ManagedKafkaEventSourceProps Struct
Properties for a MSK event source.
Example
ts
import { ManagedKafkaEventSource } from 'aws-cdk-lib/aws-lambda-event-sources';
// Your MSK cluster arn
const clusterArn = 'arn:aws:kafka:us-east-1:0123456789019:cluster/SalesCluster/abcd1234-abcd-cafe-abab-9876543210ab-4';
declare const myFunction: lambda.Function;
// Enable basic event and error metrics
myFunction.addEventSource(new ManagedKafkaEventSource({
clusterArn,
topic: 'basic-monitoring',
startingPosition: lambda.StartingPosition.LATEST,
// Provisioned mode is required for observability features
provisionedPollerConfig: {
minimumPollers: 2,
maximumPollers: 10,
},
metricsConfig: {
metrics: [
lambda.MetricType.EVENT_COUNT,
lambda.MetricType.ERROR_COUNT
]
}
}));
Properties
The largest number of records that AWS Lambda will retrieve from your event source at the time of invoking your function.
notes
Your function receives an event with all the retrieved records.
Valid Range:
- Minimum value of 1
- Maximum value of:
- 1000 for
DynamoEventSource - 10000 for
KinesisEventSource,ManagedKafkaEventSourceandSelfManagedKafkaEventSource
- 1000 for
bisectBatchOnErrorbooleanbisect_batch_on_errorboolbisectBatchOnErrorBooleanBisectBatchOnErrorboolBisectBatchOnError*bool= false#* If the function returns an error, split the batch in two and retry.
An MSK cluster construct.
consumerGroupIdstringconsumer_group_idstrconsumerGroupIdStringConsumerGroupIdstringConsumerGroupId*string= none#The identifier for the Kafka consumer group to join.
notes
The consumer group ID must be unique among all your Kafka event sources. After creating a Kafka event source mapping with the consumer group ID specified, you cannot update this value. The value must have a length between 1 and 200 and full the pattern '[a-zA-Z0-9-/:_+=.@-]'.
If the stream event source mapping should be enabled.
filterEncryptionIKeyfilter_encryptionIKeyfilterEncryptionIKeyFilterEncryptionIKeyFilterEncryptionIKey= none#Add Customer managed KMS key to encrypt Filter Criteria.
filters{ [key: string]: any }[]filterstyping.List[typing.Mapping[str, Any]]filtersList<Map<String, Object>>FiltersIDictionary<string, object>[]Filters*[]*map[string]interface{}= none#Add filter criteria to Event Source.
logLevelEventSourceMappingLogLevellog_levelEventSourceMappingLogLevellogLevelEventSourceMappingLogLevelLogLevelEventSourceMappingLogLevelLogLevelEventSourceMappingLogLevel= No logging#Configuration for logging verbosity from the event source mapping poller.
maxBatchingWindowDurationmax_batching_windowDurationmaxBatchingWindowDurationMaxBatchingWindowDurationMaxBatchingWindowDuration= Duration.seconds(0) for Kinesis,…#The maximum amount of time to gather records before invoking the function.
Maximum of Duration.minutes(5).
default
Default: Duration.seconds(0) for Kinesis, DynamoDB, and SQS event sources, Duration.millis(500) for MSK, self-managed Kafka, and Amazon MQ.
maxRecordAgeDurationmax_record_ageDurationmaxRecordAgeDurationMaxRecordAgeDurationMaxRecordAgeDuration= -1#The maximum age of a record that Lambda sends to a function for processing.
The default value is -1, which sets the maximum age to infinite. When the value is set to infinite, Lambda never discards old records. Record are valid until it expires in the event source.
metricsConfigMetricsConfigmetrics_configMetricsConfigmetricsConfigMetricsConfigMetricsConfigMetricsConfigMetricsConfigMetricsConfig= Enhanced monitoring is disabled#Configuration for enhanced monitoring metrics collection.
onFailureIEventSourceDlqon_failureIEventSourceDlqonFailureIEventSourceDlqOnFailureIEventSourceDlqOnFailureIEventSourceDlq= discarded records are ignored#Add an on Failure Destination for this Kafka event.
Supported destinations:
- {@link KafkaDlq } - Send failed records to a Kafka topic
- SNS topics - Send failed records to an SNS topic
- SQS queues - Send failed records to an SQS queue
- S3 buckets - Send failed records to an S3 bucket
provisionedPollerConfigProvisionedPollerConfigprovisioned_poller_configProvisionedPollerConfigprovisionedPollerConfigProvisionedPollerConfigProvisionedPollerConfigProvisionedPollerConfigProvisionedPollerConfigProvisionedPollerConfig= no provisioned pollers#Configuration for provisioned pollers that read from the event source.
When specified, allows control over the minimum and maximum number of pollers that can be provisioned to process events from the source.
reportBatchItemFailuresbooleanreport_batch_item_failuresboolreportBatchItemFailuresBooleanReportBatchItemFailuresboolReportBatchItemFailures*bool= false#* Allow functions to return partially successful responses for a batch of records.
retryAttemptsnumberretry_attemptsint | floatretryAttemptsNumberRetryAttemptsdoubleRetryAttempts*float64= -1 (infinite retries)#* Maximum number of retry attempts.
Set to -1 for infinite retries (until the record expires in the event source).
schemaRegistryConfigISchemaRegistryschema_registry_configISchemaRegistryschemaRegistryConfigISchemaRegistrySchemaRegistryConfigISchemaRegistrySchemaRegistryConfigISchemaRegistry= none#Specific configuration settings for a Kafka schema registry.
The secret with the Kafka credentials, see https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html for details This field is required if your Kafka brokers are accessed over the Internet.
startingPositionStartingPositionstarting_positionStartingPositionstartingPositionStartingPositionStartingPositionStartingPositionStartingPositionStartingPositionrequired#Where to begin consuming the stream.
startingPositionTimestampnumberstarting_position_timestampint | floatstartingPositionTimestampNumberStartingPositionTimestampdoubleStartingPositionTimestamp*float64= no timestamp#The time from which to start reading, in Unix time seconds.
The Kafka topic to subscribe to.