Clusters clusterArn Channels
URI
/v1/clusters/clusterArn/channels
HTTP methods
GET
Operation ID: ListChannels
Returns all channels in a cluster.
| Name | Type | Required | Description |
|---|---|---|---|
clusterArn | String | True | The Amazon Resource Name (ARN) that uniquely identifies the cluster. |
| Name | Type | Required | Description |
|---|---|---|---|
topicNameFilter | String | False | Filters results to channels whose topic name matches the specified value. |
nextToken | String | False | The paginated results marker. When the result of the operation is truncated, the call returns |
maxResults | String | False | The maximum number of results to return in the response (default maximum 100 results per API call). If there are more results, the response includes a |
| Status code | Response model | Description |
|---|---|---|
200 |
ListChannelsResponse | Successful response. |
400 | Error | The request isn't valid because the input is incorrect. Correct your input and then submit it again. |
401 | Error | The request is not authorized. The provided credentials couldn't be validated. |
403 | Error | Access forbidden. Check your credentials and then retry your request. |
404 | Error | The resource could not be found due to incorrect input. Correct the input, then retry the request. |
429 | Error | 429 response |
500 | Error | There was an unexpected internal server error. Retrying your request might resolve the issue. |
503 | Error | 503 response |
POST
Operation ID: CreateChannel
Creates a channel using the specified properties in the request on the cluster specified by the Amazon Resource Name (ARN) in the request.
| Name | Type | Required | Description |
|---|---|---|---|
clusterArn | String | True | The Amazon Resource Name (ARN) that uniquely identifies the cluster. |
| Status code | Response model | Description |
|---|---|---|
200 |
CreateChannelResponse | Successful response. |
400 | Error | The request isn't valid because the input is incorrect. Correct your input and then submit it again. |
401 | Error | The request is not authorized. The provided credentials couldn't be validated. |
403 | Error | Access forbidden. Check your credentials and then retry your request. |
404 | Error | The resource could not be found due to incorrect input. Correct the input, then retry the request. |
409 | Error | This cluster name already exists. Retry your request using another name. |
429 | Error | 429 response |
500 | Error | There was an unexpected internal server error. Retrying your request might resolve the issue. |
503 | Error | 503 response |
OPTIONS
Enable CORS by returning correct headers
| Name | Type | Required | Description |
|---|---|---|---|
clusterArn | String | True | The Amazon Resource Name (ARN) that uniquely identifies the cluster. |
| Status code | Response model | Description |
|---|---|---|
200 | None | Default response for CORS method |
Schemas
Request bodies
{ "s3DestinationConfiguration": { "dataFreshnessInSeconds": integer, "serviceExecutionRoleArn": "string", "deadLetterQueueS3": { "bucketArn": "string", "expectedBucketOwner": "string", "errorOutputPrefix": "string" }, "storage": { "storageClass": enum, "bucketArn": "string", "outputKeyTemplate": "string", "outputPrefix": "string", "expectedBucketOwner": "string", "compressionType": enum } }, "topicConfigurationList": [ { "recordConverter": { "valueConverter": enum }, "recordSchema": { "gsrArn": "string" }, "topicArn": "string" } ], "encryptionConfiguration": { "kmsKeyArn": "string" }, "channelName": "string", "loggingInfo": { "s3": { "bucket": "string", "prefix": "string", "enabled": boolean }, "firehose": { "deliveryStream": "string", "enabled": boolean }, "cloudWatchLogs": { "logGroup": "string", "enabled": boolean } }, "icebergDestinationConfiguration": { "dataFreshnessInSeconds": integer, "serviceExecutionRoleArn": "string", "catalog": { "catalogArn": "string", "warehouseLocation": "string" }, "schemaEvolution": { "enableSchemaEvolution": boolean }, "deadLetterQueueS3": { "bucketArn": "string", "expectedBucketOwner": "string", "errorOutputPrefix": "string" }, "compressionType": enum, "destinationTableList": [ { "destinationTableName": "string", "destinationDatabaseName": "string", "partitionSpec": { "partitionStrategy": enum, "sourceList": [ { "sourceName": "string" } ] } } ], "tableCreation": { "enableTableCreation": boolean }, "appendOnly": boolean }, "tags": { } }
Response bodies
{ "channels": [ { "clusterOperationArn": "string", "channelArn": "string", "creationTime": "string", "destinationType": enum, "channelName": "string", "status": enum } ], "nextToken": "string" }
{ "clusterOperationArn": "string", "channelArn": "string" }
{ "message": "string", "invalidParameter": "string" }
Properties
Catalog
Configuration of the AWS Glue Data Catalog and S3 Tables warehouse used by the Apache Iceberg destination.
| Property | Type | Required | Description |
|---|---|---|---|
catalogArn | string | False | The Amazon Resource Name (ARN) of the federated AWS Glue Data Catalog that projects the S3 Tables bucket. If omitted, MSK derives the catalog ARN from warehouseLocation. |
warehouseLocation | string | False | The Amazon Resource Name (ARN) of the S3 Tables bucket that backs the Apache Iceberg warehouse. |
ChannelDestinationType
The type of destination configured for the channel.
ICEBERGS3
ChannelInfo
Summary information about a channel returned by ListChannels.
| Property | Type | Required | Description |
|---|---|---|---|
channelArn | string | True | The Amazon Resource Name (ARN) that uniquely identifies the channel. |
channelName | string | True | The name of the channel. |
clusterOperationArn | string | False | The Amazon Resource Name (ARN) of the in-flight cluster operation. Returned only while the channel is in CREATING, UPDATING, or DELETING. |
creationTime | string | True | The time when the channel was created. |
destinationType | True | The type of destination configured for the channel. | |
status | True | The current lifecycle state of the channel. |
ChannelLoggingInfo
Configuration for the destinations to which the channel publishes operational logs.
| Property | Type | Required | Description |
|---|---|---|---|
cloudWatchLogs | False | Configuration for delivering logs to Amazon CloudWatch Logs. | |
firehose | False | Configuration for delivering logs to an Amazon Data Firehose delivery stream. | |
s3 | False | Configuration for delivering logs to an Amazon S3 bucket. |
ChannelStatus
The lifecycle state of a channel.
CREATINGACTIVEUPDATINGDELETINGFAILEDSUSPENDINGSUSPENDED
CloudWatchLogs
Details of the CloudWatch Logs destination for broker logs.
| Property | Type | Required | Description |
|---|---|---|---|
enabled | boolean | True | Specifies whether broker logs get sent to the specified CloudWatch Logs destination. |
logGroup | string | False | The CloudWatch log group that is the destination for broker logs. |
CreateChannelRequest
Creates a Channel that streams records from an Amazon MSK Express cluster topic to Amazon S3 or Apache Iceberg.
| Property | Type | Required | Description |
|---|---|---|---|
channelName | string | True | The name of the channel. Must be unique within the cluster. |
encryptionConfiguration | False | The encryption configuration applied to the channel. | |
icebergDestinationConfiguration | False | The Apache Iceberg destination for the channel. Mutually exclusive with s3DestinationConfiguration. | |
loggingInfo | False | The destinations to which the channel publishes operational logs. | |
s3DestinationConfiguration | False | The Amazon S3 destination for the channel. Mutually exclusive with icebergDestinationConfiguration. | |
tags | object | False | The tags attached to the channel. |
topicConfigurationList | Array of type TopicConfiguration | True | The list of topic configurations for the channel. Currently exactly one topic must be specified. |
CreateChannelResponse
Returns the channel ARN and the cluster-operation ARN that tracks the asynchronous create.
| Property | Type | Required | Description |
|---|---|---|---|
channelArn | string | True | The Amazon Resource Name (ARN) that uniquely identifies the channel. |
clusterOperationArn | string | False | The Amazon Resource Name (ARN) of the cluster operation. |
DeadLetterQueueS3
Configuration of the Amazon S3 bucket where records that fail to deliver are stored.
| Property | Type | Required | Description |
|---|---|---|---|
bucketArn | string | True | The Amazon Resource Name (ARN) of the dead-letter Amazon S3 bucket. |
errorOutputPrefix | string | False | An optional prefix prepended to every dead-letter Amazon S3 object key. |
expectedBucketOwner | string | False | Optional 12-digit AWS account ID expected to own the dead-letter Amazon S3 bucket. |
DestinationTable
Configuration of an Apache Iceberg destination table.
| Property | Type | Required | Description |
|---|---|---|---|
destinationDatabaseName | string | False | The name of the destination namespace (database) in the AWS Glue Data Catalog. |
destinationTableName | string | False | The name of the destination Apache Iceberg table. |
partitionSpec | False | The partition specification for the destination table. |
EncryptionConfiguration
The AWS KMS encryption configuration applied to data at rest.
| Property | Type | Required | Description |
|---|---|---|---|
kmsKeyArn | string | True | The Amazon Resource Name (ARN) of the AWS KMS key used to encrypt the data. |
Error
Returns information about an error.
| Property | Type | Required | Description |
|---|---|---|---|
invalidParameter | string | False | The parameter that caused the error. |
message | string | False | The description of the error. |
Firehose
Firehose details for BrokerLogs.
| Property | Type | Required | Description |
|---|---|---|---|
deliveryStream | string | False | The Kinesis Data Firehose delivery stream that is the destination for broker logs. |
enabled | boolean | True | Specifies whether broker logs get sent to the specified Kinesis Data Firehose delivery stream. |
IcebergCompressionType
Compression codec for Iceberg table data files. Defaults to ZSTD.
ZSTDSNAPPY
IcebergDestinationConfiguration
Configuration of an Apache Iceberg destination for a channel.
| Property | Type | Required | Description |
|---|---|---|---|
appendOnly | boolean | True | Whether the destination is append-only. Must be true; updates and deletes are not supported. |
catalog | False | The AWS Glue Data Catalog and S3 Tables warehouse used by the destination. | |
compressionType | False | The compression codec for Iceberg table data files. Defaults to ZSTD. | |
dataFreshnessInSeconds | integer | False | The maximum time, in seconds, that records buffer in MSK before being flushed to the destination. Allowed range: 300 to 900. Default: 600. |
deadLetterQueueS3 | True | The Amazon S3 bucket and prefix where MSK writes records that fail to deliver. | |
destinationTableList | Array of type DestinationTable | True | The destination Iceberg tables. Currently exactly one table must be specified. |
schemaEvolution | True | Configuration controlling whether the destination table's schema is evolved to match incoming records. | |
serviceExecutionRoleArn | string | True | The Amazon Resource Name (ARN) of the IAM role that MSK assumes to access the destination table, the AWS Glue Data Catalog, and the dead-letter Amazon S3 bucket. |
tableCreation | True | Configuration controlling whether MSK creates the destination table if it does not already exist. |
ListChannelsResponse
Returns the list of channels in the cluster.
| Property | Type | Required | Description |
|---|---|---|---|
channels | Array of type ChannelInfo | False | The list of channels in the cluster. |
nextToken | string | False | If the response from ListChannels is truncated, this token is included. Send it as the nextToken parameter on a subsequent ListChannels call to retrieve the next page. |
PartitionSource
A source column used by an Apache Iceberg destination table's partition specification.
| Property | Type | Required | Description |
|---|---|---|---|
sourceName | string | False | The name of the source column. |
PartitionSpec
Partition specification for an Apache Iceberg destination table.
| Property | Type | Required | Description |
|---|---|---|---|
partitionStrategy | True | The partitioning strategy applied to records written to the table. | |
sourceList | Array of type PartitionSource | False | The source columns used by the partitioning strategy. For TIME_HOUR, must contain exactly one source column whose value is a timestamp. |
PartitionStrategy
The partitioning strategy used to partition records in the destination Apache Iceberg table.
TIME_HOUR
RecordConverter
Configuration that controls how Apache Kafka record values are deserialized for the destination.
| Property | Type | Required | Description |
|---|---|---|---|
valueConverter | True | The deserialization format applied to Apache Kafka record values. |
RecordSchema
Schema configuration that controls how Apache Kafka record values are validated.
| Property | Type | Required | Description |
|---|---|---|---|
gsrArn | string | True | The Amazon Resource Name (ARN) of the AWS Glue Schema Registry schema (not registry) used to validate records for the destination Apache Iceberg table. |
S3
The details of the Amazon S3 destination for broker logs.
| Property | Type | Required | Description |
|---|---|---|---|
bucket | string | False | The name of the S3 bucket that is the destination for broker logs. |
enabled | boolean | True | Specifies whether broker logs get sent to the specified Amazon S3 destination. |
prefix | string | False | The S3 prefix that is the destination for broker logs. |
S3CompressionType
The compression codec applied to delivered Amazon S3 objects.
NONEGZIPZSTD
S3DestinationConfiguration
Configuration of an Amazon S3 destination for a channel.
| Property | Type | Required | Description |
|---|---|---|---|
dataFreshnessInSeconds | integer | False | The maximum time, in seconds, that records buffer in MSK before being flushed to the destination. Allowed range: 300 to 900. Default: 600. |
deadLetterQueueS3 | True | The Amazon S3 bucket and prefix where MSK writes records that fail to deliver. | |
serviceExecutionRoleArn | string | True | The Amazon Resource Name (ARN) of the IAM role that MSK assumes to write to the destination Amazon S3 bucket and the dead-letter bucket. |
storage | True | The Amazon S3 bucket, prefix, and storage class for delivered records. |
S3Storage
Storage configuration for an Amazon S3 destination bucket.
| Property | Type | Required | Description |
|---|---|---|---|
bucketArn | string | True | The Amazon Resource Name (ARN) of the destination Amazon S3 bucket. |
compressionType | True | The compression codec applied to delivered Amazon S3 objects. | |
expectedBucketOwner | string | False | Optional 12-digit AWS account ID expected to own the Amazon S3 bucket. |
outputKeyTemplate | string | False | An optional template that controls the Amazon S3 object key for each delivered record. Supports the placeholders !{partition-id}, !{sequence-number}, and !{kafka-offset}. |
outputPrefix | string | False | An optional prefix prepended to every Amazon S3 object key written by the channel. |
storageClass | True | The Amazon S3 storage class for delivered objects. |
S3StorageClass
The Amazon S3 storage class applied to delivered objects.
STANDARDINTELLIGENT_TIERINGGLACIER_IR
SchemaEvolution
Configuration controlling whether the Apache Iceberg destination table's schema is evolved as incoming records change.
| Property | Type | Required | Description |
|---|---|---|---|
enableSchemaEvolution | boolean | False | Whether to allow MSK to evolve the destination table's schema. Must be false for the current release. |
TableCreation
Configuration controlling whether MSK creates the destination Apache Iceberg table if it does not already exist.
| Property | Type | Required | Description |
|---|---|---|---|
enableTableCreation | boolean | False | Whether MSK creates the destination table on the customer's behalf. Must be true for the current release. |
TopicConfiguration
Configuration of an Apache Kafka topic that feeds a channel.
| Property | Type | Required | Description |
|---|---|---|---|
recordConverter | True | Configuration that controls how Apache Kafka record values are deserialized for the destination. | |
recordSchema | False | The schema used to validate records when the value converter requires one (for example, JSON_SCHEMA_GSR). | |
topicArn | string | True | The Amazon Resource Name (ARN) that uniquely identifies the topic. |
ValueConverter
The deserialization format applied to Apache Kafka record values.
BYTE_ARRAYJSONJSON_SCHEMA_GSRSTRING
See also
For more information about using this API in one of the language-specific AWS SDKs and references, see the following: