Clusters
Represents the Amazon MSK clusters in an account.
URI
/api/v2/clusters
HTTP methods
GET
Operation ID: ListClustersV2
Lists all serverless and provisioned clusters.
Name | Type | Required | Description |
---|---|---|---|
nextToken | String | False | If the response of ListClusters is truncated, it returns a nextToken in the response. This nextToken should be sent in the subsequent request to ListClusters. |
clusterNameFilter | String | False | Returns clusters starting with given name. |
maxResults | String | False | Maximum number of clusters to fetch in one get request. |
clusterTypeFilter | String | False | Returns clusters with the given type. |
Status code | Response model | Description |
---|---|---|
200 |
ListClustersV2Response | HTTP Status Code 200: OK. |
400 | None | HTTP Status Code 400: Bad request due to incorrect input. Correct your request and then retry it. |
401 | None | HTTP Status Code 401: Unauthorized request. The provided credentials couldn't be validated. |
403 | None | HTTP Status Code 403: Access forbidden. Correct your credentials and then retry your request. |
404 | None | HTTP Status Code 404: Resource not found due to incorrect input. Correct your request and then retry it. |
429 | None | HTTP Status Code 429: Limit exceeded. Resource limit reached. |
500 | None | HTTP Status Code 500: Unexpected internal server error. Retrying your request might resolve the issue. |
503 | None | HTTP Status Code 503: Service Unavailable. Retrying your request in some time might resolve the issue. |
POST
Operation ID: CreateClusterV2
Creates a serverless or provisioned cluster.
Status code | Response model | Description |
---|---|---|
200 |
CreateClusterV2Response | HTTP Status Code 200: OK. |
400 | None | HTTP Status Code 400: Bad request due to incorrect input. Correct your request and then retry it. |
401 | None | HTTP Status Code 401: Unauthorized request. The provided credentials couldn't be validated. |
403 | None | HTTP Status Code 403: Access forbidden. Correct your credentials and then retry your request. |
404 | None | HTTP Status Code 404: Resource not found due to incorrect input. Correct your request and then retry it. |
409 | None | HTTP Status Code 409: Conflict. This cluster name already exists. Retry your request with another name. |
429 | None | HTTP Status Code 429: Limit exceeded. Resource limit reached. |
500 | None | HTTP Status Code 500: Unexpected internal server error. Retrying your request might resolve the issue. |
503 | None | HTTP Status Code 503: Service Unavailable. Retrying your request in some time might resolve the issue. |
OPTIONS
Enable CORS by returning correct headers
Status code | Response model | Description |
---|---|---|
200 | None | 200 response |
Schemas
Request bodies
{
"provisioned": {
"encryptionInfo": {
"encryptionInTransit": {
"inCluster": boolean,
"clientBroker": enum
},
"encryptionAtRest": {
"dataVolumeKMSKeyId": "string"
}
},
"numberOfBrokerNodes": integer,
"configurationInfo": {
"arn": "string",
"revision": integer
},
"enhancedMonitoring": enum,
"storageMode": enum,
"kafkaVersion": "string",
"clientAuthentication": {
"sasl": {
"iam": {
"enabled": boolean
},
"scram": {
"enabled": boolean
}
},
"unauthenticated": {
"enabled": boolean
},
"tls": {
"certificateAuthorityArnList": [
"string"
],
"enabled": boolean
}
},
"loggingInfo": {
"brokerLogs": {
"s3": {
"bucket": "string",
"prefix": "string",
"enabled": boolean
},
"firehose": {
"deliveryStream": "string",
"enabled": boolean
},
"cloudWatchLogs": {
"logGroup": "string",
"enabled": boolean
}
}
},
"brokerNodeGroupInfo": {
"clientSubnets": [
"string"
],
"zoneIds": [
"string"
],
"instanceType": "string",
"connectivityInfo": {
"vpcConnectivity": {
"clientAuthentication": {
"sasl": {
"iam": {
"enabled": boolean
},
"scram": {
"enabled": boolean
}
},
"tls": {
"enabled": boolean
}
}
},
"publicAccess": {
"type": "string"
}
},
"securityGroups": [
"string"
],
"brokerAZDistribution": enum,
"storageInfo": {
"ebsStorageInfo": {
"provisionedThroughput": {
"volumeThroughput": integer,
"enabled": boolean
},
"volumeSize": integer
}
}
},
"openMonitoring": {
"prometheus": {
"nodeExporter": {
"enabledInBroker": boolean
},
"jmxExporter": {
"enabledInBroker": boolean
}
}
}
},
"clusterName": "string",
"serverless": {
"vpcConfigs": [
{
"securityGroupIds": [
"string"
],
"subnetIds": [
"string"
]
}
],
"clientAuthentication": {
"sasl": {
"iam": {
"enabled": boolean
}
}
}
},
"tags": {
}
}
Response bodies
{
"nextToken": "string",
"clusterInfoList": [
{
"clusterType": enum,
"clusterArn": "string",
"activeOperationArn": "string",
"provisioned": {
"encryptionInfo": {
"encryptionInTransit": {
"inCluster": boolean,
"clientBroker": enum
},
"encryptionAtRest": {
"dataVolumeKMSKeyId": "string"
}
},
"zookeeperConnectString": "string",
"customerActionStatus": enum,
"zookeeperConnectStringTls": "string",
"loggingInfo": {
"brokerLogs": {
"s3": {
"bucket": "string",
"prefix": "string",
"enabled": boolean
},
"firehose": {
"deliveryStream": "string",
"enabled": boolean
},
"cloudWatchLogs": {
"logGroup": "string",
"enabled": boolean
}
}
},
"numberOfBrokerNodes": integer,
"enhancedMonitoring": enum,
"storageMode": enum,
"clientAuthentication": {
"sasl": {
"iam": {
"enabled": boolean
},
"scram": {
"enabled": boolean
}
},
"unauthenticated": {
"enabled": boolean
},
"tls": {
"certificateAuthorityArnList": [
"string"
],
"enabled": boolean
}
},
"brokerNodeGroupInfo": {
"clientSubnets": [
"string"
],
"zoneIds": [
"string"
],
"instanceType": "string",
"connectivityInfo": {
"vpcConnectivity": {
"clientAuthentication": {
"sasl": {
"iam": {
"enabled": boolean
},
"scram": {
"enabled": boolean
}
},
"tls": {
"enabled": boolean
}
}
},
"publicAccess": {
"type": "string"
}
},
"securityGroups": [
"string"
],
"brokerAZDistribution": enum,
"storageInfo": {
"ebsStorageInfo": {
"provisionedThroughput": {
"volumeThroughput": integer,
"enabled": boolean
},
"volumeSize": integer
}
}
},
"openMonitoring": {
"prometheus": {
"nodeExporter": {
"enabledInBroker": boolean
},
"jmxExporter": {
"enabledInBroker": boolean
}
}
},
"currentBrokerSoftwareInfo": {
"configurationRevision": integer,
"kafkaVersion": "string",
"configurationArn": "string"
}
},
"creationTime": "string",
"clusterName": "string",
"serverless": {
"vpcConfigs": [
{
"securityGroupIds": [
"string"
],
"subnetIds": [
"string"
]
}
],
"kafkaVersion": "string",
"clientAuthentication": {
"sasl": {
"iam": {
"enabled": boolean
}
}
}
},
"stateInfo": {
"code": "string",
"message": "string"
},
"state": enum,
"currentVersion": "string",
"tags": {
}
}
]
}
{
"clusterType": enum,
"clusterArn": "string",
"clusterName": "string",
"state": enum
}
Properties
BrokerAZDistribution
The distribution of broker nodes across Availability Zones.
DEFAULT
BrokerLogs
Broker Logs details for cluster.
Property | Type | Required | Description |
---|---|---|---|
cloudWatchLogs | False | CloudWatch Log destination details. | |
firehose | False | ||
s3 | False | S3 Log destination details. |
BrokerNodeGroupInfo
Describes the setup to be used for the brokers.
Property | Type | Required | Description |
---|---|---|---|
brokerAZDistribution | False | The distribution of broker nodes across Availability Zones. | |
clientSubnets | Array of type string | True | The list of subnets in the client VPC to connect to. |
connectivityInfo | False | Information about the cluster access configuration. | |
instanceType | string MinLength: 5 MaxLength: 32 | True | The type of broker used for the cluster. |
securityGroups | Array of type string | False | The security groups to attach to the ENIs for the broker nodes. |
storageInfo | False | Data volume information. | |
zoneIds | Array of type string | False | The zoneIds for brokers in customer account. |
BrokerSoftwareInfo
Information about current software installed in the cluster.
Property | Type | Required | Description |
---|---|---|---|
configurationArn | string | False | ARN of the configuration used on the cluster. |
configurationRevision | integer Format: int64 | False | Revision of the configuration to use. |
kafkaVersion | string | False | The version of Apache Kafka to install and run on the cluster. |
ClientAuthentication
Includes all client authentication information.
Property | Type | Required | Description |
---|---|---|---|
sasl | False | Details for ClientAuthentication using SASL. | |
tls | False | Details for ClientAuthentication using TLS. | |
unauthenticated | False | Details for ClientAuthentication using no authentication. |
ClientBroker
Client-broker encryption in transit setting.
TLS
TLS_PLAINTEXT
PLAINTEXT
CloudWatchLogs
CloudWatchLogs details for BrokerLogs.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | True | Broker logs for destination CW enabled or not. |
logGroup | string | False | CloudWatch LogGroup where the logs will be delivered. |
Cluster
Returns information about a cluster.
Property | Type | Required | Description |
---|---|---|---|
activeOperationArn | string | False | Arn of active cluster operation. |
clusterArn | string | False | The Amazon Resource Name (ARN) of the cluster. |
clusterName | string | False | The name of the cluster. |
clusterType | False | Type of the backend cluster. | |
creationTime | string | False | The time when the cluster was created. |
currentVersion | string | False | Current version of cluster. |
provisioned | False | Properties of a provisioned cluster. | |
serverless | False | Properties of a serverless cluster. | |
state | False | State of the cluster. | |
stateInfo | False | Includes information of the cluster state. | |
tags | object | False | Tags attached to the cluster. |
ClusterState
The sate of an MSK cluster.
ACTIVE
CREATING
UPDATING
DELETING
FAILED
MAINTENANCE
REBOOTING_BROKER
HEALING
ClusterType
The type of backend cluster.
PROVISIONED
SERVERLESS
ConfigurationInfo
Specifies the configuration to be used on the brokers.
Property | Type | Required | Description |
---|---|---|---|
arn | string | True | ARN of the configuration to use. |
revision | integer Format: int64 Minimum: 1 | True | The revision of the configuration to use. |
ConnectivityInfo
Broker access controls
Property | Type | Required | Description |
---|---|---|---|
publicAccess | False | Access control settings for brokers | |
vpcConnectivity | False | VPC connection control settings for brokers |
CreateClusterV2Request
Creates a provisioned or serverless cluster.
Property | Type | Required | Description |
---|---|---|---|
clusterName | string MinLength: 1 MaxLength: 64 | True | The name of the cluster. |
provisioned | False | Properties required for creating a provisioned cluster. | |
serverless | False | Properties required for creating a serverless cluster. | |
tags | object | False | Create tags when creating the cluster. |
CreateClusterV2Response
Returns information about the created cluster.
Property | Type | Required | Description |
---|---|---|---|
clusterArn | string | False | The Amazon Resource Name (ARN) of the cluster. |
clusterName | string | False | Name of the cluster provided by the customer. |
clusterType | False | Type of the backend cluster. | |
state | False | State of the cluster. |
CustomerActionStatus
A type of an action required from the customer.
CRITICAL_ACTION_REQUIRED
ACTION_RECOMMENDED
NONE
EBSStorageInfo
Contains information about the EBS storage volumes that are attached to the brokers.
Property | Type | Required | Description |
---|---|---|---|
provisionedThroughput | False | EBS volume provisioned throughput information. | |
volumeSize | integer Minimum: 1 Maximum: 16384 | False | The size of the EBS volumes for the data drive on each of the brokers in GiB. |
EncryptionAtRest
Details for encryption at rest.
Property | Type | Required | Description |
---|---|---|---|
dataVolumeKMSKeyId | string | True | KMS key used for data volume encryption. |
EncryptionInTransit
Details for encryption in transit.
Property | Type | Required | Description |
---|---|---|---|
clientBroker | False | Client-broker encryption in transit setting. | |
inCluster | boolean | False | In-cluster encryption in transit setting. |
EncryptionInfo
Includes all encryption related information.
Property | Type | Required | Description |
---|---|---|---|
encryptionAtRest | False | Details for encryption at rest. | |
encryptionInTransit | False | Details for encryption in transit. |
EnhancedMonitoring
Controls level of cluster metrics Amazon pushes to customer's cloudwatch account.
DEFAULT
PER_BROKER
PER_TOPIC_PER_BROKER
PER_TOPIC_PER_PARTITION
Firehose
Firehose details for BrokerLogs.
Property | Type | Required | Description |
---|---|---|---|
deliveryStream | string | False | Firehose delivery stream where the logs will be delivered. |
enabled | boolean | True | Broker logs for destination firehose enabled or not. |
IAM
Details for SASL/IAM client authentication.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | SASL/IAM authentication is enabled or not. |
JmxExporter
JMX Exporter details.
Property | Type | Required | Description |
---|---|---|---|
enabledInBroker | boolean | True | JMX Exporter being enabled in broker. |
JmxExporterInfo
JMX Exporter details.
Property | Type | Required | Description |
---|---|---|---|
enabledInBroker | boolean | True | JMX Exporter being enabled in broker. |
ListClustersV2Response
The response contains an array of cluster information and a nextToken if the response is truncated.
Property | Type | Required | Description |
---|---|---|---|
clusterInfoList | Array of type Cluster | False | An array of clusters. |
nextToken | string | False | If the response of ListClustersV2 is truncated, it returns a NextToken in the response. Pass this NextToken in a subsequent request to ListClustersV2 to get the next set of results. |
LoggingInfo
Logging info details for the cluster.
Property | Type | Required | Description |
---|---|---|---|
brokerLogs | True | Broker Logs details. |
NodeExporter
Node Exporter details.
Property | Type | Required | Description |
---|---|---|---|
enabledInBroker | boolean | True | Node Exporter being enabled in broker. |
NodeExporterInfo
Node Exporter details.
Property | Type | Required | Description |
---|---|---|---|
enabledInBroker | boolean | True | Node Exporter being enabled in broker. |
OpenMonitoring
JMX and Node monitoring for cluster.
Property | Type | Required | Description |
---|---|---|---|
prometheus | True | Prometheus details. |
OpenMonitoringInfo
JMX and Node monitoring for the cluster.
Property | Type | Required | Description |
---|---|---|---|
prometheus | True | Prometheus details. |
Prometheus
Prometheus details.
Property | Type | Required | Description |
---|---|---|---|
jmxExporter | False | JMX Exporter details. | |
nodeExporter | False | Node Exporter details. |
PrometheusInfo
Prometheus details.
Property | Type | Required | Description |
---|---|---|---|
jmxExporter | False | JMX Exporter details. | |
nodeExporter | False | Node Exporter details. |
Provisioned
Properties of a provisioned cluster.
Property | Type | Required | Description |
---|---|---|---|
brokerNodeGroupInfo | False | Information about the brokers of the cluster. | |
clientAuthentication | False | Includes all client authentication information. | |
currentBrokerSoftwareInfo | False | Information about the version of the software that is deployed on the brokers of the cluster. | |
customerActionStatus | False | Determines if there is an action required from the customer. | |
encryptionInfo | False | Includes all encryption related information. | |
enhancedMonitoring | False | This knob controls level of metrics pushed customer's cloudwatch account. | |
loggingInfo | False | Logging Info details. | |
numberOfBrokerNodes | integer | False | The number of brokers to create in the cluster. |
openMonitoring | False | Open monitoring details. | |
storageMode | False | This controls storage mode for supported storage tiers. | |
zookeeperConnectString | string | False | The connection string to use to connect to zookeeper cluster on plaintext port. |
zookeeperConnectStringTls | string | False | The connection string to use to connect to zookeeper cluster on Tls port. |
ProvisionedRequest
Properties for creating a provisioned cluster.
Property | Type | Required | Description |
---|---|---|---|
brokerNodeGroupInfo | False | Information about the brokers of the cluster. | |
clientAuthentication | False | Includes all client authentication related information. | |
configurationInfo | False | The configuration to be used on the brokers. | |
encryptionInfo | False | Includes all encryption related information. | |
enhancedMonitoring | False | Specifies the monitoring level for the cluster. | |
kafkaVersion | string MinLength: 1 MaxLength: 128 | False | The version of Apache Kafka to install and run on the cluster. |
loggingInfo | False | Logging Info details. | |
numberOfBrokerNodes | integer | False | The number of brokers to create in the cluster. |
openMonitoring | False | The open-monitoring details. | |
storageMode | False | Specifies the storage mode for supported storage tiers. |
ProvisionedThroughput
Contains information about provisioned throughput for the EBS storage volumes that are attached to the brokers.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | Whether provisioned throughput is turned on. |
volumeThroughput | integer | False | Throughput value of the EBS volumes for the data drive on each broker in MiB per second. |
PublicAccess
Broker access controls
Property | Type | Required | Description |
---|---|---|---|
type | string | False | If public access is disabled, or if enabled the EIP provider |
S3
S3 details for BrokerLogs.
Property | Type | Required | Description |
---|---|---|---|
bucket | string | False | Name of the bucket where the logs will be delivered. |
enabled | boolean | True | Broker logs for destination S3 enabled or not. |
prefix | string | False | prefix to the S3 bucket where the logs will be delivered. |
Sasl
Details for client authentication using SASL.
Scram
Details for SASL/SCRAM client authentication.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | SASL/SCRAM authentication is enabled or not. |
Serverless
Properties to create a serverless cluster
Property | Type | Required | Description |
---|---|---|---|
clientAuthentication | True | Includes all client authentication related information. | |
kafkaVersion | string | False | The version of Apache Kafka for the serverless cluster. |
vpcConfigs | Array of type VpcConfig | True | VPC configuration information |
ServerlessClientAuthentication
Details for client authentication using SASL.
Property | Type | Required | Description |
---|---|---|---|
sasl | False | Details for ClientAuthentication using IAM. |
ServerlessRequest
Properties for creating a serverless cluster.
Property | Type | Required | Description |
---|---|---|---|
clientAuthentication | True | Includes all client authentication related information. | |
vpcConfigs | Array of type VpcConfig | True | VPC configuration information |
ServerlessSasl
Details for client authentication using SASL for Serverless Cluster.
Property | Type | Required | Description |
---|---|---|---|
iam | False | Details for ClientAuthentication using IAM for Serverless Cluster. |
StateInfo
Includes information about the cluster state.
Property | Type | Required | Description |
---|---|---|---|
code | string | False | Code for cluster state. |
message | string | False | Message for cluster state. |
StorageInfo
Contains information about the storage volumes that are attached to the brokers.
Property | Type | Required | Description |
---|---|---|---|
ebsStorageInfo | False | EBS volume information. |
StorageMode
Controls storage mode for various supported storage tiers.
LOCAL
TIERED
Tls
The details of client authentication using TLS.
Property | Type | Required | Description |
---|---|---|---|
certificateAuthorityArnList | Array of type string | False | List of ACM CertificateAuthority ARNs. |
enabled | boolean | False | Whether TLS authentication is turned on. |
Unauthenticated
Details for allowing no client authentication.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | Unauthenticated is enabled or not. |
VpcConfig
Includes information about subnets and security groups for a VPC.
Property | Type | Required | Description |
---|---|---|---|
securityGroupIds | Array of type string | False | The security groups to attach to the ENIs for the broker nodes. |
subnetIds | Array of type string | True | The list of subnets in the client VPC to connect to. Client subnets can't occupy the Availability Zone with ID use1-az3. |
VpcConnectivity
VPC connection control settings for brokers
Property | Type | Required | Description |
---|---|---|---|
clientAuthentication | False | VPC connection control settings for brokers |
VpcConnectivityClientAuthentication
Includes all client authentication information for VpcConnectivity.
Property | Type | Required | Description |
---|---|---|---|
sasl | False | Details for VpcConnectivity ClientAuthentication using SASL. | |
tls | False | Details for VpcConnectivity ClientAuthentication using TLS. |
VpcConnectivityIAM
Details for SASL/IAM client authentication for VpcConnectivity.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | SASL/IAM authentication is enabled or not. |
VpcConnectivitySasl
Details for client authentication using SASL for VpcConnectivity.
Property | Type | Required | Description |
---|---|---|---|
iam | False | Details for ClientAuthentication using IAM for VpcConnectivity. | |
scram | False | Details for SASL/SCRAM client authentication for VpcConnectivity. |
VpcConnectivityScram
Details for SASL/SCRAM client authentication for vpcConnectivity.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | SASL/SCRAM authentication is enabled or not. |
VpcConnectivityTls
Details for client authentication using TLS for vpcConnectivity.
Property | Type | Required | Description |
---|---|---|---|
enabled | boolean | False | TLS authentication is enabled or not. |
See also
For more information about using this API in one of the language-specific AWS SDKs and references, see the following: