Monitoring Properties

The options for monitoring an Amazon MSK cluster. You can specify which Apache Kafka metrics you want Amazon MSK to gather and send to Amazon CloudWatch. You can also configure open monitoring to gather metrics with Prometheus or Prometheus-compatible tools.

URI

/v1/clusters/clusterArn/monitoring

HTTP methods

OPTIONS

Enable CORS by returning the correct headers.

PUT

Operation ID: UpdateMonitoring

Updates the monitoring settings for the cluster. You can use this operation to specify which Apache Kafka metrics you want Amazon MSK to send to Amazon CloudWatch. You can also specify settings for open monitoring with Prometheus. The following Python 3.6 example enables open monitoring with the Node Exporter. It also sets enhanced monitoring to PER_BROKER. For more information about monitoring, see Monitoring.

import boto3
import time

client = boto3.client('kafka')

update_monitoring_response = client.update_monitoring(
    ClusterArn='arn:aws:kafka:us-east-1:0123456789019:cluster/SalesCluster/abcd1234-abcd-cafe-abab-9876543210ab-4',
    CurrentVersion='K12V3IB1VIZHHY',
    EnhancedMonitoring='PER_BROKER',
    OpenMonitoring={"Prometheus":{"JmxExporter":{"EnabledInBroker":False},"NodeExporter":{"EnabledInBroker":True}}}
)

operation_arn = update_monitoring_response['ClusterOperationArn']
print('The ARN of the update operation is ' + operation_arn)

describe_cluster_operation_response = client.describe_cluster_operation(ClusterOperationArn=operation_arn)

operation_state = describe_cluster_operation_response['ClusterOperationInfo']['OperationState']
print('The status of the update operation is ' + operation_state)

updated = False

while not updated:
    print('Sleeping for 15 seconds before checking to see if the monitoring update is done...')
    time.sleep(15)
    describe_cluster_operation_response = client.describe_cluster_operation(ClusterOperationArn=operation_arn)
    operation_state = describe_cluster_operation_response['ClusterOperationInfo']['OperationState']
    if 'UPDATE_COMPLETE' == operation_state:
        updated = True
        print('You have successfully updated the monitoring settings.')

Schemas

Request bodies

{
  "currentVersion": "string",
  "enhancedMonitoring": enum,
  "loggingInfo": {
    "brokerLogs": {
      "cloudWatchLogs": {
        "enabled": boolean,
        "logGroup": "string"
      },
      "firehose": {
        "deliveryStream": "string",
        "enabled": boolean
      },
      "s3": {
        "bucket": "string",
        "enabled": boolean,
        "prefix": "string"
      }
    }
  },
  "openMonitoring": {
    "prometheus": {
      "jmxExporter": {
        "enabledInBroker": boolean
      },
      "nodeExporter": {
        "enabledInBroker": boolean
      }
    }
  }
}

Response bodies

{
  "clusterArn": "string",
  "clusterOperationArn": "string"
}

{
  "invalidParameter": "string",
  "message": "string"
}

Properties

BrokerLogs

The broker logs configuration for this MSK cluster.

CloudWatchLogs

Details of the CloudWatch Logs destination for broker logs.

EnhancedMonitoring

Specifies which Apache Kafka metrics Amazon MSK gathers and sends to Amazon CloudWatch for this cluster. This property has three possible values: DEFAULT, PER_BROKER, and PER_TOPIC_PER_BROKER. For a list of the metrics associated with each of these three levels of monitoring, see Monitoring.

Error

Returns information about an error.

Firehose

Firehose details for BrokerLogs.

JmxExporterInfo

JMX Exporter details.

LoggingInfo

You can configure your MSK cluster to send broker logs to different destination types. This is a container for the configuration details related to broker logs.

NodeExporterInfo

Node Exporter details.

OpenMonitoringInfo

JMX and Node monitoring for cluster.

PrometheusInfo

Prometheus details.

S3

The details of the Amazon S3 destination for broker logs.

UpdateMonitoringRequest

Request body for UpdateMonitoring.

UpdateMonitoringResponse

Request body for UpdateMonitoring.

See also

For more information about using this API in one of the language-specific AWS SDKs and references, see the following:

UpdateMonitoring