Monitoring consumer lag allows you to identify slow or stuck consumers that aren't keeping up with the latest data available in a topic. When necessary, you can then take remedial actions, such as scaling or rebooting those consumers. To monitor consumer lag, you can use Amazon CloudWatch or open monitoring with Prometheus.
Consumer lag metrics quantify the difference between the latest data written to your
topics and the data read by your applications. Amazon MSK provides the following consumer-lag
metrics, which you can get through Amazon CloudWatch or through open monitoring with Prometheus:
EstimatedMaxTimeLag, EstimatedTimeLag,
MaxOffsetLag, OffsetLag, and SumOffsetLag.
For information about these metrics, see Amazon MSK metrics for monitoring Standard brokers with CloudWatch.
Note
Consumer-lag metrics are visible only for consumer groups in a STABLE state. A consumer group is STABLE after the successful completion of re-balancing, ensuring that partitions are evenly distributed among the consumers.
Amazon MSK supports consumer lag metrics for clusters with Apache Kafka 2.2.1 or a later version.
