# Container Insights 性能日志参考
本节包含有关 Container Insights 如何使用性能日志事件来收集指标的参考信息。当您部署 Container Insights 时，它会自动为性能日志事件创建日志组。您无需自行创建该日志组。
**Topics**
+ [Amazon ECS 的 Container Insights 性能日志事件](Container-Insights-reference-performance-logs-ECS.md)
+ [Amazon EKS 和 Kubernetes 的 Container Insights 性能日志事件](Container-Insights-reference-performance-logs-EKS.md)
+ [Amazon EKS 和 Kubernetes 的性能日志事件中的相关字段](Container-Insights-reference-performance-entries-EKS.md)
# Amazon ECS 的 Container Insights 性能日志事件
以下是 Container Insights 从 Amazon ECS 收集的性能日志事件的实例。
这些日志在 CloudWatch Logs 中位于名为 `/aws/ecs/containerinsights/CLUSTER_NAME/performance` 的日志组中。在该日志组中，每个容器实例都将有一个名为 `AgentTelemetry-CONTAINER_INSTANCE_ID` 的日志流。
您可以使用查询来查询这些日志，例如使用 `{ $.Type = "Container" }` 来查看所有容器日志事件。
**类型：容器**
```
{
"Version":"0",
"Type":"Container",
"ContainerName":"sleep",
"TaskId":"7ac4dfba69214411b4783a3b8189c9ba",
"TaskDefinitionFamily":"sleep360",
"TaskDefinitionRevision":"1",
"ContainerInstanceId":"0d7650e6dec34c1a9200f72098071e8f",
"EC2InstanceId":"i-0c470579dbcdbd2f3",
"ClusterName":"MyCluster",
"Image":"busybox",
"ContainerKnownStatus":"RUNNING",
"Timestamp":1623963900000,
"CpuUtilized":0.0,
"CpuReserved":10.0,
"MemoryUtilized":0,
"MemoryReserved":10,
"StorageReadBytes":0,
"StorageWriteBytes":0,
"NetworkRxBytes":0,
"NetworkRxDropped":0,
"NetworkRxErrors":0,
"NetworkRxPackets":14,
"NetworkTxBytes":0,
"NetworkTxDropped":0,
"NetworkTxErrors":0,
"NetworkTxPackets":0
}
```
**类型：任务**
尽管 `StorageReadBytes` 和 `StorageWriteBytes` 以字节/秒为单位，但这些值分别表示从存储中读取和写入存储的累积字节数。
```
{
"Version": "0",
"Type": "Task",
"TaskId": "7ac4dfba69214411b4783a3b8189c9ba",
"TaskDefinitionFamily": "sleep360",
"TaskDefinitionRevision": "1",
"ContainerInstanceId": "0d7650e6dec34c1a9200f72098071e8f",
"EC2InstanceId": "i-0c470579dbcdbd2f3",
"ClusterName": "MyCluster",
"AccountID": "637146863587",
"Region": "us-west-2",
"AvailabilityZone": "us-west-2b",
"KnownStatus": "RUNNING",
"LaunchType": "EC2",
"PullStartedAt": 1623963608201,
"PullStoppedAt": 1623963610065,
"CreatedAt": 1623963607094,
"StartedAt": 1623963610382,
"Timestamp": 1623963900000,
"CpuUtilized": 0.0,
"CpuReserved": 10.0,
"MemoryUtilized": 0,
"MemoryReserved": 10,
"StorageReadBytes": 0,
"StorageWriteBytes": 0,
"NetworkRxBytes": 0,
"NetworkRxDropped": 0,
"NetworkRxErrors": 0,
"NetworkRxPackets": 14,
"NetworkTxBytes": 0,
"NetworkTxDropped": 0,
"NetworkTxErrors": 0,
"NetworkTxPackets": 0,
"EBSFilesystemUtilized": 10,
"EBSFilesystemSize": 20,
"CloudWatchMetrics": [
{
"Namespace": "ECS/ContainerInsights",
"Metrics": [
{
"Name": "CpuUtilized",
"Unit": "None"
},
{
"Name": "CpuReserved",
"Unit": "None"
},
{
"Name": "MemoryUtilized",
"Unit": "Megabytes"
},
{
"Name": "MemoryReserved",
"Unit": "Megabytes"
},
{
"Name": "StorageReadBytes",
"Unit": "Bytes/Second"
},
{
"Name": "StorageWriteBytes",
"Unit": "Bytes/Second"
},
{
"Name": "NetworkRxBytes",
"Unit": "Bytes/Second"
},
{
"Name": "NetworkTxBytes",
"Unit": "Bytes/Second"
},
{
"Name": "EBSFilesystemSize",
"Unit": "Gigabytes"
},
{
"Name": "EBSFilesystemUtilzed",
"Unit": "Gigabytes"
}
],
"Dimensions": [
["ClusterName"],
[
"ClusterName",
"TaskDefinitionFamily"
]
]
}
]
}
```
**类型：服务**
```
{
"Version": "0",
"Type": "Service",
"ServiceName": "myCIService",
"ClusterName": "myCICluster",
"Timestamp": 1561586460000,
"DesiredTaskCount": 2,
"RunningTaskCount": 2,
"PendingTaskCount": 0,
"DeploymentCount": 1,
"TaskSetCount": 0,
"CloudWatchMetrics": [
{
"Namespace": "ECS/ContainerInsights",
"Metrics": [
{
"Name": "DesiredTaskCount",
"Unit": "Count"
},
{
"Name": "RunningTaskCount",
"Unit": "Count"
},
{
"Name": "PendingTaskCount",
"Unit": "Count"
},
{
"Name": "DeploymentCount",
"Unit": "Count"
},
{
"Name": "TaskSetCount",
"Unit": "Count"
}
],
"Dimensions": [
[
"ServiceName",
"ClusterName"
]
]
}
]
}
```
**类型：卷**
```
{
"Version": "0",
"Type": "Volume",
"TaskDefinitionFamily": "myCITaskDef",
"TaskId": "7ac4dfba69214411b4783a3b8189c9ba",
"ClusterName": "myCICluster",
"ServiceName": "myCIService",
"VolumeId": "vol-1233436545ff708cb",
"InstanceId": "i-0c470579dbcdbd2f3",
"LaunchType": "EC2",
"VolumeName": "MyVolumeName",
"EBSFilesystemUtilized": 10,
"EBSFilesystemSize": 20,
"CloudWatchMetrics": [
{
"Namespace": "ECS/ContainerInsights",
"Metrics": [
{
"Name": "EBSFilesystemSize",
"Unit": "Gigabytes"
},
{
"Name": "EBSFilesystemUtilzed",
"Unit": "Gigabytes"
}
],
"Dimensions": [
["ClusterName"],
[
"VolumeName",
"TaskDefinitionFamily",
"ClusterName"
],
[
"ServiceName",
"ClusterName"
]
]
}
]
}
```
**类型：集群**
```
{
"Version": "0",
"Type": "Cluster",
"ClusterName": "myCICluster",
"Timestamp": 1561587300000,
"TaskCount": 5,
"ContainerInstanceCount": 5,
"ServiceCount": 2,
"CloudWatchMetrics": [
{
"Namespace": "ECS/ContainerInsights",
"Metrics": [
{
"Name": "TaskCount",
"Unit": "Count"
},
{
"Name": "ContainerInstanceCount",
"Unit": "Count"
},
{
"Name": "ServiceCount",
"Unit": "Count"
}
],
"Dimensions": [
[
"ClusterName"
]
]
}
]
}
```
# Amazon EKS 和 Kubernetes 的 Container Insights 性能日志事件
以下是 Container Insights 从 Amazon EKS 和 Kubernetes 集群中收集的性能日志事件的示例。
**类型：Node**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Percent",
"Name": "node_cpu_utilization"
},
{
"Unit": "Percent",
"Name": "node_memory_utilization"
},
{
"Unit": "Bytes/Second",
"Name": "node_network_total_bytes"
},
{
"Unit": "Percent",
"Name": "node_cpu_reserved_capacity"
},
{
"Unit": "Percent",
"Name": "node_memory_reserved_capacity"
},
{
"Unit": "Count",
"Name": "node_number_of_running_pods"
},
{
"Unit": "Count",
"Name": "node_number_of_running_containers"
}
],
"Dimensions": [
[
"NodeName",
"InstanceId",
"ClusterName"
]
],
"Namespace": "ContainerInsights"
},
{
"Metrics": [
{
"Unit": "Percent",
"Name": "node_cpu_utilization"
},
{
"Unit": "Percent",
"Name": "node_memory_utilization"
},
{
"Unit": "Bytes/Second",
"Name": "node_network_total_bytes"
},
{
"Unit": "Percent",
"Name": "node_cpu_reserved_capacity"
},
{
"Unit": "Percent",
"Name": "node_memory_reserved_capacity"
},
{
"Unit": "Count",
"Name": "node_number_of_running_pods"
},
{
"Unit": "Count",
"Name": "node_number_of_running_containers"
},
{
"Name": "node_cpu_usage_total"
},
{
"Name": "node_cpu_limit"
},
{
"Unit": "Bytes",
"Name": "node_memory_working_set"
},
{
"Unit": "Bytes",
"Name": "node_memory_limit"
}
],
"Dimensions": [
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
}
],
"ClusterName": "myCICluster",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"Sources": [
"cadvisor",
"/proc",
"pod",
"calculated"
],
"Timestamp": "1567096682364",
"Type": "Node",
"Version": "0",
"kubernetes": {
"host": "ip-192-168-75-26.us-west-2.compute.internal"
},
"node_cpu_limit": 4000,
"node_cpu_request": 1130,
"node_cpu_reserved_capacity": 28.249999999999996,
"node_cpu_usage_system": 33.794636630852764,
"node_cpu_usage_total": 136.47852169244098,
"node_cpu_usage_user": 71.67075111567326,
"node_cpu_utilization": 3.4119630423110245,
"node_memory_cache": 3103297536,
"node_memory_failcnt": 0,
"node_memory_hierarchical_pgfault": 0,
"node_memory_hierarchical_pgmajfault": 0,
"node_memory_limit": 16624865280,
"node_memory_mapped_file": 406646784,
"node_memory_max_usage": 4230746112,
"node_memory_pgfault": 0,
"node_memory_pgmajfault": 0,
"node_memory_request": 1115684864,
"node_memory_reserved_capacity": 6.7109407818311055,
"node_memory_rss": 798146560,
"node_memory_swap": 0,
"node_memory_usage": 3901444096,
"node_memory_utilization": 6.601302600149552,
"node_memory_working_set": 1097457664,
"node_network_rx_bytes": 35918.392817386324,
"node_network_rx_dropped": 0,
"node_network_rx_errors": 0,
"node_network_rx_packets": 157.67565245448117,
"node_network_total_bytes": 68264.20276554905,
"node_network_tx_bytes": 32345.80994816272,
"node_network_tx_dropped": 0,
"node_network_tx_errors": 0,
"node_network_tx_packets": 154.21455923431654,
"node_number_of_running_containers": 16,
"node_number_of_running_pods": 13
}
```
**类型：NodeFS**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Percent",
"Name": "node_filesystem_utilization"
}
],
"Dimensions": [
[
"NodeName",
"InstanceId",
"ClusterName"
],
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
}
],
"ClusterName": "myCICluster",
"EBSVolumeId": "aws://us-west-2b/vol-0a53108976d4a2fda",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"Sources": [
"cadvisor",
"calculated"
],
"Timestamp": "1567097939726",
"Type": "NodeFS",
"Version": "0",
"device": "/dev/nvme0n1p1",
"fstype": "vfs",
"kubernetes": {
"host": "ip-192-168-75-26.us-west-2.compute.internal"
},
"node_filesystem_available": 17298395136,
"node_filesystem_capacity": 21462233088,
"node_filesystem_inodes": 10484720,
"node_filesystem_inodes_free": 10367158,
"node_filesystem_usage": 4163837952,
"node_filesystem_utilization": 19.400767547940255
}
```
**类型：NodeDiskIO**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"ClusterName": "myCICluster",
"EBSVolumeId": "aws://us-west-2b/vol-0a53108976d4a2fda",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"Sources": [
"cadvisor"
],
"Timestamp": "1567096928131",
"Type": "NodeDiskIO",
"Version": "0",
"device": "/dev/nvme0n1",
"kubernetes": {
"host": "ip-192-168-75-26.us-west-2.compute.internal"
},
"node_diskio_io_service_bytes_async": 9750.505814277016,
"node_diskio_io_service_bytes_read": 0,
"node_diskio_io_service_bytes_sync": 230.6174506688036,
"node_diskio_io_service_bytes_total": 9981.123264945818,
"node_diskio_io_service_bytes_write": 9981.123264945818,
"node_diskio_io_serviced_async": 1.153087253344018,
"node_diskio_io_serviced_read": 0,
"node_diskio_io_serviced_sync": 0.03603397666700056,
"node_diskio_io_serviced_total": 1.1891212300110185,
"node_diskio_io_serviced_write": 1.1891212300110185
}
```
**类型：NodeNet**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"ClusterName": "myCICluster",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"Sources": [
"cadvisor",
"calculated"
],
"Timestamp": "1567096928131",
"Type": "NodeNet",
"Version": "0",
"interface": "eni972f6bfa9a0",
"kubernetes": {
"host": "ip-192-168-75-26.us-west-2.compute.internal"
},
"node_interface_network_rx_bytes": 3163.008420864309,
"node_interface_network_rx_dropped": 0,
"node_interface_network_rx_errors": 0,
"node_interface_network_rx_packets": 16.575629266820258,
"node_interface_network_total_bytes": 3518.3935157426017,
"node_interface_network_tx_bytes": 355.385094878293,
"node_interface_network_tx_dropped": 0,
"node_interface_network_tx_errors": 0,
"node_interface_network_tx_packets": 3.9997714100370625
}
```
**类型：Pod**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Percent",
"Name": "pod_cpu_utilization"
},
{
"Unit": "Percent",
"Name": "pod_memory_utilization"
},
{
"Unit": "Bytes/Second",
"Name": "pod_network_rx_bytes"
},
{
"Unit": "Bytes/Second",
"Name": "pod_network_tx_bytes"
},
{
"Unit": "Percent",
"Name": "pod_cpu_utilization_over_pod_limit"
},
{
"Unit": "Percent",
"Name": "pod_memory_utilization_over_pod_limit"
}
],
"Dimensions": [
[
"PodName",
"Namespace",
"ClusterName"
],
[
"Service",
"Namespace",
"ClusterName"
],
[
"Namespace",
"ClusterName"
],
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
},
{
"Metrics": [
{
"Unit": "Percent",
"Name": "pod_cpu_reserved_capacity"
},
{
"Unit": "Percent",
"Name": "pod_memory_reserved_capacity"
}
],
"Dimensions": [
[
"PodName",
"Namespace",
"ClusterName"
],
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
},
{
"Metrics": [
{
"Unit": "Count",
"Name": "pod_number_of_container_restarts"
}
],
"Dimensions": [
[
"PodName",
"Namespace",
"ClusterName"
]
],
"Namespace": "ContainerInsights"
}
],
"ClusterName": "myCICluster",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"Namespace": "amazon-cloudwatch",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"PodName": "cloudwatch-agent-statsd",
"Service": "cloudwatch-agent-statsd",
"Sources": [
"cadvisor",
"pod",
"calculated"
],
"Timestamp": "1567097351092",
"Type": "Pod",
"Version": "0",
"kubernetes": {
"host": "ip-192-168-75-26.us-west-2.compute.internal",
"labels": {
"app": "cloudwatch-agent-statsd",
"pod-template-hash": "df44f855f"
},
"namespace_name": "amazon-cloudwatch",
"pod_id": "2f4ff5ac-c813-11e9-a31d-06e9dde32928",
"pod_name": "cloudwatch-agent-statsd-df44f855f-ts4q2",
"pod_owners": [
{
"owner_kind": "Deployment",
"owner_name": "cloudwatch-agent-statsd"
}
],
"service_name": "cloudwatch-agent-statsd"
},
"pod_cpu_limit": 200,
"pod_cpu_request": 200,
"pod_cpu_reserved_capacity": 5,
"pod_cpu_usage_system": 1.4504841104992765,
"pod_cpu_usage_total": 5.817016867430125,
"pod_cpu_usage_user": 1.1281543081661038,
"pod_cpu_utilization": 0.14542542168575312,
"pod_cpu_utilization_over_pod_limit": 2.9085084337150624,
"pod_memory_cache": 8192,
"pod_memory_failcnt": 0,
"pod_memory_hierarchical_pgfault": 0,
"pod_memory_hierarchical_pgmajfault": 0,
"pod_memory_limit": 104857600,
"pod_memory_mapped_file": 0,
"pod_memory_max_usage": 25268224,
"pod_memory_pgfault": 0,
"pod_memory_pgmajfault": 0,
"pod_memory_request": 104857600,
"pod_memory_reserved_capacity": 0.6307275170893897,
"pod_memory_rss": 22777856,
"pod_memory_swap": 0,
"pod_memory_usage": 25141248,
"pod_memory_utilization": 0.10988455961791709,
"pod_memory_utilization_over_pod_limit": 17.421875,
"pod_memory_working_set": 18268160,
"pod_network_rx_bytes": 9880.697124714186,
"pod_network_rx_dropped": 0,
"pod_network_rx_errors": 0,
"pod_network_rx_packets": 107.80005532263283,
"pod_network_total_bytes": 10158.829201483635,
"pod_network_tx_bytes": 278.13207676944796,
"pod_network_tx_dropped": 0,
"pod_network_tx_errors": 0,
"pod_network_tx_packets": 1.146027574644318,
"pod_number_of_container_restarts": 0,
"pod_number_of_containers": 1,
"pod_number_of_running_containers": 1,
"pod_status": "Running"
}
```
**类型：PodNet**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"ClusterName": "myCICluster",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"Namespace": "amazon-cloudwatch",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"PodName": "cloudwatch-agent-statsd",
"Service": "cloudwatch-agent-statsd",
"Sources": [
"cadvisor",
"calculated"
],
"Timestamp": "1567097351092",
"Type": "PodNet",
"Version": "0",
"interface": "eth0",
"kubernetes": {
"host": "ip-192-168-75-26.us-west-2.compute.internal",
"labels": {
"app": "cloudwatch-agent-statsd",
"pod-template-hash": "df44f855f"
},
"namespace_name": "amazon-cloudwatch",
"pod_id": "2f4ff5ac-c813-11e9-a31d-06e9dde32928",
"pod_name": "cloudwatch-agent-statsd-df44f855f-ts4q2",
"pod_owners": [
{
"owner_kind": "Deployment",
"owner_name": "cloudwatch-agent-statsd"
}
],
"service_name": "cloudwatch-agent-statsd"
},
"pod_interface_network_rx_bytes": 9880.697124714186,
"pod_interface_network_rx_dropped": 0,
"pod_interface_network_rx_errors": 0,
"pod_interface_network_rx_packets": 107.80005532263283,
"pod_interface_network_total_bytes": 10158.829201483635,
"pod_interface_network_tx_bytes": 278.13207676944796,
"pod_interface_network_tx_dropped": 0,
"pod_interface_network_tx_errors": 0,
"pod_interface_network_tx_packets": 1.146027574644318
}
```
**类型：容器**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-sample",
"ClusterName": "myCICluster",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"Namespace": "amazon-cloudwatch",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"PodName": "cloudwatch-agent-statsd",
"Service": "cloudwatch-agent-statsd",
"Sources": [
"cadvisor",
"pod",
"calculated"
],
"Timestamp": "1567097399912",
"Type": "Container",
"Version": "0",
"container_cpu_limit": 200,
"container_cpu_request": 200,
"container_cpu_usage_system": 1.87958283771964,
"container_cpu_usage_total": 6.159993652997942,
"container_cpu_usage_user": 1.6707403001952357,
"container_cpu_utilization": 0.15399984132494854,
"container_memory_cache": 8192,
"container_memory_failcnt": 0,
"container_memory_hierarchical_pgfault": 0,
"container_memory_hierarchical_pgmajfault": 0,
"container_memory_limit": 104857600,
"container_memory_mapped_file": 0,
"container_memory_max_usage": 24580096,
"container_memory_pgfault": 0,
"container_memory_pgmajfault": 0,
"container_memory_request": 104857600,
"container_memory_rss": 22736896,
"container_memory_swap": 0,
"container_memory_usage": 24453120,
"container_memory_utilization": 0.10574541028701798,
"container_memory_working_set": 17580032,
"container_status": "Running",
"kubernetes": {
"container_name": "cloudwatch-agent",
"docker": {
"container_id": "8967b6b37da239dfad197c9fdea3e5dfd35a8a759ec86e2e4c3f7b401e232706"
},
"host": "ip-192-168-75-26.us-west-2.compute.internal",
"labels": {
"app": "cloudwatch-agent-statsd",
"pod-template-hash": "df44f855f"
},
"namespace_name": "amazon-cloudwatch",
"pod_id": "2f4ff5ac-c813-11e9-a31d-06e9dde32928",
"pod_name": "cloudwatch-agent-statsd-df44f855f-ts4q2",
"pod_owners": [
{
"owner_kind": "Deployment",
"owner_name": "cloudwatch-agent-statsd"
}
],
"service_name": "cloudwatch-agent-statsd"
},
"number_of_container_restarts": 0
}
```
**类型：ContainerFS**
```
{
"AutoScalingGroupName": "eksctl-myCICluster-nodegroup-standard-workers-NodeGroup-1174PV2WHZAYU",
"ClusterName": "myCICluster",
"EBSVolumeId": "aws://us-west-2b/vol-0a53108976d4a2fda",
"InstanceId": "i-1234567890123456",
"InstanceType": "t3.xlarge",
"Namespace": "amazon-cloudwatch",
"NodeName": "ip-192-0-2-0.us-west-2.compute.internal",
"PodName": "cloudwatch-agent-statsd",
"Service": "cloudwatch-agent-statsd",
"Sources": [
"cadvisor",
"calculated"
],
"Timestamp": "1567097399912",
"Type": "ContainerFS",
"Version": "0",
"device": "/dev/nvme0n1p1",
"fstype": "vfs",
"kubernetes": {
"container_name": "cloudwatch-agent",
"docker": {
"container_id": "8967b6b37da239dfad197c9fdea3e5dfd35a8a759ec86e2e4c3f7b401e232706"
},
"host": "ip-192-168-75-26.us-west-2.compute.internal",
"labels": {
"app": "cloudwatch-agent-statsd",
"pod-template-hash": "df44f855f"
},
"namespace_name": "amazon-cloudwatch",
"pod_id": "2f4ff5ac-c813-11e9-a31d-06e9dde32928",
"pod_name": "cloudwatch-agent-statsd-df44f855f-ts4q2",
"pod_owners": [
{
"owner_kind": "Deployment",
"owner_name": "cloudwatch-agent-statsd"
}
],
"service_name": "cloudwatch-agent-statsd"
}
}
```
**类型：集群**
```
{
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Count",
"Name": "cluster_node_count"
},
{
"Unit": "Count",
"Name": "cluster_failed_node_count"
}
],
"Dimensions": [
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
}
],
"ClusterName": "myCICluster",
"Sources": [
"apiserver"
],
"Timestamp": "1567097534160",
"Type": "Cluster",
"Version": "0",
"cluster_failed_node_count": 0,
"cluster_node_count": 3
}
```
**类型：“ClusterService”**
```
{
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Count",
"Name": "service_number_of_running_pods"
}
],
"Dimensions": [
[
"Service",
"Namespace",
"ClusterName"
],
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
}
],
"ClusterName": "myCICluster",
"Namespace": "amazon-cloudwatch",
"Service": "cloudwatch-agent-statsd",
"Sources": [
"apiserver"
],
"Timestamp": "1567097534160",
"Type": "ClusterService",
"Version": "0",
"kubernetes": {
"namespace_name": "amazon-cloudwatch",
"service_name": "cloudwatch-agent-statsd"
},
"service_number_of_running_pods": 1
}
```
**类型：ClusterNamespace**
```
{
"CloudWatchMetrics": [
{
"Metrics": [
{
"Unit": "Count",
"Name": "namespace_number_of_running_pods"
}
],
"Dimensions": [
[
"Namespace",
"ClusterName"
],
[
"ClusterName"
]
],
"Namespace": "ContainerInsights"
}
],
"ClusterName": "myCICluster",
"Namespace": "amazon-cloudwatch",
"Sources": [
"apiserver"
],
"Timestamp": "1567097594160",
"Type": "ClusterNamespace",
"Version": "0",
"kubernetes": {
"namespace_name": "amazon-cloudwatch"
},
"namespace_number_of_running_pods": 7
}
```
# Amazon EKS 和 Kubernetes 的性能日志事件中的相关字段
对于 Amazon EKS 和 Kubernetes，容器化的 CloudWatch 代理将数据作为性能日志事件发出。这使 CloudWatch 能够摄取和存储高基数数据。CloudWatch 使用性能日志事件中的数据在集群、节点和 pod 级别创建聚合的 CloudWatch 指标，而不会丢失详细信息。
下表列出了这些性能日志事件中与 Container Insights 指标数据收集相关的字段。您可以使用 CloudWatch Logs Insights 查询其中的任何字段以收集数据或调查问题。有关更多信息，请参阅[使用 CloudWatch Logs Insights 分析日志数据](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html)。
| Type | 日志字段 | 来源 | 公式或备注 |
| --- | --- | --- | --- |
| Pod | `pod_cpu_utilization` | 计算 | 公式：`pod_cpu_usage_total / node_cpu_limit` |
| Pod | `pod_cpu_usage_total` `pod_cpu_usage_total` 以千分之一核心为单位报道。 | cadvisor | |
| Pod | `pod_cpu_limit` | 计算 | 公式：`sum(container_cpu_limit)` `sum(container_cpu_limit)` 包括已经完成的容器组（pod）。 如果 pod 中的任何容器未定义 CPU 限制，则该字段不会显示在日志事件中。这包括 [init 容器](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/#resources)。 |
| Pod | `pod_cpu_request` | 计算 | 公式：`sum(container_cpu_request)` 不保证设置 `container_cpu_request`。仅在总和中包括设置的字段。 |
| Pod | `pod_cpu_utilization_over_pod_limit` | 计算 | 公式：`pod_cpu_usage_total / pod_cpu_limit` |
| Pod | `pod_cpu_reserved_capacity` | 计算 | 公式：`pod_cpu_request / node_cpu_limit` |
| Pod | `pod_memory_utilization` | 计算 | 公式：`pod_memory_working_set / node_memory_limit` 它是 pod 内存使用量相比节点内存限制的百分比。 |
| Pod | `pod_memory_working_set` | cadvisor | |
| Pod | `pod_memory_limit` | 计算 | 公式：`sum(container_memory_limit)` 如果 pod 中的任何容器没有定义内存限制，则该字段不会显示在日志事件中。这包括 [init 容器](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/#resources)。 |
| Pod | `pod_memory_request` | 计算 | 公式：`sum(container_memory_request)` 不保证设置 `container_memory_request`。仅在总和中包括设置的字段。 |
| Pod | `pod_memory_utilization_over_pod_limit` | 计算 | 公式：`pod_memory_working_set / pod_memory_limit` 如果 pod 中的任何容器没有定义内存限制，则该字段不会显示在日志事件中。这包括 [init 容器](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/#resources)。 |
| Pod | `pod_memory_reserved_capacity` | 计算 | 公式：`pod_memory_request / node_memory_limit` |
| Pod | `pod_network_tx_bytes` | 计算 | 公式：`sum(pod_interface_network_tx_bytes)` 该数据适用于每个 pod 的所有网络接口。CloudWatch 代理计算总数并添加指标提取规则。 |
| Pod | `pod_network_rx_bytes` | 计算 | 公式：`sum(pod_interface_network_rx_bytes)` |
| Pod | `pod_network_total_bytes` | 计算 | 公式：`pod_network_rx_bytes + pod_network_tx_bytes` |
| PodNet | `pod_interface_network_rx_bytes` | cadvisor | 该数据是 pod 网络接口每秒接收的网络字节数。 |
| PodNet | `pod_interface_network_tx_bytes` | cadvisor | 该数据是 pod 网络接口每秒发送的网络字节数。 |
| 容器 | `container_cpu_usage_total` | cadvisor | |
| 容器 | `container_cpu_limit` | cadvisor | 不保证设置该字段。如果未设置，则不会发出该指标。 |
| 容器 | `container_cpu_request` | cadvisor | 不保证设置该字段。如果未设置，则不会发出该指标。 |
| 容器 | `container_memory_working_set` | cadvisor | |
| 容器 | `container_memory_limit` | 容器组（pod） | 不保证设置该字段。如果未设置，则不会发出该指标。 |
| 容器 | `container_memory_request` | 容器组（pod） | 不保证设置该字段。如果未设置，则不会发出该指标。 |
| 节点 | `node_cpu_utilization` | 计算 | 公式：`node_cpu_usage_total / node_cpu_limit` |
| 节点 | `node_cpu_usage_total` | cadvisor | |
| 节点 | `node_cpu_limit` | /proc | |
| 节点 | `node_cpu_request` | 计算 | 公式：`sum(pod_cpu_request)` 对于 cronjobs，`node_cpu_request` 还包括来自已完成的容器组（pod）的请求。这可能会导致 `node_cpu_reserved_capacity` 的值很高。 |
| 节点 | `node_cpu_reserved_capacity` | 计算 | 公式：`node_cpu_request / node_cpu_limit` |
| 节点 | `node_memory_utilization` | 计算 | 公式：`node_memory_working_set / node_memory_limit` |
| 节点 | `node_memory_working_set` | cadvisor | |
| 节点 | `node_memory_limit` | /proc | |
| 节点 | `node_memory_request` | 计算 | 公式：`sum(pod_memory_request)` |
| 节点 | `node_memory_reserved_capacity` | 计算 | 公式：`node_memory_request / node_memory_limit` |
| 节点 | `node_network_rx_bytes` | 计算 | 公式：`sum(node_interface_network_rx_bytes)` |
| 节点 | `node_network_tx_bytes` | 计算 | 公式：`sum(node_interface_network_tx_bytes)` |
| 节点 | `node_network_total_bytes` | 计算 | 公式：`node_network_rx_bytes + node_network_tx_bytes` |
| 节点 | `node_number_of_running_pods` | Pod 列表 | |
| 节点 | `node_number_of_running_containers` | Pod 列表 | |
| NodeNet | `node_interface_network_rx_bytes` | cadvisor | 该数据是工作线程节点网络接口每秒接收的网络字节数。 |
| NodeNet | `node_interface_network_tx_bytes` | cadvisor | 该数据是工作线程节点网络接口每秒发送的网络字节数。 |
| NodeFS | `node_filesystem_capacity` | cadvisor | |
| NodeFS | `node_filesystem_usage` | cadvisor | |
| NodeFS | `node_filesystem_utilization` | 计算 | 公式：`node_filesystem_usage / node_filesystem_capacity` 该数据适用于每个设备名称。 |
| 集群 | `cluster_failed_node_count` | API 服务器 | |
| 集群 | `cluster_node_count` | API 服务器 | |
| 服务 | `service_number_of_running_pods` | API 服务器 | |
| `Namespace` | `namespace_number_of_running_pods` | API 服务器 | |
## 指标计算示例
本节包含的示例说明了如何计算上表中的某些值。
假设您具有一个处于以下状态的集群。
```
Node1
node_cpu_limit = 4
node_cpu_usage_total = 3
Pod1
pod_cpu_usage_total = 2
Container1
container_cpu_limit = 1
container_cpu_request = 1
container_cpu_usage_total = 0.8
Container2
container_cpu_limit = null
container_cpu_request = null
container_cpu_usage_total = 1.2
Pod2
pod_cpu_usage_total = 0.4
Container3
container_cpu_limit = 1
container_cpu_request = 0.5
container_cpu_usage_total = 0.4
Node2
node_cpu_limit = 8
node_cpu_usage_total = 1.5
Pod3
pod_cpu_usage_total = 1
Container4
container_cpu_limit = 2
container_cpu_request = 2
container_cpu_usage_total = 1
```
下表说明了如何使用该数据计算 pod CPU 指标。
| 指标 | Formula | Pod1 | Pod2 | Pod3 |
| --- | --- | --- | --- | --- |
| `pod_cpu_utilization` | `pod_cpu_usage_total / node_cpu_limit` | 2 / 4 = 50% | 0.4 / 4 = 10% | 1 / 8 = 12.5% |
| `pod_cpu_utilization_over_pod_limit` | `pod_cpu_usage_total / sum(container_cpu_limit)` | 不适用，因为没有定义 `Container2` 的 CPU 限制 | 0.4 / 1 = 40% | 1 / 2 = 50% |
| `pod_cpu_reserved_capacity` | `sum(container_cpu_request) / node_cpu_limit` | (1 \$1 0) / 4 = 25% | 0.5 / 4 = 12.5% | 2 / 8 = 25% |
下表说明了如何使用该数据计算节点 CPU 指标。
| 指标 | Formula | Node1 | Node2 |
| --- | --- | --- | --- |
| `node_cpu_utilization` | `node_cpu_usage_total / node_cpu_limit` | 3 / 4 = 75% | 1.5 / 8 = 18.75% |
| `node_cpu_reserved_capacity` | `sum(pod_cpu_request) / node_cpu_limit` | 1.5 / 4 = 37.5% | 2 / 8 = 25% |