本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。

审计和记录

出于各种不同的原因，收集和分析 [审计] 日志很有用。日志可以帮助进行根本原因分析和归因，即将更改归因于特定用户。收集到足够的日志后，它们也可以用来检测异常行为。在 EKS 上，审计日志会发送到亚马逊 Cloudwatch Logs。EKS 的审计政策如下：

apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
  # Log full request and response for changes to aws-auth ConfigMap in kube-system namespace
  - level: RequestResponse
    namespaces: ["kube-system"]
    verbs: ["update", "patch", "delete"]
    resources:
      - group: "" # core
        resources: ["configmaps"]
        resourceNames: ["aws-auth"]
    omitStages:
      - "RequestReceived"
  # Do not log watch operations performed by kube-proxy on endpoints and services
  - level: None
    users: ["system:kube-proxy"]
    verbs: ["watch"]
    resources:
      - group: "" # core
        resources: ["endpoints", "services", "services/status"]
  # Do not log get operations performed by kubelet on nodes and their statuses
  - level: None
    users: ["kubelet"] # legacy kubelet identity
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["nodes", "nodes/status"]
  # Do not log get operations performed by the system:nodes group on nodes and their statuses
  - level: None
    userGroups: ["system:nodes"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["nodes", "nodes/status"]
  # Do not log get and update operations performed by controller manager, scheduler, and endpoint-controller on endpoints in kube-system namespace
  - level: None
    users:
      - system:kube-controller-manager
      - system:kube-scheduler
      - system:serviceaccount:kube-system:endpoint-controller
    verbs: ["get", "update"]
    namespaces: ["kube-system"]
    resources:
      - group: "" # core
        resources: ["endpoints"]
  # Do not log get operations performed by apiserver on namespaces and their statuses/finalizations
  - level: None
    users: ["system:apiserver"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["namespaces", "namespaces/status", "namespaces/finalize"]
  # Do not log get and list operations performed by controller manager on metrics.k8s.io resources
  - level: None
    users:
      - system:kube-controller-manager
    verbs: ["get", "list"]
    resources:
      - group: "metrics.k8s.io"
  # Do not log access to health, version, and swagger non-resource URLs
  - level: None
    nonResourceURLs:
      - /healthz*
      - /version
      - /swagger*
  # Do not log events resources
  - level: None
    resources:
      - group: "" # core
        resources: ["events"]
  # Log request for updates/patches to nodes and pods statuses by kubelet and node problem detector
  - level: Request
    users: ["kubelet", "system:node-problem-detector", "system:serviceaccount:kube-system:node-problem-detector"]
    verbs: ["update", "patch"]
    resources:
      - group: "" # core
        resources: ["nodes/status", "pods/status"]
    omitStages:
      - "RequestReceived"
  # Log request for updates/patches to nodes and pods statuses by system:nodes group
  - level: Request
    userGroups: ["system:nodes"]
    verbs: ["update", "patch"]
    resources:
      - group: "" # core
        resources: ["nodes/status", "pods/status"]
    omitStages:
      - "RequestReceived"
  # Log delete collection requests by namespace-controller in kube-system namespace
  - level: Request
    users: ["system:serviceaccount:kube-system:namespace-controller"]
    verbs: ["deletecollection"]
    omitStages:
      - "RequestReceived"
  # Log metadata for secrets, configmaps, and tokenreviews to protect sensitive data
  - level: Metadata
    resources:
      - group: "" # core
        resources: ["secrets", "configmaps"]
      - group: authentication.k8s.io
        resources: ["tokenreviews"]
    omitStages:
      - "RequestReceived"
  # Log requests for serviceaccounts/token resources
  - level: Request
    resources:
      - group: "" # core
        resources: ["serviceaccounts/token"]
  # Log get, list, and watch requests for various resource groups
  - level: Request
    verbs: ["get", "list", "watch"]
    resources:
      - group: "" # core
      - group: "admissionregistration.k8s.io"
      - group: "apiextensions.k8s.io"
      - group: "apiregistration.k8s.io"
      - group: "apps"
      - group: "authentication.k8s.io"
      - group: "authorization.k8s.io"
      - group: "autoscaling"
      - group: "batch"
      - group: "certificates.k8s.io"
      - group: "extensions"
      - group: "metrics.k8s.io"
      - group: "networking.k8s.io"
      - group: "policy"
      - group: "rbac.authorization.k8s.io"
      - group: "scheduling.k8s.io"
      - group: "settings.k8s.io"
      - group: "storage.k8s.io"
    omitStages:
      - "RequestReceived"
  # Default logging level for known APIs to log request and response
  - level: RequestResponse
    resources:
      - group: "" # core
      - group: "admissionregistration.k8s.io"
      - group: "apiextensions.k8s.io"
      - group: "apiregistration.k8s.io"
      - group: "apps"
      - group: "authentication.k8s.io"
      - group: "authorization.k8s.io"
      - group: "autoscaling"
      - group: "batch"
      - group: "certificates.k8s.io"
      - group: "extensions"
      - group: "metrics.k8s.io"
      - group: "networking.k8s.io"
      - group: "policy"
      - group: "rbac.authorization.k8s.io"
      - group: "scheduling.k8s.io"
      - group: "settings.k8s.io"
      - group: "storage.k8s.io"
    omitStages:
      - "RequestReceived"
  # Default logging level for all other requests to log metadata only
  - level: Metadata
    omitStages:
      - "RequestReceived"

建议

启用审计日志

审计日志是由 EKS 管理的 Kubernetes 控制平面日志的一部分。控制平面日志 enabling/disabling 的说明，包括 Kubernetes API 服务器、控制器管理器和调度程序的日志以及审核日志，可以在此处找到，即 #enabling-control-plane-log-export。https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html

利用审计元数据

Kubernetes 审核日志包含两条注释，用于指明请求是否获得授权authorization.k8s.io/decision以及决策原因。authorization.k8s.io/reason使用这些属性来确定允许特定 API 调用的原因。

为可疑事件创建警报

创建警报以自动提醒您 403 个 “禁止” 和 401 个未经授权的响应增加了，然后使用诸如hostsourceIPs、和之类的属性k8s_user.username来找出这些请求的来源。

使用日志见解分析日志

使用 CloudWatch 日志见解监控 RBAC 对象的更改，例如角色、 RoleBindings ClusterRoles、和。 ClusterRoleBindings下面是一些示例查询：

列出对以下内容的更新 aws-auth ConfigMap：

fields @timestamp, @message
| filter @logStream like "kube-apiserver-audit"
| filter verb in ["update", "patch"]
| filter objectRef.resource = "configmaps" and objectRef.name = "aws-auth" and objectRef.namespace = "kube-system"
| sort @timestamp desc

列出新建的验证网络挂钩或对验证网络挂钩的更改：

fields @timestamp, @message
| filter @logStream like "kube-apiserver-audit"
| filter verb in ["create", "update", "patch"] and responseStatus.code = 201
| filter objectRef.resource = "validatingwebhookconfigurations"
| sort @timestamp desc

列出对角色的创建、更新、删除操作：

fields @timestamp, @message
| sort @timestamp desc
| limit 100
| filter objectRef.resource="roles" and verb in ["create", "update", "patch", "delete"]

列出创建、更新、删除操作以 RoleBindings：

fields @timestamp, @message
| sort @timestamp desc
| limit 100
| filter objectRef.resource="rolebindings" and verb in ["create", "update", "patch", "delete"]

列出创建、更新、删除操作以 ClusterRoles：

fields @timestamp, @message
| sort @timestamp desc
| limit 100
| filter objectRef.resource="clusterroles" and verb in ["create", "update", "patch", "delete"]

列出创建、更新、删除操作以 ClusterRoleBindings：

fields @timestamp, @message
| sort @timestamp desc
| limit 100
| filter objectRef.resource="clusterrolebindings" and verb in ["create", "update", "patch", "delete"]

策划针对机密的未经授权的读取操作：

fields @timestamp, @message
| sort @timestamp desc
| limit 100
| filter objectRef.resource="secrets" and verb in ["get", "watch", "list"] and responseStatus.code="401"
| stats count() by bin(1m)

失败的匿名请求列表：

fields @timestamp, @message, sourceIPs.0
| sort @timestamp desc
| limit 100
| filter user.username="system:anonymous" and responseStatus.code in ["401", "403"]

审核您的 CloudTrail 日志

由使用服务账户的 IAM 角色 (IRSA) 的 Pod 调用的 AWS API 会自动与服务账户的名称 CloudTrail 一起登录。如果未明确授权调用 API 的服务账户的名称出现在日志中，则可能表明 IAM 角色的信任策略配置不正确。一般而言，Cloudtrail是将AWS API调用归因于特定的IAM委托人的好方法。

使用 CloudTrail Insights 发掘可疑活动

CloudTrail insights 会自动分析CloudTrail 跟踪中的写入管理事件，并提醒您异常活动。这可以帮助您确定您的 AWS 账户中写入 API 的调用量何时增加，包括使用 IRSA 代替 IAM 角色的容器调用。有关更多信息，请参阅公布CloudTrail 见解：识别和应对异常 API 活动。

其他资源

随着日志数量的增加，使用 Log Insights 或其他日志分析工具解析和筛选日志可能会变得无效。作为替代方案，你可能需要考虑运行 Sys dig Falco 和 ekscloudwatch。https://github.com/sysdiglabs/ekscloudwatchFalco 会分析审计日志，并在很长一段时间内标记异常或滥用情况。ekscloudwatch 项目将审计日志事件从 Falco 转发给 Falco CloudWatch 进行分析。Falco 提供了一组默认审计规则，并可以添加自己的审计规则。

另一种选择可能是将审计日志存储在 S3 中，并使用 SageMaker Random Cut Forest 算法来处理需要进一步调查的异常行为。

工具和资源

以下商业和开源项目可用于评估您的集群与既定最佳实践的一致性：