Kelola komputasi untuk AI/ML beban kerja dengan EKS Auto Mode dan Karpenter

Untuk daftar lengkapnya, lihat Label yang Didukung Mode Otomatis EKS.

Untuk daftar lengkapnya, lihat Label Karpenter Well-Known .

Contoh di bawah ini menunjukkan kapasitas statis NodePool yang menggunakan Mode Otomatis EKS default NodeClass dan membuat statis NodePool dengan 4 node (replicas) yang dapat paling banyak 6 node (limits.nodes).

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-static-inference
spec:
  replicas: 4
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["g6e"]
        - key: "topology.kubernetes.io/zone"
          operator: In
          values: ["us-east-1a"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule
  limits:
    nodes: 6   # Allow temporary headroom during node replacement

Dengan Karpenter yang dikelola sendiri, kapasitas statis dijaga oleh StaticCapacity fitur alfa (diluncurkan di Karpenter versi v1.8), yang harus diaktifkan dalam nilai Helm:

settings:
  featureGates:
    staticCapacity: true

NodePool Referensi kustom EC2NodeClass bernama my-nodeclass dan menciptakan statis NodePool dengan 4 node (replicas) yang dapat paling banyak 6 node (limits.nodes).

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-static-inference
spec:
  replicas: 4
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: my-nodeclass
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["g6e"]
        - key: "topology.kubernetes.io/zone"
          operator: In
          values: ["us-east-1a"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule
  limits:
    nodes: 6   # Allow temporary headroom during node replacement

Buat referensi NodeClass yang mereferensikan reservasi Blok Kapasitas Anda, lalu buat NodePool yang menggunakannya.

Dengan consolidateAfter: Never set, Karpenter tidak akan mencoba mengganti, menggabungkan, atau menghentikan node untuk mengurangi biaya atau mengemas beban kerja dengan lebih efisien. Ini direkomendasikan untuk Blok Kapasitas karena kapasitasnya sudah prabayar.

apiVersion: eks.amazonaws.com/v1
kind: NodeClass
metadata:
  name: capacity-block-gpu
spec:
  capacityReservationSelectorTerms:
    - id: "cr-0123456789abcdef0"   # Your Capacity Block reservation ID
    # Alternative: select by tags
    # - tags:
    #     role: "production-inference"
    #   owner: "012345678901"
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-capacity-block
spec:
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: Never
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: capacity-block-gpu
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["reserved"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["p5", "p5e", "p5en", "p4d"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

Buat EC2NodeClass yang mencakup AMI, subnet, dan penyeleksi grup keamanan selaincapacityReservationSelectorTerms, lalu buat NodePool yang menggunakannya.

Dengan consolidateAfter: Never set, Karpenter tidak akan mencoba mengganti, menggabungkan, atau menghentikan node untuk mengurangi biaya atau mengemas beban kerja dengan lebih efisien. Ini direkomendasikan untuk Blok Kapasitas karena kapasitasnya sudah prabayar.

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: capacity-block-gpu
spec:
  amiSelectorTerms:
    - alias: al2023@latest
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: ml-cluster
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: ml-cluster
  capacityReservationSelectorTerms:
    - id: "cr-0123456789abcdef0" # Your Capacity Block reservation ID
    # Alternative: select by tags
    # - tags:
    #     role: "production-inference"
    #   owner: "012345678901"
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-capacity-block
spec:
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: Never
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: capacity-block-gpu
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["reserved"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["p5", "p5e", "p5en", "p4d"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

Buat NodeClass dengan capacityReservationSelectorTerms dan a NodePool yang memprioritaskan reserved dengan on-demand fallback. Sematkan topology.kubernetes.io/zone ke AZ ODCR:

apiVersion: eks.amazonaws.com/v1
kind: NodeClass
metadata:
  name: odcr-gpu-production
spec:
  capacityReservationSelectorTerms:
    - id: "cr-0987654321fedcba0"
    # Alternative: select by tags
    # - tags:
    #     Purpose: "production-inference"
    #   owner: "012345678901"
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-reserved-production
spec:
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: Never
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: odcr-gpu-production
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["reserved", "on-demand"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["p5", "g6e"]
        - key: "topology.kubernetes.io/zone"
          operator: In
          values: ["us-east-1a"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

Buat EC2NodeClass dengan AMI, subnet, dan penyeleksi grup keamanan sebagai tambahancapacityReservationSelectorTerms, lalu buat: NodePool

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: odcr-gpu-production
spec:
  amiSelectorTerms:
    - alias: al2023@latest
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: ml-cluster
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: ml-cluster
  capacityReservationSelectorTerms:
    - id: "cr-0987654321fedcba0"
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-reserved-production
spec:
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: Never
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: odcr-gpu-production
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["reserved", "on-demand"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["p5", "g6e"]
        - key: "topology.kubernetes.io/zone"
          operator: In
          values: ["us-east-1a"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-ondemand
spec:
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["g6", "g6e", "g7e"]
        - key: "karpenter.k8s.aws/instance-gpu-manufacturer"
          operator: In
          values: ["nvidia"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-ondemand
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["g6", "g6e", "g7e"]
        - key: "karpenter.k8s.aws/instance-gpu-manufacturer"
          operator: In
          values: ["nvidia"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

Mode Otomatis EKS menangani interupsi Spot secara native. Tidak diperlukan antrian SQS atau Node Termination Handler.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-spot
spec:
  disruption:
    budgets:
      - nodes: 10%
    consolidationPolicy: WhenEmpty
    consolidateAfter: 1h
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["spot"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["g6", "g6e", "g7e"]
        - key: "karpenter.k8s.aws/instance-gpu-manufacturer"
          operator: In
          values: ["nvidia"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule
  limits:
    resources:
      nvidia.com/gpu: "64"

Self-managed Karpenter mengharuskan Anda untuk mengaktifkan penanganan interupsi asli pada pengontrol Karpenter (bukan di NodePool) dengan mengonfigurasi antrian interupsi: antrian SQS yang menerima interupsi Spot EC2 dan peristiwa Rekomendasi Rebalance. Anda mengonfigurasi ini sekali pada waktu penginstalan.

Jika Anda menginstal Karpenter langsung dengan Helm, atur settings.interruptionQueue di: values.yaml

# karpenter values.yaml (Helm)
settings:
  clusterName: my-cluster
  interruptionQueue: my-queue   # Name of the SQS queue receiving Spot events

Jika Anda mem-bootstrap Karpenter denganeksctl, atur withSpotInterruptionQueue: true di file konfigurasi cluster Anda. eksctlmembuat antrian SQS dan EventBridge aturan dan mengkonfigurasi kontroler Karpenter untuk menggunakannya.

# eksctl ClusterConfig
karpenter:
  version: "${KARPENTER_VERSION}"
  withSpotInterruptionQueue: true

Setelah pengontrol diatur untuk menggunakan antrian Anda, tidak ada konfigurasi tambahan yang diperlukan pada NodePool sumber daya individu. Penanganan interupsi berlaku di seluruh cluster.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-spot
spec:
  disruption:
    budgets:
      - nodes: 10%
    consolidationPolicy: WhenEmpty
    consolidateAfter: 1h
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["spot"]
        - key: "karpenter.k8s.aws/instance-family"
          operator: In
          values: ["g6", "g6e", "g7e"]
        - key: "karpenter.k8s.aws/instance-gpu-manufacturer"
          operator: In
          values: ["nvidia"]
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule
  limits:
    resources:
      nvidia.com/gpu: "64"