Run critical add-ons on dedicated instances

In this topic, you will learn how to deploy a workload with a CriticalAddonsOnly toleration so EKS Auto Mode will schedule it onto the system node pool.

EKS Auto Mode’s built-in system node pool is designed for running critical add-ons on dedicated instances. This segregation ensures essential components have dedicated resources and are isolated from general workloads, enhancing overall cluster stability and performance.

This guide demonstrates how to deploy add-ons to the system node pool by utilizing the CriticalAddonsOnly toleration and appropriate node selectors. By following these steps, you can ensure that your critical applications are scheduled onto the dedicated system nodes, leveraging the isolation and resource allocation benefits provided by EKS Auto Mode’s specialized node pool structure.

EKS Auto Mode has two built-in node pools: general-purpose and system. For more information, see Enable or Disable Built-in NodePools.

The purpose of the system node pool is to segregate critical add-ons onto different nodes. Nodes provisioned by the system node pool have a CriticalAddonsOnly Kubernetes taint. Kubernetes will only schedule pods onto these nodes if they have a corresponding toleration. For more information, see Taints and Tolerations in the Kubernetes documentation.

Prerequisites

Procedure

Review the example yaml below. Note the following configurations:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      nodeSelector:
        karpenter.sh/nodepool: system
      tolerations:
      - key: "CriticalAddonsOnly"
        operator: "Exists"
      containers:
      - name: app
        image: nginx:latest
        resources:
          requests:
            cpu: "500m"
            memory: "512Mi"

To update a workload to run on the system node pool, you need to:

After updating the workload, it will run on dedicated nodes.