**Help improve this page** To contribute to this user guide, choose the **Edit this page on GitHub** link that is located in the right pane of every page. # Create a Node Class for Amazon EKS Amazon EKS Node Classes are templates that offer granular control over the configuration of your EKS Auto Mode managed nodes. A Node Class defines infrastructure-level settings that apply to groups of nodes in your EKS cluster, including network configuration, storage settings, and resource tagging. This topic explains how to create and configure a Node Class to meet your specific operational requirements. When you need to customize how EKS Auto Mode provisions and configures EC2 instances beyond the default settings, creating a Node Class gives you precise control over critical infrastructure parameters. For example, you can specify private subnet placement for enhanced security, configure instance ephemeral storage for performance-sensitive workloads, or apply custom tagging for cost allocation. ## Create a Node Class To create a `NodeClass`, follow these steps: 1. Create a YAML file (for example, `nodeclass.yaml`) with your Node Class configuration 1. Apply the configuration to your cluster using `kubectl` 1. Reference the Node Class in your Node Pool configuration. For more information, see [Create a Node Pool for EKS Auto Mode](create-node-pool.md). You need `kubectl` installed and configured. For more information, see [Set up to use Amazon EKS](setting-up.md). ### Basic Node Class Example Here’s an example Node Class: ``` apiVersion: eks.amazonaws.com/v1 kind: NodeClass metadata: name: private-compute spec: subnetSelectorTerms: - tags: Name: "private-subnet" kubernetes.io/role/internal-elb: "1" securityGroupSelectorTerms: - tags: Name: "eks-cluster-sg" ephemeralStorage: size: "160Gi" ``` This NodeClass increases the amount of ephemeral storage on the node. Apply this configuration by using: ``` kubectl apply -f nodeclass.yaml ``` Next, reference the Node Class in your Node Pool configuration. For more information, see [Create a Node Pool for EKS Auto Mode](create-node-pool.md). ## Create node class access entry If you create a custom node class, you need to create an EKS Access Entry to permit the nodes to join the cluster. EKS automatically creates access entries when you use the built-in node class and node pools. For information about how Access Entries work, see [Grant IAM users access to Kubernetes with EKS access entries](access-entries.md). When creating access entries for EKS Auto Mode node classes, you need to use the `EC2` access entry type. ### Create access entry with CLI **To create an access entry for EC2 nodes and associate the EKS Auto Node Policy:** Update the following CLI commands with your cluster name, and node role ARN. The node role ARN is specified in the node class YAML. ``` # Create the access entry for EC2 nodes aws eks create-access-entry \ --cluster-name \ --principal-arn \ --type EC2 # Associate the auto node policy aws eks associate-access-policy \ --cluster-name \ --principal-arn \ --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy \ --access-scope type=cluster ``` ### Create access entry with CloudFormation **To create an access entry for EC2 nodes and associate the EKS Auto Node Policy:** Update the following CloudFormation with your cluster name, and node role ARN. The node role ARN is specified in the node class YAML. ``` EKSAutoNodeRoleAccessEntry: Type: AWS::EKS::AccessEntry Properties: ClusterName: PrincipalArn: Type: "EC2" AccessPolicies: - AccessScope: Type: cluster PolicyArn: arn:aws:eks::aws:cluster-access-policy/AmazonEKSAutoNodePolicy DependsOn: [ ] # previously defined in CloudFormation ``` For information about deploying CloudFormation stacks, see [Getting started with CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/GettingStarted.html) ## Node Class Specification ``` apiVersion: eks.amazonaws.com/v1 kind: NodeClass metadata: name: my-node-class spec: # Required fields # role and instanceProfile are mutually exclusive fields. role: MyNodeRole # IAM role for EC2 instances # instanceProfile: eks-MyNodeInstanceProfile # IAM instance-profile for EC2 instances subnetSelectorTerms: - tags: Name: "private-subnet" kubernetes.io/role/internal-elb: "1" # Alternative using direct subnet ID # - id: "subnet-0123456789abcdef0" securityGroupSelectorTerms: - tags: Name: "eks-cluster-sg" # Alternative approaches: # - id: "sg-0123456789abcdef0" # - name: "eks-cluster-security-group" # Optional: Pod subnet selector for advanced networking podSubnetSelectorTerms: - tags: Name: "pod-subnet" kubernetes.io/role/pod: "1" # Alternative using direct subnet ID # - id: "subnet-0987654321fedcba0" # must include Pod security group selector also podSecurityGroupSelectorTerms: - tags: Name: "eks-pod-sg" # Alternative using direct security group ID # - id: "sg-0123456789abcdef0" # Optional: Selects on-demand capacity reservations and capacity blocks # for EKS Auto Mode to prioritize. capacityReservationSelectorTerms: - id: cr-56fac701cc1951b03 # Alternative Approaches - tags: Name: "targeted-odcr" # Optional owning account ID filter owner: "012345678901" # Optional fields snatPolicy: Random # or Disabled networkPolicy: DefaultAllow # or DefaultDeny networkPolicyEventLogs: Disabled # or Enabled ephemeralStorage: size: "80Gi" # Range: 1-59000Gi or 1-64000G or 1-58Ti or 1-64T iops: 3000 # Range: 3000-16000 throughput: 125 # Range: 125-1000 # Optional KMS key for encryption kmsKeyID: "arn:aws:kms:region:account:key/key-id" # Accepted formats: # KMS Key ID # KMS Key ARN # Key Alias Name # Key Alias ARN advancedNetworking: # Optional: Controls whether public IP addresses are assigned to instances that are launched with the nodeclass. # If not set, defaults to the MapPublicIpOnLaunch setting on the subnet. associatePublicIPAddress: false # Optional: Forward proxy, commonly requires certificateBundles as well # for EC2, see https://repost.aws/knowledge-center/eks-http-proxy-containerd-automation httpsProxy: http://192.0.2.4:3128 #commonly port 3128 (Squid) or 8080 (NGINX) #Max 255 characters #httpsProxy: http://[2001:db8::4]:3128 # IPv6 address with port, use [] noProxy: #Max 50 entries - localhost #Max 255 characters each - 127.0.0.1 #- ::1 # IPv6 localhost #- 0:0:0:0:0:0:0:1 # IPv6 localhost - 169.254.169.254 # EC2 Instance Metadata Service #- [fd00:ec2::254] # IPv6 EC2 Instance Metadata Service # Domains to exclude, put all VPC endpoints here - .internal - .eks.amazonaws.com # ipv4PrefixSize is default to Auto which is prefix and fallback to secondary IP. "32" is the secondary IP mode. ipv4PrefixSize: Auto # or "32" # enableV4Egress is default to true. Setting it to false when using network policy or blocking IPv4 traffic in IPv6 clusters enableV4Egress: false advancedSecurity: # Optional, US regions only: Specifying `fips: true` will cause nodes in the nodeclass to run FIPS compatible AMIs. fips: false # Optional: Custom certificate bundles. certificateBundles: - name: "custom-cert" data: "base64-encoded-cert-data" # Optional: Additional EC2 tags (with restrictions) tags: Environment: "production" Team: "platform" # Note: Cannot use restricted tags like: # - kubernetes.io/cluster/* # - karpenter.sh/provisioner-name # - karpenter.sh/nodepool # - karpenter.sh/nodeclaim # - karpenter.sh/managed-by # - eks.amazonaws.com/nodeclass ``` ## Considerations + If you want to verify how much local storage an instance has, you can describe the node to see the ephemeral storage resource. + **Volume Encryption** - EKS uses the configured custom KMS key to encrypt the read-only root volume of the instance and the read/write data volume. + **Replace the node IAM role** - If you change the node IAM role associated with a `NodeClass`, you will need to create a new Access Entry. EKS automatically creates an Access Entry for the node IAM role during cluster creation. The node IAM role requires the `AmazonEKSAutoNodePolicy` EKS Access Policy. For more information, see [Grant IAM users access to Kubernetes with EKS access entries](access-entries.md). + **maximum Pod density** - EKS limits the maximum number of Pods on a node to 110. This limit is applied after the existing max Pods calculation. For more information, see [Choose an optimal Amazon EC2 node instance type](choosing-instance-type.md). + **Tags** - If you want to propagate tags from Kubernetes to EC2, you need to configure additional IAM permissions. For more information, see [Learn about identity and access in EKS Auto Mode](auto-learn-iam.md). + **Default node class** - Do not name your custom node class `default`. This is because EKS Auto Mode includes a `NodeClass` called `default` that is automatically provisioned when you enable at least one built-in `NodePool`. For information about enabling built-in `NodePools`, see [Enable or Disable Built-in NodePools](set-builtin-node-pools.md). + ** `subnetSelectorTerms` behavior with multiple subnets** - If there are multiple subnets that match the `subnetSelectorTerms` conditions or that you provide by ID, EKS Auto Mode creates nodes distributed across the subnets. + If the subnets are in different Availability Zones (AZs), you can use Kubernetes features like [Pod topology spread constraints](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#pod-topology-spread-constraints) and [Topology Aware Routing](https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing/) to spread Pods and traffic across the zones, respectively. + If there are multiple subnets *in the same AZ* that match the `subnetSelectorTerms`, EKS Auto Mode creates Pods on each node distributed across the subnets in that AZ. EKS Auto Mode creates secondary network interfaces on each node in the other subnets in the same AZ. It chooses based on the number of available IP addresses in each subnet, to use the subnets more efficiently. However, you can’t specify which subnet EKS Auto Mode uses for each Pod; if you need Pods to run in specific subnets, use [Separate subnets and security groups for Pods](#pod-subnet-selector) instead. ## Separate subnets and security groups for Pods The `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms` fields enable advanced networking configurations by allowing Pods to use different subnets and security groups than their nodes. Both fields must be specified together. This separation provides enhanced control over network traffic routing and security policies. **Note** This feature is different from the [Security Groups for Pods](security-groups-for-pods.md) (SGPP) feature used with the AWS VPC CNI for non-EKS Auto Mode compute. SGPP is not supported in EKS Auto Mode. Instead, use `podSecurityGroupSelectorTerms` in the `NodeClass` to apply separate security groups to Pod traffic. The security groups apply at the `NodeClass` level, meaning all Pods on nodes using that `NodeClass` share the same Pod security groups. ### How it works When you configure `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms`: 1. The node’s primary ENI uses the subnets and security groups from `subnetSelectorTerms` and `securityGroupSelectorTerms`. Only the node’s own IP address is assigned to this interface. 1. EKS Auto Mode creates secondary ENIs in the subnets matching `podSubnetSelectorTerms`, with the security groups from `podSecurityGroupSelectorTerms` attached. Pod IP addresses are allocated from these secondary ENIs using /28 prefixes by default, with automatic fallback to secondary IPs (/32) when a contiguous prefix block is not available. If `ipv4PrefixSize` is set to `"32"` in `advancedNetworking`, only secondary IPs are used. 1. The security groups specified in `podSecurityGroupSelectorTerms` apply to Pod traffic within the VPC. For traffic destined outside the VPC, Pods use the node’s primary ENI (and its security groups) because source network address translation (SNAT) translates the Pod IP to the node IP. You can modify this behavior with the `snatPolicy` field in the `NodeClass`. ### Use cases Use `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms` when you need to: + Apply different security groups to control traffic for nodes and Pods separately. + Separate infrastructure traffic (node-to-node communication) from application traffic (Pod-to-Pod communication). + Apply different network configurations to node subnets than Pod subnets. + Configure reverse proxies or network filtering specifically for node traffic without affecting Pod traffic. Use `advancedNetworking` and `certificateBundles` to define your reverse proxy and any self-signed or private certificates for the proxy. ### Example configuration ``` apiVersion: eks.amazonaws.com/v1 kind: NodeClass metadata: name: advanced-networking spec: role: MyNodeRole # Subnets and security groups for EC2 instances (nodes) subnetSelectorTerms: - tags: Name: "node-subnet" kubernetes.io/role/internal-elb: "1" securityGroupSelectorTerms: - tags: Name: "eks-cluster-sg" # Separate subnets and security groups for Pods podSubnetSelectorTerms: - tags: Name: "pod-subnet" kubernetes.io/role/pod: "1" podSecurityGroupSelectorTerms: - tags: Name: "eks-pod-sg" ``` ### Considerations for separate Pod subnets and security groups + **Security group scope**: The security groups from `podSecurityGroupSelectorTerms` are attached to the secondary ENIs and apply to Pod traffic within the VPC. When SNAT is enabled (the default `snatPolicy: Random`), traffic leaving the VPC is translated to the node’s primary ENI IP address, so the node’s security groups from `securityGroupSelectorTerms` apply to that traffic instead. If you set `snatPolicy: Disabled`, Pods use their own IP addresses for all traffic, and you must ensure that routing and security groups are configured accordingly. + **NodeClass-level granularity**: The Pod security groups apply to all Pods scheduled on nodes using the `NodeClass`. To apply different security groups to different workloads, create separate `NodeClass` and `NodePool` resources and use taints, tolerations, or node selectors to schedule workloads to the appropriate nodes. + **Reduced Pod density**: Fewer Pods can run on each node because the primary network interface of the node is reserved for the node IP and can’t be used for Pods. + **Subnet selector limitations**: The standard `subnetSelectorTerms` and `securityGroupSelectorTerms` configurations don’t apply to Pod subnet or security group selection. + **Network planning**: Ensure adequate IP address space in both node and Pod subnets to support your workload requirements. + **Routing configuration**: Verify that route table and network Access Control List (ACL) of the Pod subnets are properly configured for communication between node and Pod subnets. + **Availability Zones**: Verify that you’ve created Pod subnets across multiple AZs. If you are using a specific Pod subnet, it must be in the same AZ as the node subnet AZ. ## Secondary IP Mode for Pods The `ipv4PrefixSize` field enables advanced networking configurations by allocating only secondary IP addresses to nodes. This feature doesn’t allocate prefixes (/28) to nodes and maintains only one secondary IP as MinimalIPTarget. ### Use cases Use `ipv4PrefixSize` when you need to: + **Reduced IP utilization**: Only one IP address will be warmed up in every node. + **Lower pod churning rate**: Pod creation velocity is not a major concern. + **No prefix fragmentation**: Prefix-caused fragmentation is a major concern or blocker to use Auto Mode. ### Example configuration ``` apiVersion: eks.amazonaws.com/v1 kind: NodeClass metadata: name: advanced-networking spec: role: MyNodeRole advancedNetworking: ipv4PrefixSize: "32" ``` ### Considerations for secondary IP mode + **Reduced Pod creation velocity**: Since only one secondary IP is warmed up, the IPAM service needs more time to provision IPs when more pods are created. ## Disable IPv4 egress from IPv6 pods in IPv6 clusters. The `enableV4Egress` field is `true` by default. For Auto Mode IPv6 clusters, the feature can be disabled so that Auto Mode won’t create an egress-only IPv4 interface for IPv6 pods. This is important because the IPv4 egress interface is not subject to Network Policy enforcement. Network policies are only enforced on the Pod’s primary interface (eth0). ### Use cases Use `enableV4Egress` when you need to: + **Use IPv6 Cluster**: IPv4 egress traffic is allowed by default. + **Use Network Policy**: Currently EKS Network Policy doesn’t support dual stack. Disabling `enableV4Egress` can prevent pod traffic from egressing over IPv4 unexpectedly. ### Example configuration ``` apiVersion: eks.amazonaws.com/v1 kind: NodeClass metadata: name: advanced-networking spec: role: MyNodeRole advancedNetworking: enableV4Egress: false ``` ### Considerations for disabling enableV4Egress + **Network Policy in IPv6 Cluster**: IPv6 clusters allow IPv4 traffic by default. Setting `enableV4Egress: false` blocks IPv4 egress traffic, providing enhanced security especially when used with Network Policies.