Amazon EKS Hybrid Nodes overview

Amazon EKS Hybrid Nodes must have a reliable connection between your on-premises environment and AWS. Amazon EKS Hybrid Nodes aren’t a fit for disconnected, disrupted, intermittent or limited (DDIL) environments. If you are running in a DDIL environment, consider Amazon EKS Anywhere.

Running Amazon EKS Hybrid Nodes on cloud infrastructure, including AWS Regions, AWS Local Zones, AWS Outposts, or in other clouds, is not supported. Use Amazon EKS Auto Mode, Karpenter, Amazon EC2 managed node groups, self-managed nodes, or AWS Fargate when running in AWS Regions. Use Amazon EC2 managed node groups or Amazon EC2 self-managed nodes when running on AWS Local Zones. Only Amazon EC2 self-managed nodes can be used on AWS Outposts or AWS Wavelength Zones.

A single Amazon EKS cluster can be used to run hybrid nodes and nodes in AWS Regions, AWS Local Zones, or AWS Outposts.

Amazon EKS Hybrid Nodes is available in all AWS Regions, except the AWS GovCloud (US) Regions and the AWS China Regions.

You will be charged the hybrid nodes fee if you run hybrid nodes on Amazon EC2 instances.

Billing for hybrid nodes starts when the nodes join the Amazon EKS cluster and stops when the nodes are removed from the cluster. Be sure to remove your hybrid nodes from your Amazon EKS cluster if you are not using them.

Amazon EKS Hybrid Nodes follows a bring your own infrastructure approach where it is your responsibility to provision and manage the physical or virtual machines and the operating system you use for hybrid nodes.

Amazon EKS Hybrid Nodes are agnostic to the infrastructure they run on. You can run hybrid nodes on physical or virtual machines, and x86 and ARM architectures.

Amazon Linux 2023 (AL2023): You can use Amazon Linux 2023 (AL2023) as the node operating system for hybrid nodes, but only in virtualized environments such as VMWare, KVM, and Hyper-V. AWS supports the integration of hybrid nodes with AL2023, but AL2023 isn’t covered by the AWS Support Plans when you run it outside of Amazon EC2.

Ubuntu: You can use Ubuntu 20.04, Ubuntu 22.04, and Ubuntu 24.04 as the node operating system for hybrid nodes.

Red Hat Enterprise Linux (RHEL): You can use RHEL 8 and RHEL 9 as the node operating system for hybrid nodes.

Amazon EKS Hybrid Nodes supports the same Kubernetes versions and deprecation schedule as Amazon EKS, including standard and extended Kubernetes version support. For more information on Kubernetes versions in Amazon EKS, see Understand the Kubernetes version lifecycle on EKS. For more information about Amazon EKS platform versions, see View Amazon EKS platform versions for each Kubernetes version.

You must create new Amazon EKS clusters to use Amazon EKS Hybrid Nodes. Hybrid nodes can’t be used with existing Amazon EKS clusters.

The communication between the Amazon EKS control plane and hybrid nodes is routed through the VPC and subnets you pass during cluster creation, which builds on the existing mechanism in Amazon EKS for control plane to node networking.

Amazon EKS Hybrid Nodes is flexible to your preferred method of connecting your on-premises networks to a VPC in AWS. There are several documented options available including AWS Site-to-Site VPN and AWS Direct Connect, and you can choose the method that best fits your use case.

IP address family: Hybrid nodes can be used with Amazon EKS clusters configured with the IPv4 IP address family only. You can’t use Amazon EKS clusters configured with the IPv6 IP address family. Similarly, your on-premises node and Pod CIDRs must be IPv4 RFC1918 CIDR blocks.

You must enable the required domains, protocols, and ports for Amazon EKS Hybrid Nodes in your on-premises environments and firewalls. For more information, including minimum networking requirements, see Prepare networking for hybrid nodes.

Cluster endpoint access: You can use “Public” or “Private” cluster endpoint access. You should not use “Public and Private” cluster endpoint access, as the endpoint DNS resolution will always resolve to the public addresses for queries originating from your on-premises environment.

For information and best practices during scenarios where there are network disconnections between hybrid nodes and the AWS Region, see the hybrid nodes section of the Amazon EKS Best Practices Guide.

Application load balancing: Kubernetes has a Service object to define the names and domain names for your applications and resolve and load balance to them. By default, the type:LoadBalancer type of Service additionally creates an AWS Classic Load Balancer for traffic from outside the cluster. You can change this behavior with add-ons. Specifically, we recommend the AWS Application Load Balancer and AWS Network Load Balancer which are created by the AWS Load Balancer Controller, instead of the AWS Classic Load Balancer. For steps to install the AWS Load Balancer Controller in a hybrid environment, see AWS Load Balancer Controller.

Amazon EKS Hybrid Nodes use temporary IAM credentials to authenticate with your Amazon EKS cluster. You can use either AWS IAM Roles Anywhere or AWS Systems Manager (SSM) hybrid activations for provisioning the on-premises IAM credentials for hybrid nodes. It is recommended to use AWS SSM hybrid activations if you do not have existing Public Key Infrastructure (PKI) with a Certificate Authority (CA) and certificates for your on-premises environments. If you do have existing PKI and certificates on-premises, use AWS IAM Roles Anywhere.

You can use API or API_AND_CONFIG_MAP cluster authentication modes for your hybrid nodes-enabled Amazon EKS clusters. Use the cluster access entry type called HYBRID_LINUX with your hybrid nodes IAM role to enable hybrid nodes to join the Amazon EKS cluster.

OIDC authentication is supported for hybrid nodes-enabled Amazon EKS clusters.

You can use Amazon EKS Pod Identities and IAM Roles for Service Accounts (IRSA) with applications running on hybrid nodes to enable granular access for your Pods running on hybrid nodes with other AWS services.

You can use Amazon GuardDuty EKS Protection with hybrid nodes-enabled Amazon EKS clusters to analyze activities of users and applications accessing your cluster.

Container Networking Interface (CNI): The AWS VPC CNI can’t be used with hybrid nodes. The core capabilities of Cilium and Calico are supported for use with hybrid nodes. You can manage your CNI with your choice of tooling such as Helm. For more information, see Configure a CNI for hybrid nodes.

kube-proxy and CoreDNS: kube-proxy and CoreDNS are installed automatically when hybrid nodes join the Amazon EKS cluster. These add-ons can be managed as Amazon EKS add-ons after cluster creation.

Ingress and Load Balancing: You can use the AWS Load Balancer Controller and Application Load Balancer (ALB) or Network Load Balancer (NLB) with the target type ip for workloads on hybrid nodes connected with AWS Direct Connect or AWS Site-to-Site VPN. You can alternatively use your choice of Ingress controller or load balancer for application traffic that stays local to your on-premises environment.

Metrics: You can use Amazon Managed Prometheus (AMP) agent-less scrapers, AWS Distro for Open Telemetry (ADOT), and the Amazon CloudWatch Observability Agent with hybrid nodes. To use AMP agent-less scrapers for Pod metrics on hybrid nodes, your Pods must be accessible from the VPC that you use for the Amazon EKS cluster.

Logs: You can enable Amazon EKS control plane logging for hybrid nodes-enabled clusters. You can use the ADOT EKS add-on and the Amazon CloudWatch Observability Agent EKS add-on for hybrid node and Pod logging.

Node management: The Amazon EKS Hybrid Nodes CLI is called nodeadm and is run on each on-premises host to simplify the installation, configuration, registration, and uninstall of the hybrid nodes components. The hybrid nodes nodeadm version is different than the nodeadm version used in the AL2023 Amazon EKS-optimized AMIs. You should not use the hybrid nodes nodeadm version for nodes running in Amazon EC2.

Cluster management: The Amazon EKS user interfaces for cluster management are the same with hybrid nodes-enabled Amazon EKS clusters. This includes the AWS Management Console, AWS API, AWS SDKs, AWS CLI, eksctl CLI, AWS CloudFormation, and Terraform.