**Help improve this page** To contribute to this user guide, choose the **Edit this page on GitHub** link that is located in the right pane of every page. # Connect a Kubernetes cluster to an Amazon EKS Management Console with Amazon EKS Connector You can use Amazon EKS Connector to register and connect any conformant Kubernetes cluster to AWS and visualize it in the Amazon EKS console. After a cluster is connected, you can see the status, configuration, and workloads for that cluster in the Amazon EKS console. You can use this feature to view connected clusters in Amazon EKS console, but you can’t manage them. The Amazon EKS Connector requires an agent that is an [open source project on Github](https://github.com/aws/amazon-eks-connector). For additional technical content, including frequently asked questions and troubleshooting, see [Troubleshoot Amazon EKS Connector issues](troubleshooting-connector.md). The Amazon EKS Connector can connect the following types of Kubernetes clusters to Amazon EKS. + On-premises Kubernetes clusters + Self-managed clusters that are running on Amazon EC2 + Managed clusters from other cloud providers ## Amazon EKS Connector considerations Before you use Amazon EKS Connector, understand the following: + You must have administrative privileges to the Kubernetes cluster to connect the cluster to Amazon EKS. + The Kubernetes cluster must have Linux 64-bit (x86) worker nodes present before connecting. ARM worker nodes aren’t supported. + You must have worker nodes in your Kubernetes cluster that have outbound access to the `ssm.` and `ssmmessages.` Systems Manager endpoints. For more information, see [Systems Manager endpoints](https://docs.aws.amazon.com/general/latest/gr/ssm.html) in the * AWS General Reference*. + By default, you can connect up to 10 clusters in a Region. You can request an increase through the [service quota console](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html). See [Requesting a quota increase](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html) for more information. + Only the Amazon EKS `RegisterCluster`, `ListClusters`, `DescribeCluster`, and `DeregisterCluster` APIs are supported for external Kubernetes clusters. + You must have the following permissions to register a cluster: + eks:RegisterCluster + ssm:CreateActivation + ssm:DeleteActivation + iam:PassRole + You must have the following permissions to deregister a cluster: + eks:DeregisterCluster + ssm:DeleteActivation + ssm:DeregisterManagedInstance ## Required IAM roles for Amazon EKS Connector Using the Amazon EKS Connector requires the following two IAM roles: + The [Amazon EKS Connector](using-service-linked-roles-eks-connector.md) service-linked role is created when you register a cluster for the first time. + You must create the Amazon EKS Connector agent IAM role. See [Amazon EKS connector IAM role](connector-iam-role.md) for details. To enable cluster and workload view permission for [IAM principals](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal), apply the `eks-connector` and Amazon EKS Connector cluster roles to your cluster. Follow the steps in [Grant access to view Kubernetes cluster resources on an Amazon EKS console](connector-grant-access.md). # Connect an external Kubernetes cluster to the Amazon EKS Management Console You can connect an external Kubernetes cluster to Amazon EKS by using multiple methods in the following process. This process involves two steps: Registering the cluster with Amazon EKS and installing the `eks-connector` agent in the cluster. **Important** You must complete the second step within 3 days of completing the first step, before the registration expires. ## Considerations You can use YAML manifests when installing the agent. Alternatively, you can use Helm if you register the cluster with the AWS Management Console or AWS Command Line Interface. However, you cannot use Helm to install the agent if you register the cluster with `eksctl`. ## Prerequisites + Ensure the Amazon EKS Connector agent role was created. Follow the steps in [Creating the Amazon EKS connector agent role](connector-iam-role.md#create-connector-role). + You must have the following permissions to register a cluster: + `eks:RegisterCluster` + `ssm:CreateActivation` + `ssm:DeleteActivation` + `iam:PassRole` ## Step 1: Registering the cluster To register a cluster to Amazon EKS connector, you can use one of these tools: + [AWS CLI](#awscli_register_cluster_connect) + [AWS Management Console](#console_register_cluster_connect) + [`eksctl`](#eksctl_register_cluster_connect) ### AWS CLI 1. AWS CLI must be installed. To install or upgrade it, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html). 1. For the Connector configuration, specify your Amazon EKS Connector agent IAM role. For more information, see [Required IAM roles for Amazon EKS Connector](eks-connector.md#connector-iam-permissions). ``` aws eks register-cluster \ --name my-first-registered-cluster \ --connector-config roleArn=arn:aws:iam::111122223333:role/AmazonEKSConnectorAgentRole,provider="OTHER" \ --region aws-region ``` An example output is as follows. ``` { "cluster": { "name": "my-first-registered-cluster", "arn": "arn:aws:eks:region:111122223333:cluster/my-first-registered-cluster", "createdAt": 1627669203.531, "ConnectorConfig": { "activationId": "xxxxxxxxACTIVATION_IDxxxxxxxx", "activationCode": "xxxxxxxxACTIVATION_CODExxxxxxxx", "activationExpiry": 1627672543.0, "provider": "OTHER", "roleArn": "arn:aws:iam::111122223333:role/AmazonEKSConnectorAgentRole" }, "status": "CREATING" } } ``` You use the `aws-region`, `activationId`, and `activationCode` values in the next step. ### AWS Management Console 1. Open the [Amazon EKS console](https://console.aws.amazon.com/eks/home#/clusters). 1. Choose **Add cluster** and select **Register** to bring up the configuration page. 1. On the **Configure cluster** section, fill in the following fields: + **Name** – A unique name for your cluster. + **Provider** – Choose to display the dropdown list of Kubernetes cluster providers. If you don’t know the specific provider, select **Other**. + **EKS Connector role** – Select the role to use for connecting the cluster. 1. Select **Register cluster**. 1. The Cluster overview page displays. If you want to use the Helm chart, copy the `helm install` command and continue to the next step. If you want to use the YAML manifest, choose **Download YAML file** to download the manifest file to your local drive. **Important** This is your only opportunity to copy the `helm install` command or download this file. Don’t navigate away from this page, as the link will not be accessible and you must deregister the cluster and start the steps from the beginning. The command or manifest file can be used only once for the registered cluster. If you delete resources from the Kubernetes cluster, you must re-register the cluster and obtain a new manifest file. Continue to the next step to apply the manifest file to your Kubernetes cluster. ### `eksctl` 1. `eksctl` version `0.68` or later must be installed. To install or upgrade it, see [Get started with Amazon EKS – `eksctl`](getting-started-eksctl.md). 1. Register the cluster by providing a name, provider, and region. ``` eksctl register cluster --name my-cluster --provider my-provider --region region-code ``` Example output: ``` 2021-08-19 13:47:26 [ℹ] creating IAM role "eksctl-20210819194112186040" 2021-08-19 13:47:26 [ℹ] registered cluster "" successfully 2021-08-19 13:47:26 [ℹ] wrote file eks-connector.yaml to 2021-08-19 13:47:26 [ℹ] wrote file eks-connector-clusterrole.yaml to 2021-08-19 13:47:26 [ℹ] wrote file eks-connector-console-dashboard-full-access-group.yaml to 2021-08-19 13:47:26 [!] note: "eks-connector-clusterrole.yaml" and "eks-connector-console-dashboard-full-access-group.yaml" give full EKS Console access to IAM identity "", edit if required; read https://eksctl.io/usage/eks-connector for more info 2021-08-19 13:47:26 [ℹ] run `kubectl apply -f eks-connector.yaml,eks-connector-clusterrole.yaml,eks-connector-console-dashboard-full-access-group.yaml` before expiry> to connect the cluster ``` This creates files on your local computer. These files must be applied to the external cluster within 3 days, or the registration expires. 1. In a terminal that can access the cluster, apply the `eks-connector-binding.yaml` file: ``` kubectl apply -f eks-connector-binding.yaml ``` ## Step 2: Installing the `eks-connector` agent To install the `eks-connector` agent, use one of the following tools: + [Helm](#helm_agent_cluster_connect) + [yaml](#yaml_agent_cluster_connect) ### Helm **Note** If you registered the cluster with `eksctl`, use the YAML manifest method instead of the Helm chart method. 1. If you used the AWS CLI in the previous step, replace the `ACTIVATION_CODE` and `ACTIVATION_ID` in the following command with the `activationId`, and `activationCode` values respectively. Replace the `aws-region` with the AWS Region that you used in the previous step. Then run the command to install the `eks-connector` agent on the registering cluster: ``` $ helm install eks-connector \ --namespace eks-connector \ oci://public.ecr.aws/eks-connector/eks-connector-chart \ --set eks.activationCode=ACTIVATION_CODE \ --set eks.activationId=ACTIVATION_ID \ --set eks.agentRegion=aws-region ``` If you used the AWS Management Console in the previous step, use the command that you copied from the previous step that has these values filled in. 1. Check the healthiness of the installed `eks-connector` deployment and wait for the status of the registered cluster in Amazon EKS to be `ACTIVE`. ### yaml Complete the connection by applying the Amazon EKS Connector manifest file to your Kubernetes cluster. To do this, you must use the methods described previously. If the manifest isn’t applied within three days, the Amazon EKS Connector registration expires. If the cluster connection expires, the cluster must be deregistered before connecting the cluster again. 1. Download the Amazon EKS Connector YAML file. ``` curl -O https://amazon-eks.s3.us-west-2.amazonaws.com/eks-connector/manifests/eks-connector/latest/eks-connector.yaml ``` 1. Edit the Amazon EKS Connector YAML file to replace all references of `%AWS_REGION%`, `%EKS_ACTIVATION_ID%`, `%EKS_ACTIVATION_CODE%` with the `aws-region`, `activationId`, and `activationCode` from the output of the previous step. The following example command can replace these values. ``` sed -i "s~%AWS_REGION%~$aws-region~g; s~%EKS_ACTIVATION_ID%~$EKS_ACTIVATION_ID~g; s~%EKS_ACTIVATION_CODE%~$(echo -n $EKS_ACTIVATION_CODE | base64)~g" eks-connector.yaml ``` **Important** Ensure that your activation code is in the base64 format. 1. In a terminal that can access the cluster, you can apply the updated manifest file by running the following command: ``` kubectl apply -f eks-connector.yaml ``` 1. After the Amazon EKS Connector manifest and role binding YAML files are applied to your Kubernetes cluster, confirm that the cluster is now connected. ``` aws eks describe-cluster \ --name "my-first-registered-cluster" \ --region AWS_REGION ``` The output should include `status=ACTIVE`. 1. (Optional) Add tags to your cluster. For more information, see [Organize Amazon EKS resources with tags](eks-using-tags.md). ## Next steps If you have any issues with these steps, see [Troubleshoot Amazon EKS Connector issues](troubleshooting-connector.md). To grant additional [IAM principals](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal) access to the Amazon EKS console to view Kubernetes resources in a connected cluster, see [Grant access to view Kubernetes cluster resources on an Amazon EKS console](connector-grant-access.md). # Grant access to view Kubernetes cluster resources on an Amazon EKS console Grant [IAM principals](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal) access to the Amazon EKS console to view information about Kubernetes resources running on your connected cluster. ## Prerequisites The [IAM principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal) that you use to access the AWS Management Console must meet the following requirements: + It must have the `eks:AccessKubernetesApi` IAM permission. + The Amazon EKS Connector service account can impersonate the IAM principal in the cluster. This allows the Amazon EKS Connector to map the IAM principal to a Kubernetes user. **To create and apply the Amazon EKS Connector cluster role** 1. Download the `eks-connector` cluster role template. ``` curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/eks-connector/manifests/eks-connector-console-roles/eks-connector-clusterrole.yaml ``` 1. Edit the cluster role template YAML file. Replace references of `%IAM_ARN%` with the Amazon Resource Name (ARN) of your IAM principal. 1. Apply the Amazon EKS Connector cluster role YAML to your Kubernetes cluster. ``` kubectl apply -f eks-connector-clusterrole.yaml ``` For an IAM principal to view Kubernetes resources in Amazon EKS console, the principal must be associated with a Kubernetes `role` or `clusterrole` with necessary permissions to read the resources. For more information, see [Using RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) in the Kubernetes documentation. **To configure an IAM principal to access the connected cluster** 1. You can download either of these example manifest files to create a `clusterrole` and `clusterrolebinding` or a `role` and `rolebinding`, respectively: **View Kubernetes resources in all namespaces** + The `eks-connector-console-dashboard-full-access-clusterrole` cluster role gives access to all namespaces and resources that can be visualized in the console. You can change the name of the `role`, `clusterrole` and their corresponding binding before applying it to your cluster. Use the following command to download a sample file. ``` curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/eks-connector/manifests/eks-connector-console-roles/eks-connector-console-dashboard-full-access-group.yaml ``` **View Kubernetes resources in a specific namespace** + The namespace in this file is `default`, so if you want to specify a different namespace, edit the file before applying it to your cluster. Use the following command to download a sample file. ``` curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/eks-connector/manifests/eks-connector-console-roles/eks-connector-console-dashboard-restricted-access-group.yaml ``` 1. Edit the full access or restricted access YAML file to replace references of `%IAM_ARN%` with the Amazon Resource Name (ARN) of your IAM principal. 1. Apply the full access or restricted access YAML files to your Kubernetes cluster. Replace the YAML file value with your own. ``` kubectl apply -f eks-connector-console-dashboard-full-access-group.yaml ``` To view Kubernetes resources in your connected cluster, see [View Kubernetes resources in the AWS Management Console](view-kubernetes-resources.md). Data for some resource types on the **Resources** tab isn’t available for connected clusters. # Deregister a Kubernetes cluster from the Amazon EKS console If you are finished using a connected cluster, you can deregister it. After it’s deregistered, the cluster is no longer visible in the Amazon EKS console. You must have the following permissions to call the deregisterCluster API: + `eks:DeregisterCluster` + `ssm:DeleteActivation` + `ssm:DeregisterManagedInstance` This process involves two steps: Deregistering the cluster with Amazon EKS and uninstalling the eks-connector agent in the cluster. ## Deregister the Kubernetes cluster To deregister a cluster from Amazon EKS connector, you can use one of these tools: + [AWS CLI](#awscli_deregister_cluster_connect) + [AWS Management Console](#console_deregister_cluster_connect) + [`eksctl`](#eksctl_deregister_cluster_connect) ### AWS CLI 1. AWS CLI must be installed. To install or upgrade it, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html). 1. Ensure the Amazon EKS Connector agent role was created. 1. Deregister the connected cluster. ``` aws eks deregister-cluster \ --name my-cluster \ --region region-code ``` ### AWS Management Console 1. Open the [Amazon EKS console](https://console.aws.amazon.com/eks/home#/clusters). 1. Choose **Clusters**. 1. On the **Clusters** page, select the connected cluster and select **Deregister**. 1. Confirm that you want to deregister the cluster. ### `eksctl` 1. Install `eksctl` version `0.68` or later. To install or upgrade it, see [Get started with Amazon EKS – `eksctl`](getting-started-eksctl.md). 1. Ensure the Amazon EKS Connector agent role was created. 1. Deregister the connected cluster: ``` eksctl deregister cluster --name my-cluster ``` ## Clean up the resources in your Kubernetes cluster To uninstall the `eks-connector` agent, use one of the following tools: + [helm](#helm_agent_cluster_deregister) + [yaml](#yaml_agent_cluster_deregister) ### helm Run the following command to uninstall the agent. ``` helm -n eks-connector uninstall eks-connector ``` ### yaml 1. Delete the Amazon EKS Connector YAML file from your Kubernetes cluster. ``` kubectl delete -f eks-connector.yaml ``` 1. If you created `clusterrole` or `clusterrolebindings` for additional [IAM principals](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal) to access the cluster, delete them from your Kubernetes cluster. # Troubleshoot Amazon EKS Connector issues This topic covers some of the common errors that you might encounter while using the Amazon EKS Connector, including instructions on how to resolve them and workarounds. ## Basic troubleshooting This section describes steps to diagnose Amazon EKS Connector issues. ### Check Amazon EKS Connector status To check the Amazon EKS Connector status, type: ``` kubectl get pods -n eks-connector ``` ### Inspect Amazon EKS Connector logs The Amazon EKS Connector Pod consists of three containers. To retrieve full logs for all of these containers so that you can inspect them, run the following commands: + `connector-init` ``` kubectl logs eks-connector-0 --container connector-init -n eks-connector kubectl logs eks-connector-1 --container connector-init -n eks-connector ``` + `connector-proxy` ``` kubectl logs eks-connector-0 --container connector-proxy -n eks-connector kubectl logs eks-connector-1 --container connector-proxy -n eks-connector ``` + `connector-agent` ``` kubectl exec eks-connector-0 --container connector-agent -n eks-connector -- cat /var/log/amazon/ssm/amazon-ssm-agent.log kubectl exec eks-connector-1 --container connector-agent -n eks-connector -- cat /var/log/amazon/ssm/amazon-ssm-agent.log ``` ### Get the effective cluster name Amazon EKS clusters are uniquely identified by `clusterName` within a single AWS account and AWS Region. If you have multiple connected clusters in Amazon EKS, you can confirm which Amazon EKS cluster that the current Kubernetes cluster is registered to. To do this, enter the following to find out the `clusterName` of the current cluster. ``` kubectl exec eks-connector-0 --container connector-agent -n eks-connector \ -- cat /var/log/amazon/ssm/amazon-ssm-agent.log | grep -m1 -oE "eks_c:[a-zA-Z0-9_-]+" | sed -E "s/^.*eks_c:([a-zA-Z0-9_-]+)_[a-zA-Z0-9]+.*$/\1/" kubectl exec eks-connector-1 --container connector-agent -n eks-connector \ -- cat /var/log/amazon/ssm/amazon-ssm-agent.log | grep -m1 -oE "eks_c:[a-zA-Z0-9_-]+" | sed -E "s/^.*eks_c:([a-zA-Z0-9_-]+)_[a-zA-Z0-9]+.*$/\1/" ``` ### Miscellaneous commands The following commands are useful to retrieve information that you need to troubleshoot issues. + Use the following command to gather images that’s used by Pods in Amazon EKS Connector. ``` kubectl get pods -n eks-connector -o jsonpath="{.items[*].spec.containers[*].image}" | tr -s '[[:space:]]' '\n' ``` + Use the following command to determine the node names that Amazon EKS Connector is running on. ``` kubectl get pods -n eks-connector -o jsonpath="{.items[*].spec.nodeName}" | tr -s '[[:space:]]' '\n' ``` + Run the following command to get your Kubernetes client and server versions. ``` kubectl version ``` + Run the following command to get information about your nodes. ``` kubectl get nodes -o wide --show-labels ``` ## Helm issue: 403 Forbidden If you received the following error when running helm install commands: ``` Error: INSTALLATION FAILED: unexpected status from HEAD request to https://public.ecr.aws/v2/eks-connector/eks-connector-chart/manifests/0.0.6: 403 Forbidden ``` You can run the following line to fix it: ``` docker logout public.ecr.aws ``` ## Console error: the cluster is stuck in the Pending state If the cluster gets stuck in the `Pending` state on the Amazon EKS console after you’re registered it, it might be because the Amazon EKS Connector didn’t successfully connect the cluster to AWS yet. For a registered cluster, the `Pending` state means that the connection isn’t successfully established. To resolve this issue, make sure that you have applied the manifest to the target Kubernetes cluster. If you applied it to the cluster, but the cluster is still in the `Pending` state, then the `eks-connector` statefulset might be unhealthy. To troubleshoot this issue, see [Amazon EKS connector Pods are crash looping](#symp-loop)in this topic. ## Console error: User system:serviceaccount:eks-connector:eks-connector can’t impersonate resource users in API group at cluster scope The Amazon EKS Connector uses Kubernetes [user impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation) to act on behalf of [IAM principals](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal) from the AWS Management Console. Each principal that accesses the Kubernetes API from the AWS `eks-connector` service account must be granted permission to impersonate the corresponding Kubernetes user with an IAM ARN as its Kubernetes user name. In the following examples, the IAM ARN is mapped to a Kubernetes user. + IAM user *john* from AWS account *111122223333* is mapped to a Kubernetes user. [IAM best practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html) recommend that you grant permissions to roles instead of users. ``` arn:aws:iam::111122223333:user/john ``` + IAM role *admin* from AWS account *111122223333* is mapped to a Kubernetes user: ``` arn:aws:iam::111122223333:role/admin ``` The result is an IAM role ARN, instead of the AWS STS session ARN. For instructions on how to configure the `ClusterRole` and `ClusterRoleBinding` to grant the `eks-connector` service account privilege to impersonate the mapped user, see [Grant access to view Kubernetes cluster resources on an Amazon EKS console](connector-grant-access.md). Make sure that in the template, `%IAM_ARN%` is replaced with the IAM ARN of the AWS Management Console IAM principal. ## Console error: […​] is forbidden: User […​] cannot list resource […​] in API group at the cluster scope Consider the following problem. The Amazon EKS Connector has successfully impersonated the requesting AWS Management Console IAM principal in the target Kubernetes cluster. However, the impersonated principal doesn’t have RBAC permission for Kubernetes API operations. To resolve this issue, there are two methods to give permissions to additional users. If you previously installed eks-connector via helm chart, you can easily grant users access by running the following command. Replace the `userARN1` and `userARN2` with a list of the ARNs of the IAM roles to give access to view the Kubernetes resources: ``` helm upgrade eks-connector oci://public.ecr.aws/eks-connector/eks-connector-chart \ --reuse-values \ --set 'authentication.allowedUserARNs={userARN1,userARN2}' ``` Or, as the cluster administrator, grant the appropriate level of RBAC privileges to individual Kubernetes users. For more information and examples, see [Grant access to view Kubernetes cluster resources on an Amazon EKS console](connector-grant-access.md). ## Console error: Amazon EKS can’t communicate with your Kubernetes cluster API server. The cluster must be in an ACTIVE state for successful connection. Try again in few minutes. If the Amazon EKS service can’t communicate with the Amazon EKS connector in the target cluster, it might be because of one of the following reasons: + The Amazon EKS Connector in the target cluster is unhealthy. + Poor connectivity or an interrupted connection between the target cluster and the AWS Region. To resolve this problem, check the [Amazon EKS Connector logs](#tsc-logs). If you don’t see an error for the Amazon EKS Connector, retry the connection after a few minutes. If you regularly experience high latency or intermittent connectivity for the target cluster, consider re-registering the cluster to an AWS Region that’s located closer to you. ## Amazon EKS connector Pods are crash looping There are many reasons that can cause an Amazon EKS connector Pod to enter the `CrashLoopBackOff` status. This issue likely involves the `connector-init` container. Check the status of the Amazon EKS connector Pod. ``` kubectl get pods -n eks-connector ``` An example output is as follows. ``` NAME READY STATUS RESTARTS AGE eks-connector-0 0/2 Init:CrashLoopBackOff 1 7s ``` If your output is similar to the previous output, see [Inspect Amazon EKS Connector logs](#tsc-logs) to troubleshoot the issue. ## Failed to initiate eks-connector: InvalidActivation When you start the Amazon EKS Connector for the first time, it registers an `activationId` and `activationCode` with Amazon Web Services. The registration might fail, which can cause the `connector-init` container to crash with an error similar to the following error. ``` F1116 20:30:47.261469 1 init.go:43] failed to initiate eks-connector: InvalidActivation: ``` To troubleshoot this issue, consider the following causes and recommended fixes: + Registration might have failed because the `activationId` and `activationCode` aren’t in your manifest file. If this is the case, make sure that they are the correct values that were returned from the `RegisterCluster` API operation, and that the `activationCode` is in the manifest file. The `activationCode` is added to Kubernetes secrets, so it must be `base64` encoded. For more information, see [Step 1: Registering the cluster](connecting-cluster.md#connector-connecting). + Registration might have failed because your activation expired. This is because, for security reasons, you must activate the Amazon EKS Connector within three days after registering the cluster. To resolve this issue, make sure that the Amazon EKS Connector manifest is applied to the target Kubernetes cluster before the expiry date and time. To confirm your activation expiry date, call the `DescribeCluster` API operation. ``` aws eks describe-cluster --name my-cluster ``` In the following example response, the expiry date and time is recorded as `2021-11-12T22:28:51.101000-08:00`. ``` { "cluster": { "name": "my-cluster", "arn": "arn:aws:eks:region:111122223333:cluster/my-cluster", "createdAt": "2021-11-09T22:28:51.449000-08:00", "status": "FAILED", "tags": { }, "connectorConfig": { "activationId": "00000000-0000-0000-0000-000000000000", "activationExpiry": "2021-11-12T22:28:51.101000-08:00", "provider": "OTHER", "roleArn": "arn:aws:iam::111122223333:role/my-connector-role" } } } ``` If the `activationExpiry` passed, deregister the cluster and register it again. Doing this generates a new activation. ## Cluster node is missing outbound connectivity To work properly, the Amazon EKS Connector requires outbound connectivity to several AWS endpoints. You can’t connect a private cluster without outbound connectivity to a target AWS Region. To resolve this issue, you must add the necessary outbound connectivity. For information about connector requirements, see [Amazon EKS Connector considerations](eks-connector.md#connect-cluster-reqts). ## Amazon EKS connector Pods are in `ImagePullBackOff` state If you run the `get pods` command and Pods are in the `ImagePullBackOff` state, they can’t work properly. If the Amazon EKS Connector Pods are in the `ImagePullBackOff` state, they can’t work properly. Check the status of your Amazon EKS Connector Pods. ``` kubectl get pods -n eks-connector ``` An example output is as follows. ``` NAME READY STATUS RESTARTS AGE eks-connector-0 0/2 Init:ImagePullBackOff 0 4s ``` The default Amazon EKS Connector manifest file references images from the [Amazon ECR Public Gallery](https://gallery.ecr.aws/). It’s possible that the target Kubernetes cluster can’t pull images from the Amazon ECR Public Gallery. Either resolve the Amazon ECR Public Gallery image pull issue, or consider mirroring the images in the private container registry of your choice. # AWS Connector frequently asked questions **Q: How does the underlying technology behind the Amazon EKS Connector work?** A: The Amazon EKS Connector is based on the AWS Systems Manager (Systems Manager) agent. The Amazon EKS Connector runs as a `StatefulSet` on your Kubernetes cluster. It establishes a connection and proxies the communication between the API server of your cluster and Amazon Web Services. It does this to display cluster data in the Amazon EKS console until you disconnect the cluster from AWS. The Systems Manager agent is an open source project. For more information about this project, see the [GitHub project page](https://github.com/aws/amazon-ssm-agent). **Q: I have an on-premises Kubernetes cluster that I want to connect. Do I need to open firewall ports to connect it?** A: No, you don’t need to open any firewall ports. The Kubernetes cluster only requires outbound connection to AWS Regions. AWS services never access resources in your on-premises network. The Amazon EKS Connector runs on your cluster and initiates the connection to AWS. When the cluster registration completes, AWS only issues commands to the Amazon EKS Connector after you start an action from the Amazon EKS console that requires information from the Kubernetes API server on your cluster. **Q: What data is sent from my cluster to AWS by the Amazon EKS Connector?** A: The Amazon EKS Connector sends technical information that’s necessary for your cluster to be registered on AWS. It also sends cluster and workload metadata for the Amazon EKS console features that customers request. The Amazon EKS Connector only gathers or sends this data if you start an action from the Amazon EKS console or the Amazon EKS API that necessitates the data to be sent to AWS. Other than the Kubernetes version number, AWS doesn’t store any data by default. It stores data only if you authorize it to. **Q: Can I connect a cluster outside of an AWS Region?** A: Yes, you can connect a cluster from any location to Amazon EKS. Moreover, your Amazon EKS service can be located in any AWS public commercial AWS Region. This works with a valid network connection from your cluster to the target AWS Region. We recommend that you pick an AWS Region that is closest to your cluster location for UI performance optimization. For example, if you have a cluster running in Tokyo, connect your cluster to the AWS Region in Tokyo (that is, the `ap-northeast-1` AWS Region) for low latency. You can connect a cluster from any location to Amazon EKS in any of the public commercial AWS Regions, except the China or GovCloud AWS Regions. # Understand security in Amazon EKS Connector The Amazon EKS Connector is an open source component that runs on your Kubernetes cluster. This cluster can be located outside of the AWS environment. This creates additional considerations for security responsibilities. This configuration can be illustrated by the following diagram. Orange represents AWS responsibilities, and blue represents customer responsibilities: ![\[EKS Connector Responsibilities\]](http://docs.aws.amazon.com/eks/latest/userguide/images/connector-model.png) This topic describes the differences in the responsibility model if the connected cluster is outside of AWS. ## AWS responsibilities + Maintaining, building, and delivering Amazon EKS Connector, which is an [open source component](https://github.com/aws/amazon-eks-connector) that runs on a customer’s Kubernetes cluster and communicates with AWS. + Maintaining transport and application layer communication security between the connected Kubernetes cluster and AWS services. ## Customer responsibilities + Kubernetes cluster specific security, specifically along the following lines: + Kubernetes secrets must be properly encrypted and protected. + Lock down access to the `eks-connector` namespace. + Configuring role-based access control (RBAC) permissions to manage [IAM principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html#iam-term-principal) access from AWS. For instructions, see [Grant access to view Kubernetes cluster resources on an Amazon EKS console](connector-grant-access.md). + Installing and upgrading Amazon EKS Connector. + Maintaining the hardware, software, and infrastructure that supports the connected Kubernetes cluster. + Securing their AWS accounts (for example, through safeguarding your [root user credentials](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#lock-away-credentials)).