# Secure Kubernetes with AWS Private Certificate Authority
<a name="PcaKubernetes"></a>

You can use AWS Private Certificate Authority to provide certificates for secure authentication and encryption over TLS and mTLS. AWS Private CA provides an open source plugin, [AWS Private CA Connector for Kubernetes](https://github.com/cert-manager/aws-privateca-issuer), (`aws-privateca-issuer`) for the widely adopted [cert-manager](https://cert-manager.io/docs/) add-on to Kubernetes that requests certificates, distributes them to Kubernetes secrets, and automates certificate renewal.

The `aws-privateca-issuer` plugin allows you to issue AWS Private CA certificates through `cert-manager`. You can use the plugin with Amazon Elastic Kubernetes Service (Amazon EKS), a self-managed Kubernetes cluster on AWS, or in an on-premise Kubernetes cluster. The plugin works on both x86 and ARM architectures.

AWS Private CA has HSM backed keys that can't be exported. If you have regulatory requirements for controlling access and auditing your CA operations, you can use AWS Private CA to improve auditability and to support compliance.

**Note**  
If you are running on Amazon EKS, we recommend that you use the `cert-manager` and `aws-privateca-connector-for-kubernetes` add-ons for a managed installation experience. For more information, refer to [AWS add-ons](https://docs.aws.amazon.com/eks/latest/userguide/workloads-add-ons-available-eks.html#add-ons-aws-privateca-connector).

# Concepts
<a name="PcaKubernetes-concepts"></a>

The following diagram shows some of the options available for using TLS in an Amazon EKS cluster. The example cluster sits behind a load balancer. The numbers identify possible endpoints for TLS-secured communications.

![\[A diagram showing the possible endpoints for TLS encryption. Each endpoint has a number that corresponds to the following list.\]](http://docs.aws.amazon.com/privateca/latest/userguide/images/kubernetes-pca.png)


1. **Termination at the load balancer**

   Elastic Load Balancing (Elastic Load Balancing) is integrated with the AWS Certificate Manager service. You don't need to install `cert-manager` on the load balancer. You can provision ACM with a private CA, sign a certificate with the private CA, and install the certificate using the Elastic Load Balancing console. AWS Private CA certificates are automatically renewed.

   As an alternative, you can provide a private certificate to a non-AWS load balancer to terminate TLS.

   This provides encrypted communication between a remote client and the load balancer. Data after the load balancer is passed unencrypted to the Amazon EKS cluster.

1. **Termination at the Kubernetes ingress controller**

   The ingress controller is inside the Amazon EKS cluster and acts as a load balancer and router. To use the ingress controller as the cluster's endpoint for external communication, you must:
   + Install both `cert-manager` and `aws-privateca-issuer` 
   + Provision the controller with a TLS private certificate from the AWS Private CA.

   Communications between the load balancer and the ingress controller are encrypted, data passes unencrypted to the cluster's resources.

1. **Termination at a pod**

   Each pod is a group of one or more containers that share storage and network resources. If you install both `cert-manager` and `aws-privateca-issuer` and provision the cluster with a private CA, Kubernetes can install a signed TLS private certificate on pods as needed. A TLS connection terminating at a pod is unavailable to other pods in the cluster by default. 

1. **Secure communications between pods.**

   You can provision multiple pods with certificates to allow them to communicate with one another. The following scenarios are possible:
   + Provisioning with Kubernetes generated self-signed certificates. This secures communications between pods, but self-signed certificates don't satisfy HIPAA or FIPS requirements.
   + Provisioning with certificates signed by AWS Private CA. This requires installing both `cert-manager` and `aws-privateca-issuer`. Kubernetes can then install signed mTLS certificates on the pods as needed.

# Considerations
<a name="PcaKubernetes-considerations"></a>

When using AWS Private Certificate Authority with Kubernetes, keep the following considerations in mind.

## Cross-account use of cert-manager
<a name="kubernetes-cross-account"></a>

Administrators with cross-account access to a CA can use the `cert-manager` add on for Kubernetes to provision certificates for a cluster using the shared CA. For more information, refer to [Security best practices for Cross-account access to private CAs](pca-resource-sharing.md).

You can use only certain AWS Private CA certificate templates in cross-account scenarios.

The following table lists AWS Private CA templates that you can use with cert-manager to provision a Kubernetes cluster.


| Templates supported for Kubernetes | Support for cross-account use | 
| --- | --- | 
| [BlankEndEntityCertificate\$1CSRPassthrough/V1 definition](template-definitions.md#BlankEndEntityCertificate_CSRPassthrough) | No | 
| [CodeSigningCertificate/V1 definition](template-definitions.md#CodeSigningCertificate-V1) | No | 
| [EndEntityCertificate/V1 definition](template-definitions.md#EndEntityCertificate-V1) | Yes | 
| [EndEntityClientAuthCertificate/V1 definition](template-definitions.md#EndEntityClientAuthCertificate-V1) | Yes | 
| [EndEntityServerAuthCertificate/V1 definition](template-definitions.md#EndEntityServerAuthCertificate-V1) | Yes | 
| [OCSPSigningCertificate/V1 definition](template-definitions.md#OCSPSigningCertificate-V1) | No | 

# Get started with AWS Private CA Connector for Kubernetes.
<a name="PcaKubernetes-get-started"></a>

The following topics show how to use AWS Private CA to secure communications in a Kubernetes cluster. For another example, refer to [ Encryption in transit for Kubernetes ](https://github.com/aws-samples/sample-encryption-in-transit-for-kubernetes) on GitHub.

You can use a private certificate authority to secure communications with your Amazon EKS clusters. Before you begin, ensure that you have the following:
+ An AWS account with appropriate permissions scoped to your security policies.

------
#### [ Amazon EKS clusters ]

------
#### [ JSON ]

****  

  ```
  {
    "Version":"2012-10-17",		 	 	 
    "Statement": [
      {
        "Sid": "IAM",
        "Effect": "Allow",
        "Action": [
          "iam:CreateRole",
          "iam:AttachRolePolicy",
          "iam:GetRole"
        ],
        "Resource": "*"
      },
      {
        "Sid": "EKS",
        "Effect": "Allow",
        "Action": [
          "eks:CreateAddon",
          "eks:DescribeAddon",
          "eks:CreatePodIdentityAssociation",
          "eks:DescribeCluster"
        ],
        "Resource": "*"
      },
      {
        "Sid": "IAMPassRole",
        "Effect": "Allow",
        "Action": [
          "iam:PassRole"
        ],
        "Resource": "arn:aws:iam::*:role/CertManagerPrivateCARole"
      }
    ]
  }
  ```

------

------
#### [ Other clusters ]

------
#### [ JSON ]

****  

  ```
  {
    "Version":"2012-10-17",		 	 	 
    "Statement": [
      {
        "Sid": "GetAndIssuePCACertificates",
        "Effect": "Allow",
        "Action": [
          "acm-pca:GetCertificate",
          "acm-pca:IssueCertificate"
        ],
        "Resource": "*"
      },
      {
        "Sid": "RolesAnywhere",
        "Effect": "Allow",
        "Action": [
          "rolesanywhere:CreateProfile"
        ],
        "Resource": "*"
      },
      {
        "Sid": "IAM",
        "Effect": "Allow",
        "Action": [
          "iam:CreateRole",
          "iam:AttachRolePolicy"
        ],
        "Resource": "*"
      },
      {
        "Sid": "IAMPassRole",
        "Effect": "Allow",
        "Action": [
          "iam:PassRole"
        ],
        "Resource": "arn:aws:iam::*:role/CertManagerPrivateCARole"
      }
    ]
  }
  ```

------

------
+ A Kubernetes cluster. To create a Amazon Elastic Kubernetes Service cluster, refer to the [Amazon EKS quickstart guide](https://docs.aws.amazon.com/eks/latest/userguide/quickstart.html). For simplicity, create an environment variable to hold the cluster name:

  ```
  export CLUSTER=aws-privateca-demo
  ```

  
+ The AWS Region where your CA and Amazon EKS cluster are located. For simplicity, create an environment variable to hold the Region:

  ```
  export REGION=aws-region
  ```
+ The Amazon Resource Name (ARN) of a AWS Private CA private certificate authority. For simplicity, create an environment variable to hold the private CA ARN:

  ```
  export CA_ARN="arn:aws:acm-pca:region:account:certificate-authority/CA_ID/certificate/certificate_ID"
  ```

  To create a private CA, refer to [https://docs.aws.amazon.com/privateca/latest/userguide/create-CA.html](https://docs.aws.amazon.com/privateca/latest/userguide/create-CA.html)Create a private CA in AWS Private CA
+ A computer with the following software installed:
  + [AWS CLI v2](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) configured
  + [kubectl v1.13\$1](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
  + For non-Amazon EKS clusters, [Helm v3](https://helm.sh/docs/intro/install/)

## Install cert-manager
<a name="kubernetes-install-cert-manager"></a>

To use a private CA, you must install the `cert-manager>` add-on that requests certificates, distributes them, and automates certificate renewal. You must also install the `aws-private-ca-issuer` plugin that allows you to issue private certificates from AWS Private CA. Use the following steps to install the add-on and plugin.

------
#### [ Amazon EKS clusters ]

Install `cert-manager` as an Amazon EKS add-on:

```
aws eks create-addon \
  --cluster-name $CLUSTER \
  --addon-name cert-manager \
  --region $REGION
```

------
#### [ Other clusters ]

Install `cert-manager` using Helm:

```
helm repo add jetstack https://charts.jetstack.io
helm repo update

helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true
```

------

## Configure IAM permissions
<a name="kubernetes-iam-permissions"></a>

The `aws-privateca-issuer` plugin requires permission the interact with AWS Private CA. For Amazon EKS clusters you use the pod identity. For other clusters you use AWS Identity and Access Management Roles Anywhere.

Fist, create an IAM policy. The policy uses the `AWSPrivateCAConnectorForKubernetesPolicy` managed policy. For more information about the policy, refer to [AWSPrivateCAConnectorForKubernetesPolicy](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AWSPrivateCAConnectorForKubernetesPolicy.html) in the *AWS Managed policy reference guide*.

------
#### [ Amazon EKS clusters ]

1. Create a file named `trust-policy.json` containing the following trust policy:

------
#### [ JSON ]

****  

   ```
   {
     "Version":"2012-10-17",		 	 	 
     "Statement": [
       {
         "Sid": "TrustPolicyForEKSClusters",
         "Effect": "Allow",
         "Principal": {
           "Service": "pods.eks.amazonaws.com"
         },
         "Action": [
           "sts:AssumeRole",
           "sts:TagSession"
         ]
       }
     ]
   }
   ```

------

1. Run the following commands to create an IAM role:

   ```
   ROLE_ARN=$(aws iam create-role \
     --role-name CertManagerPrivateCARole \
     --assume-role-policy-document file://trust-policy.json \
     --region $REGION \
     --output text \
     --query "Role.Arn")
    
    aws iam attach-role-policy \
     --role-name CertManagerPrivateCARole \
     --policy-arn arn:aws:iam::aws:policy/AWSPrivateCAConnectorForKubernetesPolicy
   ```

------
#### [ Other clusters ]

1. Create a trust anchor that trusts the private CA stored in `CA_ARN`. For instructions, refer to [Getting started with IAM Roles Anywhere](https://docs.aws.amazon.com/rolesanywhere/latest/userguide/getting-started.html). Create an environment variable to store the trust anchor ARN:

   ```
   export TRUST_ANCHOR_ARN=trustAnchorArn
   ```

1. Create a file called `trust-policy.json` containing the following trust policy:

------
#### [ JSON ]

****  

   ```
   {
       "Version":"2012-10-17",		 	 	 
       "Statement": [
           {
               "Sid": "TrustPolicyForSelfManagedOrOnPremiseClusters",
               "Effect": "Allow",
               "Principal": {
                   "Service": "rolesanywhere.amazonaws.com"
               },
               "Action": [
                   "sts:AssumeRole",
                   "sts:SetSourceIdentity",
                   "sts:TagSession"
               ],
               "Condition": {
                   "ArnEquals": {
                       "aws:SourceArn": [
                           "arn:aws:rolesanywhere:us-east-1:123456789012:trust-anchor/TRUST_ANCHOR_ARN"
                       ]
                   },
                   "StringEquals": {
                       "aws:PrincipalTag/x509Subject/CN": "aws-privateca-issuer"
                   }
               }
           }
       ]
   }
   ```

------

1. Run the following commands to create an IAM role:

   ```
   ROLE_ARN=$(aws iam create-role \
     --role-name CertManagerPrivateCARole \
     --assume-role-policy-document file://trust-policy.json \
     --query "Role.Arn" \
     --region $REGION \
     --output text)
     
    aws iam attach-role-policy \
     --role-name CertManagerPrivateCARole \
     --region $REGION \
     --policy-arn arn:aws:iam::aws:policy/AWSPrivateCAConnectorForKubernetesPolicy
   ```

------

## Install and configure the AWS Private CA cluster issuer
<a name="kubernetes-install-aws-private-ca-cluster-issuer"></a>

To install the `aws-privateca-connector-for-kubernetes` add-on, use the following commands: 

------
#### [ Amazon EKS clusters ]

Create the add-on:

```
aws eks create-addon --region $REGION \
  --cluster-name $CLUSTER \
  --addon-name aws-privateca-connector-for-kubernetes \
  --pod-identity-associations "[{
    \"serviceAccount\": \"aws-privateca-issuer\",
    \"roleArn\": \"$ROLE_ARN\"
  }]"
```

Then wait for the add-on to be active:

```
aws eks describe-addon \
  --cluster-name $CLUSTER \
  --addon-name aws-privateca-connector-for-kubernetes \
  --region $REGION \
  --query 'addon.status'
```

------
#### [ Other clusters ]

1. Create a profile in IAM Roles Anywhere:

   ```
   PROFILE_ARN=$(aws rolesanywhere create-profile \
     --name "privateca-profile" \
     --role-arns "$ROLE_ARN" \
     --region "$REGION" \
     --query 'profile.profileArn' \
     --enabled \
     --output text)
   ```

1. Generate a client certificate for use with the Connector for Kubernetes and IAM Roles Anywhere to authenticate with AWS Private CA:

   1. Generate a private key for the client certificate:

      ```
      openssl genrsa -out client.key 2048
      ```

   1. Generate a certificate signing request (CSR) for the client certificate:

      ```
      openssl req -new \
        -key client.key \
        -out client.csr \
        -subj "/CN=aws-privateca-issuer"
      ```

   1. Issue the client certificate from AWS Private CA:

      ```
      CERT_ARN=$(aws acm-pca issue-certificate \
        --signing-algorithm SHA256WITHRSA \
        --csr fileb://client.csr \
        --validity Value=1,Type=DAYS \
        --certificate-authority-arn "$CA_ARN" \
        --region "$REGION" \
        --query 'CertificateArn' \
        --output text)
      ```

   1. Store the client certificate locally:

      ```
      aws acm-pca get-certificate \
        --certificate-authority-arn $CA_ARN \
        --certificate-arn $CERT_ARN \
        --region $REGION \
        --query 'Certificate' 
        --output text > pca-issuer-client-cert.pem
      ```

1. Install the AWS Private CA issuer in the cluster with the client certificate:

   1. Add the `awspca` Helm repository:

      ```
      helm repo add awspca https://cert-manager.github.io/aws-privateca-issuer 
      helm repo update
      ```

   1. Create a namespace:

      ```
      kubectl create namespace aws-privateca-issuer
      ```

   1. Put the certificate created earlier into a secret:

      ```
      kubectl create secret tls aws-privateca-credentials \
        -n aws-privateca-issuer \
        --cert=pca-issuer-client-cert.pem \
        --key=client.key
      ```

1. Install the AWS Private CA issuer with IAM Roles Anywhere:

   1. Create a file named `values.yaml` to configure the AWS Private CA issuer plugin to use with IAM Roles Anywhere: 

      ```
      cat > values.yaml <<EOF
      env:
        AWS_EC2_METADATA_SERVICE_ENDPOINT: "http://127.0.0.1:9911"
      
      extraContainers:
        - name: "rolesanywhere-credential-helper"
          image: "public.ecr.aws/rolesanywhere/credential-helper:latest"
          command: ["aws_signing_helper"]
          args:
            - "serve"
            - "--private-key"
            - "/etc/cert/tls.key"
            - "--certificate"
            - "/etc/cert/tls.crt"
            - "--role-arn"
            - "$ROLE_ARN"
            - "--profile-arn"
            - "$PROFILE_ARN"
            - "--trust-anchor-arn"
            - "$TRUST_ANCHOR_ARN"
          volumeMounts:
            - name: cert
              mountPath: /etc/cert/
              readOnly: true
      
      volumes:
        - name: cert
          secret:
            secretName: aws-privateca-credentials
      EOF
      ```

   1. Install the AWS Private CA issuer with IAM Roles Anywhere:

      ```
      helm install aws-privateca-issuer awspca/aws-privateca-issuer \
        -n aws-privateca-issuer \
        -f values.yaml
      ```

------

Wait for the issuer to be ready. Use the following command: 

```
kubectl wait --for=condition=ready pods --all -n aws-privateca-issuer --timeout=120s
```

And then verify the installation to make sure that all pods have reached the `READY` state:

```
kubectl -n aws-privateca-issuer get all
```

To configure the `aws-private-ca-cluster-issuer`, create a YAML file named `cluster-issuer.yaml`containing the configuration of the issuer:

```
cat > cluster-issuer.yaml <<EOF
apiVersion: awspca.cert-manager.io/v1beta1
kind: AWSPCAClusterIssuer
metadata:
  name: aws-privateca-cluster-issuer
spec:
  arn: "$CA_ARN"
  region: "$REGION"
EOF
```

Next, apply the cluster configuration:

```
kubectl apply -f cluster-issuer.yaml
```

Check the status of the issuer:

```
kubectl describe awspcaclusterissuer aws-privateca-cluster-issuer
```

You should see a response similar to the following:

```
Status:
  Conditions:
    Last Transition Time:  2025-08-13T21:00:00Z
    Message:               AWS PCA Issuer is ready
    Reason:                Verified
    Status:                True
    Type:                  Ready
```

## Manage the AWS Private CA client certificate with cert-manager
<a name="kubernetes-manage-pca-certificate"></a>

If you are not using an Amazon EKS cluster, after you manually bootstrap a trusted certificate in `aws-privateca-issuer` you can transition to a client authentication certificate managed by `cert-manager`. This allows `cert-manager` to automatically renew the client authentication certificate.

1. Create a file called `pca-auth-cert.yaml`:

   ```
   cat > pca-auth-cert.yaml <<EOF
   apiVersion: cert-manager.io/v1
   kind: Certificate
   metadata:
     name: aws-privateca-client-cert
     namespace: aws-privateca-issuer
   spec:
     secretName: aws-privateca-credentials
     duration: 168h
     renewBefore: 48h
     commonName: aws-privateca-issuer
     privateKey:
       algorithm: ECDSA
       size: 256
       rotationPolicy: Always
     usages:
       - client auth
     issuerRef:
       name: aws-privateca-cluster-issuer
       kind: AWSPCAClusterIssuer
       group: awspca.cert-manager.io
   EOF
   ```

1. Create the new managed client authentication certificate:

   ```
   kubectl apply -f pca-auth-cert.yaml
   ```

1. Validate that the certificate was created:

   ```
   kubectl get certificate aws-privateca-client-cert -n aws-privateca-issuer
   ```

   You should see a response similar to the following:

   ```
   NAME                        READY   SECRET                      AGE
   aws-privateca-client-cert   True    aws-privateca-credentials   19m
   ```

## Issue your first TLS certificate
<a name="kubernetes-issue-certificate"></a>

Now that the `cert-manager` and `aws-privateca-issuer` are installed, you can issue a certificate.

Create a YAML file named `certificate.yaml` containing the certificate resource:

```
cat > certificate.yaml <<EOF
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-certificate
  namespace: default
spec:
  secretName: example-certificate-tls
  issuerRef:
    name: aws-privateca-cluster-issuer
    kind: AWSPCAClusterIssuer
    group: awspca.cert-manager.io
  commonName: example.internal
  dnsNames:
    - example.internal
    - api.example.internal
  duration: 2160h # 90 days
  renewBefore: 360h # 15 days
  usages:
    - digital signature
    - key encipherment
    - server auth
EOF
```

Apply the certificate using the following command:

```
kubectl apply -f certificate.yaml
```

You can then check the status of the certificate with the following commands:

```
kubectl get certificate example-certificate
kubectl describe certificate example-certificate
```

You should see a response similar to this:

```
NAME                 READY   SECRET                    AGE
example-certificate  True    example-certificate-tls   30s
```

You can inspect the issued certificate with the following command:

```
kubectl get secret example-certificate-tls -o yaml
```

You can also decode and examine the certificate with the following command:

```
kubectl get secret example-certificate-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
```

# Examples
<a name="PcaKubernetes-examples"></a>

The following examples show how AWS Private CA can be used with Kubernetes clusters.

**Topics**
+ [Sample: Encryption in transit for Kubernetes](https://github.com/aws-samples/sample-encryption-in-transit-for-kubernetes?tab=readme-ov-file)
+ [TLS-enabled Kubernetes clusters with AWS Private CA and Amazon EKS](https://go.aws/3ifFNEJ)
+ [Setting up end-to-end TLS encryption on Amazon EKS with the new AWS Load Balancer Controller](https://aws.amazon.com/blogs/containers/setting-up-end-to-end-tls-encryption-on-amazon-eks-with-the-new-aws-load-balancer-controller/)

# Monitor Kubernetes with AWS Private CA
<a name="PcaKubernetes-monitor"></a>

To monitor the Kubernetes cluster private CA, use the techniques outlined in [Monitor AWS Private CA resources](logging-and-monitoring.md). You can use the following to monitor a private CA:
+ [AWS Private CA CloudWatch metricsAWS Private CA CloudWatch metrics](PcaCloudWatch.md)
+ [Monitor AWS Private CA with CloudWatch Events](CloudWatchEvents.md)
+ [Logging AWS Private Certificate Authority API calls using AWS CloudTrail](logging-using-cloudtrail-pca.md)

# Troubleshoot Kubernetes with AWS Private CA
<a name="PcaKubernetes-troubleshoot"></a>

You can get the logs for `aws-private-ca-issuer` with the following procedure:

1. Get the name of the pod:

   ```
   kubectl get pods -A
   ```

1. To view the issuer logs, use the following command:

   ```
   kubectl logs -n aws-privateca-issuer <pod-name> aws-privateca-issuer
   ```

1. To view the IAM Roles Anywhere logs, use the following command:

   ```
   kubectl logs -n aws-privateca-issuer <pod-name> rolesanywhere-credentials-helper
   ```

To check the status of your AWS Private CA issuer, use one of the following:

**To check that your issuer is ready, use the following command:**

```
kubectl get AWSPCAClusterIssuers -o json | jq '.items[].status
```

The response should be similar to the following:

```
{
  "conditions": [
    {
      "lastTransitionTime": "2024-07-03T13:56:37Z",
      "message": "Issuer verified",
      "reason": "Verified",
      "status": "True",
      "type": "Ready"
    }
  ]
}
```

If the issuer is not in the `Ready` state, the `message` field provides information on why the issuer was unable to reach the `Ready` state.

**To check that your certificate is ready, use the following command:**

```
kubectl get certificates -o json | jq '.items[].status'
```

The response should be similar to the following:

```
{
  "conditions": [
    {
      "lastTransitionTime": "2024-07-03T13:58:13Z",
      "message": "Certificate is up to date and has not expired",
      "observedGeneration": 1,
      "reason": "Ready",
      "status": "True",
      "type": "Ready"
    }
  ],
  "notAfter": "2024-10-01T13:58:12Z",
  "notBefore": "2024-07-03T12:58:12Z",
  "renewalTime": "2024-09-16T13:58:12Z",
  "revision": 1
}
```

If the certificate is not in the `Ready` state, the `message` field provides information on why the certificate was not able to reach the `Ready` state.