Kerberos architecture options with Amazon EMR

Advantages

• Amazon EMR has full ownership of the KDC.

• The KDC on the EMR cluster is independent from centralized KDC implementations such as Microsoft Active Directory or AWS Managed Microsoft AD.

• Performance impact is minimal because the KDC manages authentication only for local nodes within the cluster.

• Optionally, other Kerberized clusters can reference the KDC as an external KDC. For more information, see External KDC—primary node on a different cluster.

Considerations and limitations

• Kerberized clusters can not authenticate to one another, so applications can not interoperate. If cluster applications need to interoperate, you must establish a cross-realm trust between clusters, or set up one cluster as the external KDC for other clusters. If a cross-realm trust is established, the KDCs must have different Kerberos realms.

• You must create Linux users on the EC2 instance of the primary node that correspond to KDC user principals, along with the HDFS directories for each user.

• User principals must use an EC2 private key file and kinit credentials to connect to the cluster using SSH.

Advantages

• The EMR cluster on which the KDC is installed maintains full ownership of the KDC.

• With Active Directory, Amazon EMR automatically creates Linux users that correspond to user principals from the KDC. You still must create HDFS directories for each user. In addition, user principals in the Active Directory domain can access Kerberized clusters using kinit credentials, without the EC2 private key file. This eliminates the need to share the private key file among cluster users.

• Because each cluster KDC manages authentication for the nodes in the cluster, the effects of network latency and processing overhead for a large number of nodes across clusters is minimized.

Considerations and limitations

• If you are establishing a trust with an Active Directory realm, you must provide an Active Directory user name and password with permissions to join principals to the domain when you create the cluster.

• Cross-realm trusts cannot be established between Kerberos realms with the same name.

• Cross-realm trusts must be established explicitly. For example, if Cluster A and Cluster B both establish a cross-realm trust with a KDC, they do not inherently trust one another and their applications cannot authenticate to one another to interoperate.

• KDCs must be maintained independently and coordinated so that credentials of user principals match precisely.

• External KDC—MIT KDC

• External KDC—primary node on a different cluster

• External KDC—cluster KDC on a different cluster with Active Directory cross-realm trust

Advantages

• Managing principals is consolidated in a single KDC.

• Multiple clusters can use the same KDC in the same Kerberos realm. For more information, see Requirements for using multiple clusters with the same KDC.

• The primary node on a Kerberized cluster does not have the performance burden associated with maintaining the KDC.

Considerations and limitations

• You must create Linux users on the EC2 instance of each Kerberized cluster's primary node that correspond to KDC user principals, along with the HDFS directories for each user.

• User principals must use an EC2 private key file and kinit credentials to connect to Kerberized clusters using SSH.

• Each node in Kerberized EMR clusters must have a network route to the KDC.

• Each node in Kerberized clusters places an authentication burden on the external KDC, so the configuration of the KDC affects cluster performance. When you configure the hardware of the KDC server, consider the maximum number of Amazon EMR nodes to be supported simultaneously.

• Cluster performance is dependent on the network latency between nodes in Kerberized clusters and the KDC.

• Troubleshooting can be more difficult because of interdependencies.

Advantages

• Managing principals is consolidated in a single KDC.

• Multiple clusters can use the same KDC in the same Kerberos realm. For more information, see Requirements for using multiple clusters with the same KDC.

Considerations and limitations

• You must create Linux users on the EC2 instance of each Kerberized cluster's primary node that correspond to KDC user principals, along with the HDFS directories for each user.

• User principals must use an EC2 private key file and kinit credentials to connect to Kerberized clusters using SSH.

• Each node in each EMR cluster must have a network route to the KDC.

• Each Amazon EMR node in Kerberized clusters places an authentication burden on the external KDC, so the configuration of the KDC affects cluster performance. When you configure the hardware of the KDC server, consider the maximum number of Amazon EMR nodes to be supported simultaneously.

• Cluster performance is dependent on the network latency between nodes in the clusters and the KDC.

• Troubleshooting can be more difficult because of interdependencies.

Advantages

• Managing principals is consolidated in the Active Directory domain.

• Amazon EMR joins the Active Directory realm, which eliminates the need to create Linux users that correspond Active Directory users. You still must create HDFS directories for each user.

• Multiple clusters can use the same KDC in the same Kerberos realm. For more information, see Requirements for using multiple clusters with the same KDC.

• User principals in the Active Directory domain can access Kerberized clusters using kinit credentials, without the EC2 private key file. This eliminates the need to share the private key file among cluster users.

• Only one Amazon EMR primary node has the burden of maintaining the KDC, and only that cluster must be created with Active Directory credentials for the cross-realm trust between the KDC and Active Directory.

Considerations and limitations

• Each node in each EMR cluster must have a network route to the KDC and the Active Directory domain controller.

• Each Amazon EMR node places an authentication burden on the external KDC, so the configuration of the KDC affects cluster performance. When you configure the hardware of the KDC server, consider the maximum number of Amazon EMR nodes to be supported simultaneously.

• Cluster performance is dependent on the network latency between nodes in the clusters and the KDC server.

• Troubleshooting can be more difficult because of interdependencies.