Running Spark jobs with Amazon EMR on EKS
A job run is a unit of work, such as a Spark jar, PySpark script, or SparkSQL query, that you submit to Amazon EMR on EKS. This topic provides an overview of managing job runs using the AWS CLI, viewing job runs using the Amazon EMR console, and troubleshooting common job run errors.
Note that you can't run IPv6 Spark jobs on Amazon EMR on EKS
Note
Before you submit a job run with Amazon EMR on EKS, you must complete the steps in Setting up Amazon EMR on EKS.
Topics
- Running Spark jobs with StartJobRun
- Running Spark jobs with the Spark operator
- Running Spark jobs with spark-submit
- Using Apache Livy with Amazon EMR on EKS
- Managing Amazon EMR on EKS job runs
- Using job templates
- Using pod templates
- Using job retry policies
- Using Spark event log rotation
- Using Spark container log rotation
- Using vertical autoscaling with Amazon EMR Spark jobs
