Getting started with EMR Serverless from the console

To run a Spark job

1. Upload the sample script wordcount.py into your new bucket with the following command.

   aws s3 cp s3://us-east-1.elasticmapreduce/emr-containers/samples/wordcount/scripts/wordcount.py s3://amzn-s3-demo-bucket/scripts/

2. Completing Step 1: Create an EMR Serverless application takes you to the Application details page in EMR Studio. There, choose the Submit job option.

3. On the Submit job page, complete the following.

   • In the Name field, enter the name that you want to call your job run.

   • In the Runtime role field, enter the name of the role that you created in Create a job runtime role.

   • In the Script location field, enter s3://amzn-s3-demo-bucket/scripts/wordcount.py as the S3 URI.

   • In the Script arguments field, enter ["s3://amzn-s3-demo-bucket/emr-serverless-spark/output"].

   • In the Spark properties section, choose Edit as text and enter the following configurations.

     --conf spark.executor.cores=1 --conf spark.executor.memory=4g --conf spark.driver.cores=1 --conf spark.driver.memory=4g --conf spark.executor.instances=1

4. To start the job run, choose Submit job .

5. In the Job runs tab, you should see your new job run with a Running status.

To run a Hive job

1. Create a file called hive-query.ql that contains all the queries that you want to run in your Hive job.

   create database if not exists emrserverless;
   use emrserverless;
   create table if not exists test_table(id int);
   drop table if exists Values__Tmp__Table__1;
   insert into test_table values (1),(2),(2),(3),(3),(3);
   select id, count(id) from test_table group by id order by id desc;

2. Upload hive-query.ql to your S3 bucket with the following command.

   aws s3 cp hive-query.ql s3://amzn-s3-demo-bucket/emr-serverless-hive/query/hive-query.ql

3. Completing Step 1: Create an EMR Serverless application takes you to the Application details page in EMR Studio. There, choose the Submit job option.

4. On the Submit job page, complete the following.

   • In the Name field, enter the name that you want to call your job run.

   • In the Runtime role field, enter the name of the role that you created in Create a job runtime role.

   • In the Script location field, enter s3://amzn-s3-demo-bucket/emr-serverless-hive/query/hive-query.ql as the S3 URI.

   • In the Hive properties section, choose Edit as text, and enter the following configurations.

     --hiveconf hive.log.explain.output=false

   • In the Job configuration section, choose Edit as JSON, and enter the following JSON.

     {
        "applicationConfiguration": 
        [{
            "classification": "hive-site",
               "properties": {
                   "hive.exec.scratchdir": "s3://amzn-s3-demo-bucket/emr-serverless-hive/hive/scratch",
                   "hive.metastore.warehouse.dir": "s3://amzn-s3-demo-bucket/emr-serverless-hive/hive/warehouse",
                   "hive.driver.cores": "2",
                   "hive.driver.memory": "4g",
                   "hive.tez.container.size": "4096",
                   "hive.tez.cpu.vcores": "1"
                }
         }]
     }

5. To start the job run, choose Submit job.

6. In the Job runs tab, you should see your new job run with a Running status.

1. Navigate to the Workspaces tab in EMR Studio. If you still need to configure an Amazon S3 storage location and EMR Studio service role, select the Configure studio button in the banner at the top of the screen.

2. To access a notebook, select a Workspace or create a new Workspace. Use Quick launch to open your Workspace in a new tab.

3. Go to the newly opened tab. Select the Compute icon from the left navigation. Select EMR Serverless as the Compute type.

4. Select the interactive-enabled application that you created in the previous section.

5. In the Runtime role field, enter the name of the IAM role that your EMR Serverless application can assume for the job run. To learn more about runtime roles, see Job runtime roles in the Amazon EMR Serverless User Guide.

6. Select Attach. This may take up to a minute. The page will refresh when attached.

7. Pick a kernel and start a notebook. You can also browse example notebooks on EMR Serverless and copy them to your Workspace. To access the example notebooks, navigate to the {...} menu in the left navigation and browse through notebooks that have serverless in the notebook file name.

8. In the notebook, you can access the driver log link and a link to the Apache Spark UI, a real-time interface that provides metrics to monitor your job. For more information, see Monitoring EMR Serverless applications and jobs in the Amazon EMR Serverless User Guide.