Create detailed cost and usage reports for Amazon EMR clusters by using AWS Cost Explorer - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpics

Create detailed cost and usage reports for Amazon EMR clusters by using AWS Cost Explorer

Created by Parijat Bhide (AWS) and Aromal Raj Jayarajan (AWS)

Environment: Production

Technologies: Cost management; Analytics; Big data

AWS services: AWS Billing and Cost Management; Amazon EMR

Summary

This pattern shows how to track the usage costs of Amazon EMR clusters by configuring user-defined cost allocation tags. You can use these tags to create detailed cost and usage reports in AWS Cost Explorer for clusters across multiple dimensions. For example, you can track usage costs at the team, project, or cost center level.

Prerequisites and limitations

Prerequisites

  • An active AWS account

  • One or more EMR clusters that have user-defined tags activated

Architecture

Target technology stack

  • Amazon EMR

  • AWS Cost Explorer

Target architecture

The following diagram shows how you can apply tags to track usage costs for specific Amazon EMR clusters.

Using cost allocation tags for Amazon EMR clusters.

The diagram shows the following workflow:

  1. A data engineer or AWS administrator creates user-defined cost allocation tags for the Amazon EMR clusters.

  2. An AWS administrator activates the tags.

  3. The tags report metadata to AWS Cost Explorer.

Tools

Tools

  • Amazon EMR is a managed cluster platform that simplifies running big data frameworks on AWS to process and analyze large amounts of data.

  • AWS Cost Explorer helps you view and analyze your AWS costs and usage.

Epics

TaskDescriptionSkills required

Create user-defined cost allocation tags for your Amazon EMR clusters.

To add tags to an existing Amazon EMR cluster

Follow the instructions in Adding tags to an existing cluster in the Amazon EMR Management Guide.

To add tags to a new Amazon EMR cluster

Follow the instructions in Add tags to a new cluster in the Amazon EMR Management Guide.

For more information about how to set up an Amazon EMR cluster, see Plan and configure clusters in the Amazon EMR Management Guide.

Data engineer

Activate the user-defined cost allocation tags.

Follow the instructions in Activating user-defined cost allocation tags in the AWS Billing User Guide.

AWS administrator
TaskDescriptionSkills required

Create cost and usage reports for your Amazon EMR clusters by using tag filters in AWS Cost Explorer.

  1. Sign in to the AWS Management Console and open the AWS Cost Management console.

  2. In the left navigation pane, choose Reports.

  3. Choose Create new report.

  4. For Select a report type, choose Cost and usage (recommended). Then, choose Create Report.

  5. For Filters, choose Service. The Service dropdown appears.

  6. Select the check boxes next to EMR (Elastic MapReduce) and EC2-Instances (Elastic Compute Cloud – Compute). Then, choose Apply filters.

  7. For Filters, choose Tag. The Tag dropdown appears.

  8. Choose Team. Then, select the check boxes next to the teams that you’ve assigned tags to. Exclude any teams that you haven’t assigned tags to. Then, choose Apply filters.

  9. At the top of the chart, choose Tag. Then, choose the tags for the Amazon EMR clusters that you want to create a report for.

  10. At the top of the chart, choose the Last 3 Months dropdown and choose the timeframe that you want the report to cover. Then, choose the Monthly dropdown and choose how you want the line items in the report to be aggregated based on timeframe.

  11. Choose Save as. Then, enter a title for your report.

  12. Choose Save Report.

For more information, see Exploring your data using Cost Explorer in the AWS Cost Management User Guide.

General AWS, AWS administrator