# Migrating from CUR to Data Exports CUR 2.0
<a name="dataexports-migrate"></a>

AWS Data Exports allows you to create exports of Cost and Usage Report 2.0 (CUR 2.0). The CUR 2.0 table provides the same information as Cost and Usage Reports (CUR) along with some improvements. Data Exports enables you to create a CUR 2.0 export that is backwards compatible with the data pipelines you’ve been using to process CUR.

CUR 2.0 provides the following improvements over CUR:
+ **Consistent schema:** CUR 2.0 contains a fixed set of columns, whereas the columns included for CUR can vary monthly depending on your usage of AWS services, cost categories, and resource tags.
+ **Nested data:** CUR 2.0 reduces data sparsity by collapsing certain columns from CUR into individual columns with key-value pairs of the collapsed columns. Optionally, you can query the nested keys in Data Exports as separate columns to match the original CUR schema and data.
+ **Additional columns:** CUR 2.0 contains two additional columns: **bill\$1payer\$1account\$1name** and **line\$1item\$1usage\$1account\$1name**.

The following table outlines the differences between CUR 2.0 and legacy CUR in more detail:


****  

|  | CUR 2.0 | Legacy CUR | 
| --- | --- | --- | 
| Data schema |  Fixed schema. For the complete column list, see [Cost and Usage Report (CUR) 2.0](https://docs.aws.amazon.com/cur/latest/userguide/table-dictionary-cur2.html).  |  Dynamic schema based on AWS usage and activity. For the partial column list, see [Data dictionary](https://docs.aws.amazon.com/cur/latest/userguide/data-dictionary.html).  | 
| Exclusive columns |  `bill_payer_account_name` `line_item_usage_account_name`  | None | 
| Export customization |  Enables basic SQL for column selections, row filtering, and column aliasing (renaming). For details about the supported SQL syntax, see [Data query](https://docs.aws.amazon.com/cur/latest/userguide/dataexports-data-query.html).  | Not supported. You must manually set up Athena/QuickSight to create the view you require. | 
| Nested columns with key-value pairs |  `resource_tags` `cost_category` `product` `discount`  |  No nested columns. The four nested columns in CUR 2.0 are split into separate columns in legacy CUR (for example, `resource_tags_user_creator`).  | 
| File delivery destination | S3 bucket | S3 bucket | 
| File output formats | GZIP, Parquet | ZIP, GZIP, Parquet | 
| Integration with other AWS services | Amazon QuickSight | Amazon Athena, Amazon Redshift, Amazon QuickSight | 
| Amazon CloudFormation support |  Yes For details, see [AWS Data Exports resource type reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_BCMDataExports.html) in the *AWS CloudFormation User Guide*.  |  Yes For details, see [AWS Cost and Usage Report resource type reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_CUR.html) in the *AWS CloudFormation User Guide*.  | 
| Tag and cost category data | Tag and cost category names are normalized to remove special characters and spaces. In the event that there are conflicting tags or cost categories after normalization, only one value is kept. For more information, see [Column names](https://docs.aws.amazon.com/cur/latest/userguide/cur-ate-run.html#column-transformations). |  The behavior is different between legacy CUR Parquet and CSV file formats. **Legacy CUR Parquet:** Tag and cost category names are normalized to remove special characters and spaces. In the event that there are conflicting tags or cost categories after normalization, only one value is kept. For more information, see [Column names](https://docs.aws.amazon.com/cur/latest/userguide/cur-ate-run.html#column-transformations). **Legacy CUR CSV:** Tag and cost category names are not changed.  | 

For more detailed information about the schema of CUR 2.0, see the [Data Exports table dictionary](https://docs.aws.amazon.com/cur/latest/userguide/dataexports-table-dictionary.html).

You can migrate to CUR 2.0 in Data Exports in two ways:
+ [Method one: Create an export with an SQL query using the CUR schema](https://docs.aws.amazon.com/cur/latest/userguide/data-exports-migrate-one.html)
+ [Method two: Create an export of CUR 2.0 with its new schema](https://docs.aws.amazon.com/cur/latest/userguide/data-exports-migrate-two.html)

# Method one: Create an export with an SQL query using the CUR schema
<a name="data-exports-migrate-one"></a>

You can create an export with an SQL query. The export schema matches what you receive today in CUR. You do this using the AWS API or SDK.

1. Determine (a) the list of columns and (b) the CUR content settings (**Include resource IDs**, **Split cost allocation data**, and **Time granularity**) needed in order to match your CUR today.

   1. You can determine the list of columns either by viewing the schema of one of your CUR files or going to the manifest file and extracting the list of columns from there.

   1. You can determine the CUR content settings by going to Data Exports in the console and choosing your CUR export to view its details.

1. Write an SQL query that selects the columns you identified from the CUR 2.0 table named `COST_AND_USAGE_REPORT`.

   1. All column names in the CUR 2.0 table are in snake case (for example, `line_item_usage_amount`). For your SQL statement, you might need to convert the previous column names to snake case.

   1. For your SQL statement, you need to convert all `resource_tag` and `cost_category` columns, and certain `product` and `discount` columns, to have the dot operator in order to select the nested columns in CUR 2.0. For example, to select the `product_from_location` column in CUR 2.0, write an SQL statement selecting `product.from_location`.

      Example: `SELECT product.from_location FROM COST_AND_USAGE_REPORT`

      This selects the `from_location` column of the `product` map column.

   1. By default, the column selected with a dot operator is named by the attribute (for example, `from_location`). To match your existing CUR, you’ll need to declare an alias for the column in order to have the same as before.

      Example: `SELECT product.from_location AS product_from_location FROM COST_AND_USAGE_REPORT`

      For more details on nested columns, see the [Data Exports table dictionary](https://docs.aws.amazon.com/cur/latest/userguide/dataexports-table-dictionary.html).

1. Write the CUR content settings, identified in step 1, into the table configuration format for the `CreateExport` API. You need to provide these table configurations with your data query in the next step.

1. In the AWS SDK/CLI for Data Exports, use the `CreateExport` API to input your SQL query and table configurations into the data-query field.

   1. Specify delivery preferences, such as the target Amazon S3 bucket and the overwrite preference. We recommend choosing the same delivery preferences you had before. For more information on the required fields, see [AWS Data Exports](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_Operations_AWS_Billing_and_Cost_Management_Data_Exports.html) in the *AWS Billing and Cost Management API Reference*.

   1. Update the permissions of the target Amazon S3 bucket to allow Data Exports to write to the bucket. For more information, see [Setting up an Amazon S3 bucket for data exports](https://docs.aws.amazon.com/cur/latest/userguide/dataexports-s3-bucket.html).

1. Direct your data ingestion pipeline to read data from the directory in the Amazon S3 bucket where your CUR 2.0 is being delivered.

# Method two: Create an export of CUR 2.0 with its new schema
<a name="data-exports-migrate-two"></a>

You can create an export of CUR 2.0 with its new schema of nested columns and additional columns. However, you’ll need to adjust your current data pipeline to process these new columns. You do this using the console, the AWS API, or SDK.

1. Determine the CUR content settings (**Include resource IDs**, **Split cost allocation data**, and **Time granularity**) needed in order to match your CUR today.
   + You can determine the CUR content settings by going to Data Exports in the console and choosing your CUR export to view its details.

1. Using either the Data Exports console page (**Option A**) or the AWS SDK/CLI (**Option B**), create an export of CUR 2.0 that selects all columns from the “Cost and usage report” table.

1. **(Option A)** To create the export in the console:

   1. In the navigation pane, choose **Data Exports**.

   1. On the **Data Exports** page, choose **Create**.

   1. Choose **Standard data export**.

      For the **Cost and Usage Report (CUR 2.0)** table, all columns are selected by default.

   1. Specify the CUR content settings that you identified in step 1.

   1. Under **Data table delivery options**, choose your options.

   1. Choose **Create**.

1. **(Option B)** To create the export using the AWS API/SDK, first write a query that selects all columns in the `COST_AND_USAGE_REPORT` table.

   1. Use the `GetTable` API to determine the complete list of columns and receive the full schema.

   1. Write the CUR content settings, identified in step 1, into the table configuration format for the `CreateExport` API.

   1. Use the `CreateExport` API to input your SQL query and table configurations into the `data-query` field.

   1. Specify delivery preferences, such as the target Amazon S3 bucket and the overwrite preference. We recommend choosing the same delivery preferences you had before. For more information on the required fields, see [AWS Data Exports](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_Operations_AWS_Billing_and_Cost_Management_Data_Exports.html) in the *AWS Billing and Cost Management API Reference*.

   1. Update the permissions of the target Amazon S3 bucket to allow Data Exports to write to the bucket. For more information, see [Setting up an Amazon S3 bucket for data exports](https://docs.aws.amazon.com/cur/latest/userguide/dataexports-s3-bucket.html).

1. Direct your data ingestion pipeline to read data from the directory in the Amazon S3 bucket where your CUR 2.0 is being delivered.

   You also need to update your data ingestion pipeline and your business intelligence tools to process the following new columns with nested key-values: `product`, `resource_tags`, `cost_category`, and `discounts`.