Get Started with Data Wrangler

1. Complete each step in Prerequisites.

2. Follow the procedure in Access Data Wrangler to start using Data Wrangler.

1. To use Data Wrangler, you need access to an Amazon Elastic Compute Cloud (Amazon EC2) instance. For more information about the Amazon EC2 instances that you can use, see Instances. To learn how to view your quotas and, if necessary, request a quota increase, see AWS service quotas.

2. Configure the required permissions described in Security and Permissions.

3. If your organization is using a firewall that blocks internet traffic, you must have access to the following URLs:

   • https://ui.prod-1.data-wrangler.sagemaker.aws/

   • https://ui.prod-2.data-wrangler.sagemaker.aws/

   • https://ui.prod-3.data-wrangler.sagemaker.aws/

   • https://ui.prod-4.data-wrangler.sagemaker.aws/

To access Data Wrangler in Studio Classic, do the following.

1. Sign in to Studio Classic. For more information, see Amazon SageMaker domain overview.

2. Choose Studio.

3. Choose Launch app.

4. From the dropdown list, select Studio.

5. Choose the Home icon.

6. Choose Data.

7. Choose Data Wrangler.

8. You can also create a Data Wrangler flow by doing the following.

   1. In the top navigation bar, select File.

   2. Select New.

   3. Select Data Wrangler Flow.

   

9. (Optional) Rename the new directory and the .flow file.

10. When you create a new .flow file in Studio Classic, you might see a carousel that introduces you to Data Wrangler.

    This may take a few minutes.

    This messaging persists as long as the KernelGateway app on your User Details page is Pending. To see the status of this app, in the SageMaker console on the Amazon SageMaker Studio Classic page, select the name of the user you are using to access Studio Classic. On the User Details page, you see a KernelGateway app under Apps. Wait until this app status is Ready to start using Data Wrangler. This can take around 5 minutes the first time you launch Data Wrangler.

    

11. To get started, choose a data source and use it to import a dataset. See Import to learn more.

    When you import a dataset, it appears in your data flow. To learn more, see Create and Use a Data Wrangler Flow.

12. After you import a dataset, Data Wrangler automatically infers the type of data in each column. Choose + next to the Data types step and select Edit data types.

    Important

    After you add transforms to the Data types step, you cannot bulk-update column types using Update types.

13. Use the data flow to add transforms and analyses. To learn more see Transform Data and Analyze and Visualize.

14. To export a complete data flow, choose Export and choose an export option. To learn more, see Export.

15. Finally, choose the Components and registries icon, and select Data Wrangler from the dropdown list to see all the .flow files that you've created. You can use this menu to find and move between data flows.

Import the Titanic dataset to Data Wrangler

1. Choose the Import data button in your Data flow tab or choose the Import tab.

2. Select Amazon S3.

3. Use the Import a dataset from S3 table to find the bucket to which you added the Titanic dataset. Choose the Titanic dataset CSV file to open the Details pane.

4. Under Details, the File type should be CSV. Check First row is header to specify that the first row of the dataset is a header. You can also name the dataset something more friendly, such as Titanic-train.

5. Choose the Import button.

1. Choose the + next to the Data type step in your data flow and select Add analysis.

2. In the Analysis area, select Table summary from the dropdown list.

3. Give the table summary a Name.

4. Select Preview to preview the table that will be created.

5. Choose Save to save it to your data flow. It appears under All Analyses.

To drop the unused columns.

1. Open the Data Wrangler flow.

2. There are two nodes in your Data Wrangler flow. Choose the + to the right of the Data types node.

3. Choose Add transform.

4. In the All steps column, choose Add step.

5. In the Standard transform list, choose Manage Columns. The standard transformations are ready-made, built-in transformations. Make sure that Drop column is selected.

6. Under Columns to drop, check the following column names:

   • cabin

   • ticket

   • name

   • sibsp

   • parch

   • home.dest

   • boat

   • body

7. Choose Preview.

8. Verify that the columns have been dropped, then choose Add.

1. In the All steps column, choose Add step.

2. In the Custom transform list, choose Custom transform.

3. Provide a name for your transformation, and choose Python (Pandas) from the dropdown list.

4. Enter the following Python script in the code box.

   cols = ['name', 'ticket', 'cabin', 'sibsp', 'parch', 'home.dest','boat', 'body']
   df = df.drop(cols, axis=1)

5. Choose Preview to preview the change, and then choose Add to add the transformation.

1. Choose Handle missing.

2. Choose Drop missing for the Transformer.

3. Choose age for the Input column.

4. Choose Preview to see the new data frame, and then choose Add to add the transform to your flow.

5. Repeat the same process for fare.

1. In the Custom Transform section, choose Python (Pandas) from the dropdown list.

2. Enter the following in the code box.

   import pandas as pd
   
   dummies = []
   cols = ['pclass','sex','embarked']
   for col in cols:
       dummies.append(pd.get_dummies(df[col]))
       
   encoded = pd.concat(dummies, axis=1)
   
   df = pd.concat((df, encoded),axis=1)

3. Choose Preview to preview the change. The encoded version of each column is added to the dataset.

4. Choose Add to add the transformation.

1. In the Custom Transform section, select SQL (PySpark SQL) from the dropdown list.

2. Enter the following in the code box.

   SELECT survived, age, fare, 1, 2, 3, female, male, C, Q, S FROM df;

3. Choose Preview to preview the change. The columns listed in your SELECT statement are the only remaining columns.

4. Choose Add to add the transformation.

1. Save your data flow. Select File and then select Save Data Wrangler Flow.

2. Back to the Data Flow tab, select the last step in your data flow (SQL), then choose the + to open the navigation.

3. Choose Export, and Amazon S3 (via Jupyter Notebook). This opens a Jupyter Notebook.

   

4. Choose any Python 3 (Data Science) kernel for the Kernel.

5. When the kernel starts, run the cells in the notebook book until Kick off SageMaker Training Job (Optional).

6. Optionally, you can run the cells in Kick off SageMaker Training Job (Optional) if you want to create a SageMaker training job to train an XGBoost classifier. You can find the cost to run a SageMaker training job in Amazon SageMaker Pricing.

   Alternatively, you can add the code blocks found in Training XGBoost Classifier to the notebook and run them to use the XGBoost open source library to train an XGBoost classifier.

7. Uncomment and run the cell under Cleanup and run it to revert the SageMaker Python SDK to its original version.

1. First, upgrade necessary modules using pip and remove the _SUCCESS file (this last file is problematic when using awswrangler).

   ! pip install --upgrade awscli awswrangler boto sklearn
   ! aws s3 rm {output_path} --recursive  --exclude "*" --include "*_SUCCESS*"

2. Read the data from Amazon S3. You can use awswrangler to recursively read all the CSV files in the S3 prefix. The data is then split into features and labels. The label is the first column of the dataframe.

   import awswrangler as wr
   
   df = wr.s3.read_csv(path=output_path, dataset=True)
   X, y = df.iloc[:,:-1],df.iloc[:,-1]

   • Finally, create DMatrices (the XGBoost primitive structure for data) and do cross-validation using the XGBoost binary classification.

     import xgboost as xgb
     
     dmatrix = xgb.DMatrix(data=X, label=y)
     
     params = {"objective":"binary:logistic",'learning_rate': 0.1, 'max_depth': 5, 'alpha': 10}
     
     xgb.cv(
         dtrain=dmatrix, 
         params=params, 
         nfold=3,
         num_boost_round=50,
         early_stopping_rounds=10,
         metrics="rmse", 
         as_pandas=True, 
         seed=123)