Pipelines overview

The example DAG includes the following steps:

1. AbaloneProcess, an instance of the Processing step, runs a preprocessing script on the data used for training. For example, the script could fill in missing values, normalize numerical data, or split data into the train, validation, and test datasets.

2. AbaloneTrain, an instance of the Training step, configures hyperparameters and trains a model from the preprocessed input data.

3. AbaloneEval, another instance of the Processing step, evaluates the model for accuracy. This step shows an example of a data dependency—this step uses the test dataset output of the AbaloneProcess.

4. AbaloneMSECond is an instance of a Condition step which, in this example, checks to make sure the mean-square-error result of model evaluation is below a certain limit. If the model does not meet the criteria, the pipeline run stops.

5. The pipeline run proceeds with the following steps:

   1. AbaloneRegisterModel, where SageMaker AI calls a RegisterModel step to register the model as a versioned model package group into the Amazon SageMaker Model Registry.

   2. AbaloneCreateModel, where SageMaker AI calls a CreateModel step to create the model in preparation for batch transform. In AbaloneTransform, SageMaker AI calls a Transform step to generate model predictions on a dataset you specify.