SageMaker / Paginator / ListPipelineExecutionSteps

ListPipelineExecutionSteps

class SageMaker.Paginator.ListPipelineExecutionSteps

paginator = client.get_paginator('list_pipeline_execution_steps')

paginate(**kwargs)

Creates an iterator that will paginate through responses from SageMaker.Client.list_pipeline_execution_steps().

See also: AWS API Documentation

Request Syntax

response_iterator = paginator.paginate(
    PipelineExecutionArn='string',
    SortOrder='Ascending'|'Descending',
    PaginationConfig={
        'MaxItems': 123,
        'PageSize': 123,
        'StartingToken': 'string'
    }
)

Parameters:

• PipelineExecutionArn (string) – The Amazon Resource Name (ARN) of the pipeline execution.

• SortOrder (string) – The field by which to sort results. The default is CreatedTime.

• PaginationConfig (dict) –

  A dictionary that provides parameters to control pagination.

  • MaxItems (integer) –

    The total number of items to return. If the total number of items available is more than the value specified in max-items then a NextToken will be provided in the output that you can use to resume pagination.

  • PageSize (integer) –

    The size of each page.

  • StartingToken (string) –

    A token to specify where to start paginating. This is the NextToken from a previous response.

Return type:

dict

Returns:

Response Syntax

{
    'PipelineExecutionSteps': [
        {
            'StepName': 'string',
            'StepDisplayName': 'string',
            'StepDescription': 'string',
            'StartTime': datetime(2015, 1, 1),
            'EndTime': datetime(2015, 1, 1),
            'StepStatus': 'Starting'|'Executing'|'Stopping'|'Stopped'|'Failed'|'Succeeded',
            'CacheHitResult': {
                'SourcePipelineExecutionArn': 'string'
            },
            'FailureReason': 'string',
            'Metadata': {
                'TrainingJob': {
                    'Arn': 'string'
                },
                'ProcessingJob': {
                    'Arn': 'string'
                },
                'TransformJob': {
                    'Arn': 'string'
                },
                'TuningJob': {
                    'Arn': 'string'
                },
                'Model': {
                    'Arn': 'string'
                },
                'RegisterModel': {
                    'Arn': 'string'
                },
                'Condition': {
                    'Outcome': 'True'|'False'
                },
                'Callback': {
                    'CallbackToken': 'string',
                    'SqsQueueUrl': 'string',
                    'OutputParameters': [
                        {
                            'Name': 'string',
                            'Value': 'string'
                        },
                    ]
                },
                'Lambda': {
                    'Arn': 'string',
                    'OutputParameters': [
                        {
                            'Name': 'string',
                            'Value': 'string'
                        },
                    ]
                },
                'EMR': {
                    'ClusterId': 'string',
                    'StepId': 'string',
                    'StepName': 'string',
                    'LogFilePath': 'string'
                },
                'EMRServerless': {
                    'ApplicationId': 'string',
                    'JobRunId': 'string',
                    'JobRunName': 'string',
                    'LogFilePath': 'string'
                },
                'QualityCheck': {
                    'CheckType': 'string',
                    'BaselineUsedForDriftCheckStatistics': 'string',
                    'BaselineUsedForDriftCheckConstraints': 'string',
                    'CalculatedBaselineStatistics': 'string',
                    'CalculatedBaselineConstraints': 'string',
                    'ModelPackageGroupName': 'string',
                    'ViolationReport': 'string',
                    'CheckJobArn': 'string',
                    'SkipCheck': True|False,
                    'RegisterNewBaseline': True|False
                },
                'ClarifyCheck': {
                    'CheckType': 'string',
                    'BaselineUsedForDriftCheckConstraints': 'string',
                    'CalculatedBaselineConstraints': 'string',
                    'ModelPackageGroupName': 'string',
                    'ViolationReport': 'string',
                    'CheckJobArn': 'string',
                    'SkipCheck': True|False,
                    'RegisterNewBaseline': True|False
                },
                'Fail': {
                    'ErrorMessage': 'string'
                },
                'AutoMLJob': {
                    'Arn': 'string'
                },
                'Endpoint': {
                    'Arn': 'string'
                },
                'EndpointConfig': {
                    'Arn': 'string'
                },
                'BedrockCustomModel': {
                    'Arn': 'string'
                },
                'BedrockCustomModelDeployment': {
                    'Arn': 'string'
                },
                'BedrockProvisionedModelThroughput': {
                    'Arn': 'string'
                },
                'BedrockModelImport': {
                    'Arn': 'string'
                },
                'InferenceComponent': {
                    'Arn': 'string'
                },
                'Lineage': {
                    'ActionArns': {
                        'string': 'string'
                    },
                    'ArtifactArns': {
                        'string': 'string'
                    },
                    'ContextArns': {
                        'string': 'string'
                    },
                    'Associations': [
                        {
                            'SourceArn': 'string',
                            'DestinationArn': 'string'
                        },
                    ]
                },
                'Job': {
                    'Arn': 'string'
                },
                'Athena': {
                    'QueryExecutionId': 'string',
                    'S3OutputLocation': 'string'
                },
                'Cleanup': {
                    'CleanedSteps': [
                        'string',
                    ],
                    'FailedSteps': [
                        {
                            'StepName': 'string',
                            'ErrorCode': 'ClientError'|'InternalFailure',
                            'ErrorMessage': 'string'
                        },
                    ],
                    'SkippedSteps': [
                        'string',
                    ]
                },
                'ForEach': {
                    'Status': 'Succeeded'|'Failed'|'Stopped',
                    'InputItemCount': 123,
                    'TotalStepCount': 123,
                    'SucceededStepCount': 123,
                    'FailedStepCount': 123,
                    'StoppedStepCount': 123,
                    'Selector': {
                        'StepName': 'string',
                        'Index': 123
                    }
                }
            },
            'AttemptCount': 123,
            'SelectiveExecutionResult': {
                'SourcePipelineExecutionArn': 'string'
            },
            'IterationContext': {
                'ParentStepName': 'string',
                'Index': 123,
                'Item': 'string'
            }
        },
    ],

}

Response Structure

• (dict) –

  • PipelineExecutionSteps (list) –

    A list of PipeLineExecutionStep objects. Each PipeLineExecutionStep consists of StepName, StartTime, EndTime, StepStatus, and Metadata. Metadata is an object with properties for each job that contains relevant information about the job created by the step.

    • (dict) –

      An execution of a step in a pipeline.

      • StepName (string) –

        The name of the step that is executed.

      • StepDisplayName (string) –

        The display name of the step.

      • StepDescription (string) –

        The description of the step.

      • StartTime (datetime) –

        The time that the step started executing.

      • EndTime (datetime) –

        The time that the step stopped executing.

      • StepStatus (string) –

        The status of the step execution.

      • CacheHitResult (dict) –

        If this pipeline execution step was cached, details on the cache hit.

        • SourcePipelineExecutionArn (string) –

          The Amazon Resource Name (ARN) of the pipeline execution.

      • FailureReason (string) –

        The reason why the step failed execution. This is only returned if the step failed its execution.

      • Metadata (dict) –

        Metadata to run the pipeline step.

        • TrainingJob (dict) –

          The Amazon Resource Name (ARN) of the training job that was run by this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the training job that was run by this step execution.

        • ProcessingJob (dict) –

          The Amazon Resource Name (ARN) of the processing job that was run by this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the processing job.

        • TransformJob (dict) –

          The Amazon Resource Name (ARN) of the transform job that was run by this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the transform job that was run by this step execution.

        • TuningJob (dict) –

          The Amazon Resource Name (ARN) of the tuning job that was run by this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the tuning job that was run by this step execution.

        • Model (dict) –

          The Amazon Resource Name (ARN) of the model that was created by this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the created model.

        • RegisterModel (dict) –

          The Amazon Resource Name (ARN) of the model package that the model was registered to by this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the model package.

        • Condition (dict) –

          The outcome of the condition evaluation that was run by this step execution.

          • Outcome (string) –

            The outcome of the Condition step evaluation.

        • Callback (dict) –

          The URL of the Amazon SQS queue used by this step execution, the pipeline generated token, and a list of output parameters.

          • CallbackToken (string) –

            The pipeline generated token from the Amazon SQS queue.

          • SqsQueueUrl (string) –

            The URL of the Amazon Simple Queue Service (Amazon SQS) queue used by the callback step.

          • OutputParameters (list) –

            A list of the output parameters of the callback step.

            • (dict) –

              An output parameter of a pipeline step.

              • Name (string) –

                The name of the output parameter.

              • Value (string) –

                The value of the output parameter.

        • Lambda (dict) –

          The Amazon Resource Name (ARN) of the Lambda function that was run by this step execution and a list of output parameters.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the Lambda function that was run by this step execution.

          • OutputParameters (list) –

            A list of the output parameters of the Lambda step.

            • (dict) –

              An output parameter of a pipeline step.

              • Name (string) –

                The name of the output parameter.

              • Value (string) –

                The value of the output parameter.

        • EMR (dict) –

          The configurations and outcomes of an Amazon EMR step execution.

          • ClusterId (string) –

            The identifier of the EMR cluster.

          • StepId (string) –

            The identifier of the EMR cluster step.

          • StepName (string) –

            The name of the EMR cluster step.

          • LogFilePath (string) –

            The path to the log file where the cluster step’s failure root cause is recorded.

        • EMRServerless (dict) –

          The configurations and outcomes of an Amazon EMR Serverless step execution.

          • ApplicationId (string) –

            The identifier of the Amazon EMR Serverless application.

          • JobRunId (string) –

            The identifier of the Amazon EMR Serverless job run.

          • JobRunName (string) –

            The name of the Amazon EMR Serverless job run.

          • LogFilePath (string) –

            The path to the log file for the Amazon EMR Serverless job run.

        • QualityCheck (dict) –

          The configurations and outcomes of the check step execution. This includes:

          • The type of the check conducted.

          • The Amazon S3 URIs of baseline constraints and statistics files to be used for the drift check.

          • The Amazon S3 URIs of newly calculated baseline constraints and statistics.

          • The model package group name provided.

          • The Amazon S3 URI of the violation report if violations detected.

          • The Amazon Resource Name (ARN) of check processing job initiated by the step execution.

          • The Boolean flags indicating if the drift check is skipped.

          • If step property BaselineUsedForDriftCheck is set the same as CalculatedBaseline.

          • CheckType (string) –

            The type of the Quality check step.

          • BaselineUsedForDriftCheckStatistics (string) –

            The Amazon S3 URI of the baseline statistics file used for the drift check.

          • BaselineUsedForDriftCheckConstraints (string) –

            The Amazon S3 URI of the baseline constraints file used for the drift check.

          • CalculatedBaselineStatistics (string) –

            The Amazon S3 URI of the newly calculated baseline statistics file.

          • CalculatedBaselineConstraints (string) –

            The Amazon S3 URI of the newly calculated baseline constraints file.

          • ModelPackageGroupName (string) –

            The model package group name.

          • ViolationReport (string) –

            The Amazon S3 URI of violation report if violations are detected.

          • CheckJobArn (string) –

            The Amazon Resource Name (ARN) of the Quality check processing job that was run by this step execution.

          • SkipCheck (boolean) –

            This flag indicates if the drift check against the previous baseline will be skipped or not. If it is set to False, the previous baseline of the configured check type must be available.

          • RegisterNewBaseline (boolean) –

            This flag indicates if a newly calculated baseline can be accessed through step properties BaselineUsedForDriftCheckConstraints and BaselineUsedForDriftCheckStatistics. If it is set to False, the previous baseline of the configured check type must also be available. These can be accessed through the BaselineUsedForDriftCheckConstraints and BaselineUsedForDriftCheckStatistics properties.

        • ClarifyCheck (dict) –

          Container for the metadata for a Clarify check step. The configurations and outcomes of the check step execution. This includes:

          • The type of the check conducted,

          • The Amazon S3 URIs of baseline constraints and statistics files to be used for the drift check.

          • The Amazon S3 URIs of newly calculated baseline constraints and statistics.

          • The model package group name provided.

          • The Amazon S3 URI of the violation report if violations detected.

          • The Amazon Resource Name (ARN) of check processing job initiated by the step execution.

          • The boolean flags indicating if the drift check is skipped.

          • If step property BaselineUsedForDriftCheck is set the same as CalculatedBaseline.

          • CheckType (string) –

            The type of the Clarify Check step

          • BaselineUsedForDriftCheckConstraints (string) –

            The Amazon S3 URI of baseline constraints file to be used for the drift check.

          • CalculatedBaselineConstraints (string) –

            The Amazon S3 URI of the newly calculated baseline constraints file.

          • ModelPackageGroupName (string) –

            The model package group name.

          • ViolationReport (string) –

            The Amazon S3 URI of the violation report if violations are detected.

          • CheckJobArn (string) –

            The Amazon Resource Name (ARN) of the check processing job that was run by this step’s execution.

          • SkipCheck (boolean) –

            This flag indicates if the drift check against the previous baseline will be skipped or not. If it is set to False, the previous baseline of the configured check type must be available.

          • RegisterNewBaseline (boolean) –

            This flag indicates if a newly calculated baseline can be accessed through step properties BaselineUsedForDriftCheckConstraints and BaselineUsedForDriftCheckStatistics. If it is set to False, the previous baseline of the configured check type must also be available. These can be accessed through the BaselineUsedForDriftCheckConstraints property.

        • Fail (dict) –

          The configurations and outcomes of a Fail step execution.

          • ErrorMessage (string) –

            A message that you define and then is processed and rendered by the Fail step when the error occurs.

        • AutoMLJob (dict) –

          The Amazon Resource Name (ARN) of the AutoML job that was run by this step.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the AutoML job.

        • Endpoint (dict) –

          The endpoint that was invoked during this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the endpoint in the step.

        • EndpointConfig (dict) –

          The endpoint configuration used to create an endpoint during this step execution.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the endpoint configuration used in the step.

        • BedrockCustomModel (dict) –

          The metadata of the Amazon Bedrock custom model used in the pipeline execution step.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the Amazon Bedrock custom model.

        • BedrockCustomModelDeployment (dict) –

          The metadata of the Amazon Bedrock custom model deployment used in pipeline execution step.

          • Arn (string) –

            The Amazon Resource Name (ARN) for the Amazon Bedrock custom model deployment.

        • BedrockProvisionedModelThroughput (dict) –

          The metadata of the Amazon Bedrock provisioned model throughput used in the pipeline execution step.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the Amazon Bedrock provisioned model throughput.

        • BedrockModelImport (dict) –

          The metadata of Amazon Bedrock model import used in pipeline execution step.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the Amazon Bedrock model import.

        • InferenceComponent (dict) –

          The metadata of the inference component used in pipeline execution step.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the inference component.

        • Lineage (dict) –

          The metadata of the lineage used in pipeline execution step.

          • ActionArns (dict) –

            The Amazon Resource Name (ARN) of the lineage action.

            • (string) –

              • (string) –

          • ArtifactArns (dict) –

            The Amazon Resource Name (ARN) of the lineage artifact.

            • (string) –

              • (string) –

          • ContextArns (dict) –

            The Amazon Resource Name (ARN) of the lineage context.

            • (string) –

              • (string) –

          • Associations (list) –

            The lineage associations.

            • (dict) –

              The data type used to describe the relationship between different sources.

              • SourceArn (string) –

                The Amazon Resource Name (ARN) of the AssociationInfo source.

              • DestinationArn (string) –

                The Amazon Resource Name (ARN) of the AssociationInfo destination.

        • Job (dict) –

          The metadata for a SageMaker job used in a pipeline execution step.

          • Arn (string) –

            The Amazon Resource Name (ARN) of the SageMaker job that was run by this step execution.

        • Athena (dict) –

          The metadata for the Athena step execution.

          • QueryExecutionId (string) –

            The unique identifier of the Amazon Athena query execution that was run by this step execution.

          • S3OutputLocation (string) –

            The Amazon S3 location of the Amazon Athena query results. This is the effective output location reported by GetQueryExecution for the query. This value can be absent when the query uses Athena managed query results.

        • Cleanup (dict) –

          The metadata for the cleanup step execution.

          • CleanedSteps (list) –

            The names of upstream steps whose Amazon S3 output resources the Cleanup step deleted successfully.

            • (string) –

          • FailedSteps (list) –

            The upstream steps whose Amazon S3 output resources the Cleanup step could not delete.

            • (dict) –

              The Amazon S3 output resources associated with an upstream step that the Cleanup step could not delete.

              • StepName (string) –

                The name of the upstream step whose Amazon S3 output resources the Cleanup step could not delete.

              • ErrorCode (string) –

                The error code that identifies why the Cleanup step could not delete the output resources associated with the upstream step.

              • ErrorMessage (string) –

                A message describing why the Cleanup step could not delete the output resources associated with the upstream step.

          • SkippedSteps (list) –

            The names of upstream steps that the Cleanup step skipped. The Cleanup step skips a step when the output properties needed to locate its resources are unavailable.

            • (string) –

        • ForEach (dict) –

          The metadata for the ForEach step execution.

          • Status (string) –

            The status of the ForEach step execution.

          • InputItemCount (integer) –

            The number of input items that the ForEach step iterates over.

          • TotalStepCount (integer) –

            The total number of steps generated across all iterations of the ForEach step.

          • SucceededStepCount (integer) –

            The number of iteration steps that completed successfully.

          • FailedStepCount (integer) –

            The number of iteration steps that failed.

          • StoppedStepCount (integer) –

            The number of iteration steps that were stopped.

          • Selector (dict) –

            Identifies the body step and iteration selected within the ForEach step.

            • StepName (string) –

              The name of the step associated with the iteration.

            • Index (integer) –

              The index of the iteration within the ForEach step.

      • AttemptCount (integer) –

        The current attempt of the execution step. For more information, see Retry Policy for SageMaker Pipelines steps.

      • SelectiveExecutionResult (dict) –

        The ARN from an execution of the current pipeline from which results are reused for this step.

        • SourcePipelineExecutionArn (string) –

          The ARN from an execution of the current pipeline.

      • IterationContext (dict) –

        The context for the iteration of a ForEach step that generated this step execution.

        • ParentStepName (string) –

          The name of the ForEach step that generated this iteration.

        • Index (integer) –

          The index of the iteration within the ForEach step.

        • Item (string) –

          The input item that is being processed in this iteration.