SageMakerTrainingSessionRuntime / Client / sample_training_session_with_response_stream

sample_training_session_with_response_stream

SageMakerTrainingSessionRuntime.Client.sample_training_session_with_response_stream(**kwargs)

Streams inference responses from the model associated with the specified training session. Use this operation for real-time token-by-token generation. The service passes the request body to the model without modification.

See also: AWS API Documentation

Request Syntax

response = client.sample_training_session_with_response_stream(
    trainingSessionName='string',
    contentType='string',
    accept='string',
    modelPackageArn='string',
    body=b'bytes'|file
)
Parameters:

  • trainingSessionName (string) –

    [REQUIRED]

    The name of the training session whose model to invoke.

  • contentType (string) – The MIME type of the input data in the request body.

  • accept (string) – The desired MIME type of the inference body in the response.

  • modelPackageArn (string) – The ARN of the model package whose checkpoint to invoke.

  • body (bytes or seekable file-like object) – The raw inference request body.

Return type:

dict

Returns:

Response Syntax

{
    'contentType': 'string',
    'body': StreamingBody()
}

Response Structure

  • (dict) –

    • contentType (string) –

      The MIME type of the streaming inference result.

    • body (StreamingBody) –

      The streaming inference response from the model, delivered as a series of chunks.

Exceptions

  • SageMakerTrainingSessionRuntime.Client.exceptions.ConflictException

  • SageMakerTrainingSessionRuntime.Client.exceptions.ResourceNotFoundException

  • SageMakerTrainingSessionRuntime.Client.exceptions.ThrottlingException

  • SageMakerTrainingSessionRuntime.Client.exceptions.InternalServerException

  • SageMakerTrainingSessionRuntime.Client.exceptions.ValidationException