Capture data from real-time endpoint
Note
To prevent impact to inference requests, Data Capture stops capturing requests at high levels of disk usage. It is recommended you keep your disk utilization below 75% in order to ensure data capture continues capturing requests.
To capture data for your real-time endpoint, you must deploy a model using SageMaker hosting services. This requires that you create a SageMaker model, define an endpoint configuration, and create an HTTPS endpoint.
The steps required to turn on data capture are similar whether you use the
AWS SDK for Python (Boto) or the SageMaker Python SDK. If you use the AWS SDK, define the DataCaptureConfig dictionary, along with required fields, within the
CreateEndpointConfig method to turn on data capture. If you use the
SageMaker Python SDK, import the DataCaptureConfigDataCaptureConfig
parameter in the
sagemaker.model.Model.deploy()
method.
To use the proceeding code snippets, replace the italicized
placeholder text
in the example code with your own
information.
How to enable data capture
Specify a data capture configuration. You can capture the request payload, the response payload, or both with this configuration. The proceeding code snippet demonstrates how to enable data capture using the AWS SDK for Python (Boto) and the SageMaker Python SDK.
Note
You do not need to use Model Monitor to capture request or response payloads.
Deploy your model
Deploy your model and create an HTTPS endpoint with DataCapture
enabled.
View Captured Data
Create a predictor object from the SageMaker Python SDK PredictorPredictor
Class to invoke your endpoint in a future step.
Provide the name of your endpoint (defined earlier as
endpoint_name
), along with serializer and deserializer objects for
the serializer and deserializer, respectively. For information about serializer
types, see the Serializers
from sagemaker.predictor import Predictor from sagemaker.serializers import
<Serializer>
from sagemaker.deserializers import<Deserializers>
predictor = Predictor(endpoint_name=endpoint_name, serializer =<Serializer_Class>
, deserializer =<Deserializer_Class>
) # Example #from sagemaker.predictor import Predictor #from sagemaker.serializers import CSVSerializer #from sagemaker.deserializers import JSONDeserializer #predictor = Predictor(endpoint_name=endpoint_name, # serializer=CSVSerializer(), # deserializer=JSONDeserializer())
In the proceeding code example scenario we invoke the endpoint with sample
validation data that we have stored locally in a CSV file named
validation_with_predictions
. Our sample validation set
contains labels for each input.
The first few lines of the with statement first opens the validation set CSV
file, then splits each row within the file by the comma character
","
, and then stores the two returned objects into a label and
input_cols variables. For each row, the input (input_cols
) is
passed to the predictor variable's (predictor
) objects built-in
method Predictor.predict()
.
Suppose the model returns a probability. Probabilities range between integer values of 0 and 1.0. If the probability returned by the model is greater than 80% (0.8) we assign the prediction an integer value label of 1. Otherwise, we assign the prediction an integer value label of 0.
from time import sleep validate_dataset = "validation_with_predictions.csv" # Cut off threshold of 80% cutoff = 0.8 limit = 200 # Need at least 200 samples to compute standard deviations i = 0 with open(f"test_data/{validate_dataset}", "w") as validation_file: validation_file.write("probability,prediction,label\n") # CSV header with open("test_data/validation.csv", "r") as f: for row in f: (label, input_cols) = row.split(",", 1) probability = float(predictor.predict(input_cols)) prediction = "1" if probability > cutoff else "0" baseline_file.write(f"{probability},{prediction},{label}\n") i += 1 if i > limit: break print(".", end="", flush=True) sleep(0.5) print() print("Done!")
Because you enabled the data capture in the previous steps, the request and
response payload, along with some additional meta data, is saved in the Amazon S3
location that you specified in DataCaptureConfig
. The delivery of
capture data to Amazon S3 can require a couple of minutes.
View captured data by listing the data capture files stored in Amazon S3. The
format of the Amazon S3 path is:
s3:///
.{endpoint-name}/{variant-name}/yyyy/mm/dd/hh/filename
.jsonl
Expect to see different files from different time periods, organized based on the hour when the invocation occurred. Run the following to print out the contents of a single capture file:
print("\n".join(capture_file[-3:-1]))
This will return a SageMaker specific JSON-line formatted file. The following is a
response sample taken from a real-time endpoint that we invoked using
csv/text
data:
{"captureData":{"endpointInput":{"observedContentType":"text/csv","mode":"INPUT", "data":"69,0,153.7,109,194.0,105,256.1,114,14.1,6,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0\n", "encoding":"CSV"},"endpointOutput":{"observedContentType":"text/csv; charset=utf-8","mode":"OUTPUT","data":"0.0254181120544672","encoding":"CSV"}}, "eventMetadata":{"eventId":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee","inferenceTime":"2022-02-14T17:25:49Z"},"eventVersion":"0"} {"captureData":{"endpointInput":{"observedContentType":"text/csv","mode":"INPUT", "data":"94,23,197.1,125,214.5,136,282.2,103,9.5,5,4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,1,0,1\n", "encoding":"CSV"},"endpointOutput":{"observedContentType":"text/csv; charset=utf-8","mode":"OUTPUT","data":"0.07675473392009735","encoding":"CSV"}}, "eventMetadata":{"eventId":"aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee","inferenceTime":"2022-02-14T17:25:49Z"},"eventVersion":"0"}
In the proceeding example, the capture_file
object is a list
type. Index the first element of the list to view a single inference
request.
# The capture_file object is a list. Index the first element to view a single inference request print(json.dumps(json.loads(capture_file[0]), indent=2))
This will return a response similar to the following. The values returned will differ based on your endpoint configuration, SageMaker model, and captured data:
{ "captureData": { "endpointInput": { "observedContentType": "text/csv", # data MIME type "mode": "INPUT", "data": "50,0,188.9,94,203.9,104,151.8,124,11.6,8,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,1,1,0,1,0\n", "encoding": "CSV" }, "endpointOutput": { "observedContentType": "text/csv; charset=character-encoding", "mode": "OUTPUT", "data": "0.023190177977085114", "encoding": "CSV" } }, "eventMetadata": { "eventId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", "inferenceTime": "2022-02-14T17:25:06Z" }, "eventVersion": "0" }