TransformInput
Describes the input source of a transform job and the way the transform job consumes it.
Contents
- DataSource
-
Describes the location of the channel data, which is, the S3 location of the input data that the model can consume.
Type: TransformDataSource object
Required: Yes
- CompressionType
-
If your transform data is compressed, specify the compression type. Amazon SageMaker automatically decompresses the data for the transform job accordingly. The default value is
None.Type: String
Valid Values:
None | GzipRequired: No
- ContentType
-
The multipurpose internet mail extension (MIME) type of the data. Amazon SageMaker uses the MIME type with each http call to transfer data to the transform job.
Type: String
Length Constraints: Maximum length of 256.
Pattern:
.*Required: No
- SplitType
-
The method to use to split the transform job's data files into smaller batches. Splitting is necessary when the total size of each object is too large to fit in a single request. You can also use data splitting to improve performance by processing multiple concurrent mini-batches. The default value for
SplitTypeisNone, which indicates that input data files are not split, and request payloads contain the entire contents of an input object. Set the value of this parameter toLineto split records on a newline character boundary.SplitTypealso supports a number of record-oriented binary data formats. Currently, the supported record formats are:-
RecordIO
-
TFRecord
When splitting is enabled, the size of a mini-batch depends on the values of the
BatchStrategyandMaxPayloadInMBparameters. When the value ofBatchStrategyisMultiRecord, Amazon SageMaker sends the maximum number of records in each request, up to theMaxPayloadInMBlimit. If the value ofBatchStrategyisSingleRecord, Amazon SageMaker sends individual records in each request.Note
Some data formats represent a record as a binary payload wrapped with extra padding bytes. When splitting is applied to a binary data format, padding is removed if the value of
BatchStrategyis set toSingleRecord. Padding is not removed if the value ofBatchStrategyis set toMultiRecord.For more information about
RecordIO, see Create a Dataset Using RecordIOin the MXNet documentation. For more information about TFRecord, see Consuming TFRecord datain the TensorFlow documentation. Type: String
Valid Values:
None | Line | RecordIO | TFRecordRequired: No
-
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
