Class: Aws::SageMaker::Types::Channel
- Inherits:
-
Struct
- Object
- Struct
- Aws::SageMaker::Types::Channel
- Defined in:
- gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb
Overview
A channel is a named input source that training algorithms can consume.
Constant Summary collapse
- SENSITIVE =
[]
Instance Attribute Summary collapse
-
#channel_name ⇒ String
The name of the channel.
-
#compression_type ⇒ String
If training data is compressed, the compression type.
-
#content_type ⇒ String
The MIME type of the data.
-
#data_source ⇒ Types::DataSource
The location of the channel data.
-
#input_mode ⇒ String
(Optional) The input mode to use for the data channel in a training job.
-
#record_wrapper_type ⇒ String
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format.
-
#shuffle_config ⇒ Types::ShuffleConfig
A configuration for a shuffle option for input data in a channel.
Instance Attribute Details
#channel_name ⇒ String
The name of the channel.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |
#compression_type ⇒ String
If training data is compressed, the compression type. The default
value is None
. CompressionType
is used only in Pipe input mode.
In File mode, leave this field unset or set it to None.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |
#content_type ⇒ String
The MIME type of the data.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |
#data_source ⇒ Types::DataSource
The location of the channel data.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |
#input_mode ⇒ String
(Optional) The input mode to use for the data channel in a training
job. If you don't set a value for InputMode
, SageMaker uses the
value set for TrainingInputMode
. Use this parameter to override
the TrainingInputMode
setting in a AlgorithmSpecification
request when you have a channel that needs a different input mode
from the training job's general setting. To download the data from
Amazon Simple Storage Service (Amazon S3) to the provisioned ML
storage volume, and mount the directory to a Docker volume, use
File
input mode. To stream data directly from Amazon S3 to the
container, choose Pipe
input mode.
To use a model for incremental training, choose File
input model.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |
#record_wrapper_type ⇒ String
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO.
In File mode, leave this field unset or set it to None.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |
#shuffle_config ⇒ Types::ShuffleConfig
A configuration for a shuffle option for input data in a channel. If
you use S3Prefix
for S3DataType
, this shuffles the results of
the S3 key prefix matches. If you use ManifestFile
, the order of
the S3 object references in the ManifestFile
is shuffled. If you
use AugmentedManifestFile
, the order of the JSON lines in the
AugmentedManifestFile
is shuffled. The shuffling order is
determined using the Seed
value.
For Pipe input mode, shuffling is done at the start of every epoch.
With large datasets this ensures that the order of the training data
is different for each epoch, it helps reduce bias and possible
overfitting. In a multi-node training job when ShuffleConfig is
combined with S3DataDistributionType
of ShardedByS3Key
, the data
is shuffled across nodes so that the content sent to a particular
node on the first epoch might be sent to a different node on the
second epoch.
3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 |
# File 'gems/aws-sdk-sagemaker/lib/aws-sdk-sagemaker/types.rb', line 3685 class Channel < Struct.new( :channel_name, :data_source, :content_type, :compression_type, :record_wrapper_type, :input_mode, :shuffle_config) SENSITIVE = [] include Aws::Structure end |