Object2Vec Hyperparameters
In the CreateTrainingJob request, you specify the training algorithm. You
can also specify algorithm-specific hyperparameters as string-to-string maps. The
following table lists the hyperparameters for the Object2Vec training algorithm.
| Parameter Name | Description |
|---|---|
enc0_max_seq_len |
The maximum sequence length for the enc0 encoder. Required Valid values: 1 ≤ integer ≤ 5000 |
enc0_vocab_size |
The vocabulary size of enc0 tokens. Required Valid values: 2 ≤ integer ≤ 3000000 |
bucket_width |
The allowed difference between data sequence length when bucketing is enabled. To enable bucketing, specify a non-zero value for this parameter. Optional Valid values: 0 ≤ integer ≤ 100 Default value: 0 (no bucketing) |
comparator_list |
A list used to customize the way in which two embeddings are
compared. The Object2Vec comparator operator layer takes the
encodings from both encoders as inputs and outputs a single vector.
This vector is a concatenation of subvectors. The string values
passed to the Optional Valid values: A string that contains any combination of the names
of the three binary operators:
Default value: |
dropout |
The dropout probability for network layers. Dropout is a form of regularization used in neural networks that reduces overfitting by trimming codependent neurons. Optional Valid values: 0.0 ≤ float ≤ 1.0 Default value: 0.0 |
early_stopping_patience |
The number of consecutive epochs without improvement allowed
before early stopping is applied. Improvement is defined by with the
Optional Valid values: 1 ≤ integer ≤ 5 Default value: 3 |
early_stopping_tolerance |
The reduction in the loss function that an algorithm must achieve
between consecutive epochs to avoid early stopping after the number
of consecutive epochs specified in the
Optional Valid values: 0.000001 ≤ float ≤ 0.1 Default value: 0.01 |
enc_dim |
The dimension of the output of the embedding layer. Optional Valid values: 4 ≤ integer ≤ 10000 Default value: 4096 |
enc0_network |
The network model for the enc0 encoder. Optional Valid values:
Default value: |
enc0_cnn_filter_width |
The filter width of the convolutional neural network (CNN) enc0 encoder. Conditional Valid values: 1 ≤ integer ≤ 9 Default value: 3 |
enc0_freeze_pretrained_embedding |
Whether to freeze enc0 pretrained embedding weights. Conditional Valid values: Default value: |
enc0_layers |
The number of layers in the enc0 encoder. Conditional Valid values:
Default value: |
enc0_pretrained_embedding_file |
The filename of the pretrained enc0 token embedding file in the auxiliary data channel. Conditional Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\.\_] Default value: "" (empty string) |
enc0_token_embedding_dim |
The output dimension of the enc0 token embedding layer. Conditional Valid values: 2 ≤ integer ≤ 1000 Default value: 300 |
enc0_vocab_file |
The vocabulary file for mapping pretrained enc0 token embedding vectors to numerical vocabulary IDs. Conditional Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\.\_] Default value: "" (empty string) |
enc1_network |
The network model for the enc1 encoder. If you want the enc1
encoder to use the same network model as enc0, including the
hyperparameter values, set the value to NoteEven when the enc0 and enc1 encoder networks have symmetric architectures, you can't shared parameter values for these networks. Optional Valid values:
Default value: |
enc1_cnn_filter_width |
The filter width of the CNN enc1 encoder. Conditional Valid values: 1 ≤ integer ≤ 9 Default value: 3 |
enc1_freeze_pretrained_embedding |
Whether to freeze enc1 pretrained embedding weights. Conditional Valid values: Default value: |
enc1_layers |
The number of layers in the enc1 encoder. Conditional Valid values:
Default value: |
enc1_max_seq_len |
The maximum sequence length for the enc1 encoder. Conditional Valid values: 1 ≤ integer ≤ 5000 |
enc1_pretrained_embedding_file |
The name of the enc1 pretrained token embedding file in the auxiliary data channel. Conditional Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\.\_] Default value: "" (empty string) |
enc1_token_embedding_dim |
The output dimension of the enc1 token embedding layer. Conditional Valid values: 2 ≤ integer ≤ 1000 Default value: 300 |
enc1_vocab_file |
The vocabulary file for mapping pretrained enc1 token embeddings to vocabulary IDs. Conditional Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\.\_] Default value: "" (empty string) |
enc1_vocab_size |
The vocabulary size of enc0 tokens. Conditional Valid values: 2 ≤ integer ≤ 3000000 |
epochs |
The number of epochs to run for training. Optional Valid values: 1 ≤ integer ≤ 100 Default value: 30 |
learning_rate |
The learning rate for training. Optional Valid values: 1.0E-6 ≤ float ≤ 1.0 Default value: 0.0004 |
mini_batch_size |
The batch size that the dataset is split into for an
Optional Valid values: 1 ≤ integer ≤ 10000 Default value: 32 |
mlp_activation |
The type of activation function for the multilayer perceptron (MLP) layer. Optional Valid values:
Default value: |
mlp_dim |
The dimension of the output from MLP layers. Optional Valid values: 2 ≤ integer ≤ 10000 Default value: 512 |
mlp_layers |
The number of MLP layers in the network. Optional Valid values: 0 ≤ integer ≤ 10 Default value: 2 |
negative_sampling_rate |
The ratio of negative samples, generated to assist in training the
algorithm, to positive samples that are provided by users. Negative
samples represent data that is unlikely to occur in reality and are
labelled negatively for training. They facilitate training a model
to discriminate between the positive samples observed and the
negative samples that are not. To specify the ratio of negative
samples to positive samples used for training, set the value to a
positive integer. For example, if you train the algorithm on input
data in which all of the samples are positive and set
Optional Valid values: 0 ≤ integer Default value: 0 (off) |
num_classes |
The number of classes for classification training. Amazon SageMaker AI ignores this hyperparameter for regression problems. Optional Valid values: 2 ≤ integer ≤ 30 Default value: 2 |
optimizer |
The optimizer type. Optional Valid
values:
Default value: |
output_layer |
The type of output layer where you specify that the task is regression or classification. Optional Valid values:
Default value: |
tied_token_embedding_weight |
Whether to use a shared embedding layer for both encoders. If the inputs to both encoders use the same token-level units, use a shared token embedding layer. For example, for a collection of documents, if one encoder encodes sentences and another encodes whole documents, you can use a shared token embedding layer. That's because both sentences and documents are composed of word tokens from the same vocabulary. Optional Valid values: Default value: |
token_embedding_storage_type |
The mode of gradient update used during training: when the
Optional Valid values: Default value: |
weight_decay |
The weight decay parameter used for optimization. Optional Valid values: 0 ≤ float ≤ 10000 Default value: 0 (no decay) |
