# Object2Vec Algorithm
The Amazon SageMaker AI Object2Vec algorithm is a general-purpose neural embedding algorithm that is highly customizable. It can learn low-dimensional dense embeddings of high-dimensional objects. The embeddings are learned in a way that preserves the semantics of the relationship between pairs of objects in the original space in the embedding space. You can use the learned embeddings to efficiently compute nearest neighbors of objects and to visualize natural clusters of related objects in low-dimensional space, for example. You can also use the embeddings as features of the corresponding objects in downstream supervised tasks, such as classification or regression.
Object2Vec generalizes the well-known Word2Vec embedding technique for words that is optimized in the SageMaker AI [BlazingText algorithm](blazingtext.md). For a blog post that discusses how to apply Object2Vec to some practical use cases, see [Introduction to Amazon SageMaker AI Object2Vec](https://aws.amazon.com/blogs/machine-learning/introduction-to-amazon-sagemaker-object2vec/).
**Topics**
+ [I/O Interface for the Object2Vec Algorithm](#object2vec-inputoutput)
+ [EC2 Instance Recommendation for the Object2Vec Algorithm](#object2vec--instances)
+ [Object2Vec Sample Notebooks](#object2vec-sample-notebooks)
+ [How Object2Vec Works](object2vec-howitworks.md)
+ [Object2Vec Hyperparameters](object2vec-hyperparameters.md)
+ [Tune an Object2Vec Model](object2vec-tuning.md)
+ [Data Formats for Object2Vec Training](object2vec-training-formats.md)
+ [Data Formats for Object2Vec Inference](object2vec-inference-formats.md)
+ [Encoder Embeddings for Object2Vec](object2vec-encoder-embeddings.md)
## I/O Interface for the Object2Vec Algorithm
You can use Object2Vec on many input data types, including the following examples.
| Input Data Type | Example |
| --- | --- |
| Sentence-sentence pairs | "A soccer game with multiple males playing." and "Some men are playing a sport." |
| Labels-sequence pairs | The genre tags of the movie "Titanic", such as "Romance" and "Drama", and its short description: "James Cameron's Titanic is an epic, action-packed romance set against the ill-fated maiden voyage of the R.M.S. Titanic. She was the most luxurious liner of her era, a ship of dreams, which ultimately carried over 1,500 people to their death in the ice cold waters of the North Atlantic in the early hours of April 15, 1912." |
| Customer-customer pairs | The customer ID of Jane and customer ID of Jackie. |
| Product-product pairs | The product ID of football and product ID of basketball. |
| Item review user-item pairs | A user's ID and the items she has bought, such as apple, pear, and orange. |
To transform the input data into the supported formats, you must preprocess it. Currently, Object2Vec natively supports two types of input:
+ A discrete token, which is represented as a list of a single `integer-id`. For example, `[10]`.
+ A sequences of discrete tokens, which is represented as a list of `integer-ids`. For example, `[0,12,10,13]`.
The object in each pair can be asymmetric. For example, the pairs can be (token, sequence) or (token, token) or (sequence, sequence). For token inputs, the algorithm supports simple embeddings as compatible encoders. For sequences of token vectors, the algorithm supports the following as encoders:
+ Average-pooled embeddings
+ Hierarchical convolutional neural networks (CNNs),
+ Multi-layered bidirectional long short-term memory (BiLSTMs)
The input label for each pair can be one of the following:
+ A categorical label that expresses the relationship between the objects in the pair
+ A score that expresses the strength of the similarity between the two objects
For categorical labels used in classification, the algorithm supports the cross-entropy loss function. For ratings/score-based labels used in regression, the algorithm supports the mean squared error (MSE) loss function. Specify these loss functions with the `output_layer` hyperparameter when you create the model training job.
## EC2 Instance Recommendation for the Object2Vec Algorithm
The type of Amazon Elastic Compute Cloud (Amazon EC2) instance that you use depends on whether you are training or running inference.
When training a model using the Object2Vec algorithm on a CPU, start with an ml.m5.2xlarge instance. For training on a GPU, start with an ml.p2.xlarge instance. If the training takes too long on this instance, you can use a larger instance. Currently, the Object2Vec algorithm can train only on a single machine. However, it does offer support for multiple GPUs. Object2Vec supports P2, P3, G4dn, and G5 GPU instance families for training and inference.
For inference with a trained Object2Vec model that has a deep neural network, we recommend using ml.p3.2xlarge GPU instance. Due to GPU memory scarcity, the `INFERENCE_PREFERRED_MODE` environment variable can be specified to optimize on whether the [GPU optimization: Classification or Regression](object2vec-inference-formats.md#object2vec-inference-gpu-optimize-classification) or [GPU optimization: Encoder Embeddings](object2vec-encoder-embeddings.md#object2vec-inference-gpu-optimize-encoder-embeddings) inference network is loaded into GPU.
## Object2Vec Sample Notebooks
+ [Using Object2Vec to Encode Sentences into Fixed Length Embeddings](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/object2vec_sentence_similarity/object2vec_sentence_similarity.html)
# How Object2Vec Works
When using the Amazon SageMaker AI Object2Vec algorithm, you follow the standard workflow: process the data, train the model, and produce inferences.
**Topics**
+ [Step 1: Process Data](#object2vec-step-1-data-preprocessing)
+ [Step 2: Train a Model](#object2vec-step-2-training-model)
+ [Step 3: Produce Inferences](#object2vec-step-3-inference)
## Step 1: Process Data
During preprocessing, convert the data to the [JSON Lines](http://jsonlines.org/) text file format specified in [Data Formats for Object2Vec Training](object2vec-training-formats.md) . To get the highest accuracy during training, also randomly shuffle the data before feeding it into the model. How you generate random permutations depends on the language. For python, you could use `np.random.shuffle`; for Unix, `shuf`.
## Step 2: Train a Model
The SageMaker AI Object2Vec algorithm has the following main components:
+ **Two input channels** – The input channels take a pair of objects of the same or different types as inputs, and pass them to independent and customizable encoders.
+ **Two encoders** – The two encoders, enc0 and enc1, convert each object into a fixed-length embedding vector. The encoded embeddings of the objects in the pair are then passed into a comparator.
+ **A comparator** – The comparator compares the embeddings in different ways and outputs scores that indicate the strength of the relationship between the paired objects. In the output score for a sentence pair. For example, 1 indicates a strong relationship between a sentence pair, and 0 represents a weak relationship.
During training, the algorithm accepts pairs of objects and their relationship labels or scores as inputs. The objects in each pair can be of different types, as described earlier. If the inputs to both encoders are composed of the same token-level units, you can use a shared token embedding layer by setting the `tied_token_embedding_weight` hyperparameter to `True` when you create the training job. This is possible, for example, when comparing sentences that both have word token-level units. To generate negative samples at a specified rate, set the `negative_sampling_rate` hyperparameter to the desired ratio of negative to positive samples. This hyperparameter expedites learning how to discriminate between the positive samples observed in the training data and the negative samples that are not likely to be observed.
Pairs of objects are passed through independent, customizable encoders that are compatible with the input types of corresponding objects. The encoders convert each object in a pair into a fixed-length embedding vector of equal length. The pair of vectors are passed to a comparator operator, which assembles the vectors into a single vector using the value specified in the he `comparator_list` hyperparameter. The assembled vector then passes through a multilayer perceptron (MLP) layer, which produces an output that the loss function compares with the labels that you provided. This comparison evaluates the strength of the relationship between the objects in the pair as predicted by the model. The following figure shows this workflow.
![\[Architecture of the Object2Vec Algorithm from Data Inputs to Scores\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/object2vec-training-image.png)
## Step 3: Produce Inferences
After the model is trained, you can use the trained encoder to preprocess input objects or to perform two types of inference:
+ To convert singleton input objects into fixed-length embeddings using the corresponding encoder
+ To predict the relationship label or score between a pair of input objects
The inference server automatically figures out which of the types is requested based on the input data. To get the embeddings as output, provide only one input. To predict the relationship label or score, provide both inputs in the pair.
# Object2Vec Hyperparameters
In the `CreateTrainingJob` request, you specify the training algorithm. You can also specify algorithm-specific hyperparameters as string-to-string maps. The following table lists the hyperparameters for the Object2Vec training algorithm.
| Parameter Name | Description |
| --- | --- |
| enc0\$1max\$1seq\$1len | The maximum sequence length for the enc0 encoder. **Required** Valid values: 1 ≤ integer ≤ 5000 |
| enc0\$1vocab\$1size | The vocabulary size of enc0 tokens. **Required** Valid values: 2 ≤ integer ≤ 3000000 |
| bucket\$1width | The allowed difference between data sequence length when bucketing is enabled. To enable bucketing, specify a non-zero value for this parameter. **Optional** Valid values: 0 ≤ integer ≤ 100 Default value: 0 (no bucketing) |
| comparator\$1list | A list used to customize the way in which two embeddings are compared. The Object2Vec comparator operator layer takes the encodings from both encoders as inputs and outputs a single vector. This vector is a concatenation of subvectors. The string values passed to the `comparator_list` and the order in which they are passed determine how these subvectors are assembled. For example, if `comparator_list="hadamard, concat"`, then the comparator operator constructs the vector by concatenating the Hadamard product of two encodings and the concatenation of two encodings. If, on the other hand, `comparator_list="hadamard"`, then the comparator operator constructs the vector as the hadamard product of only two encodings. **Optional** Valid values: A string that contains any combination of the names of the three binary operators: `hadamard`, `concat`, or `abs_diff`. The Object2Vec algorithm currently requires that the two vector encodings have the same dimension. These operators produce the subvectors as follows: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `"hadamard, concat, abs_diff"` |
| dropout | The dropout probability for network layers. *Dropout* is a form of regularization used in neural networks that reduces overfitting by trimming codependent neurons. **Optional** Valid values: 0.0 ≤ float ≤ 1.0 Default value: 0.0 |
| early\$1stopping\$1patience | The number of consecutive epochs without improvement allowed before early stopping is applied. Improvement is defined by with the `early_stopping_tolerance` hyperparameter. **Optional** Valid values: 1 ≤ integer ≤ 5 Default value: 3 |
| early\$1stopping\$1tolerance | The reduction in the loss function that an algorithm must achieve between consecutive epochs to avoid early stopping after the number of consecutive epochs specified in the `early_stopping_patience` hyperparameter concludes. **Optional** Valid values: 0.000001 ≤ float ≤ 0.1 Default value: 0.01 |
| enc\$1dim | The dimension of the output of the embedding layer. **Optional** Valid values: 4 ≤ integer ≤ 10000 Default value: 4096 |
| enc0\$1network | The network model for the enc0 encoder. **Optional** Valid values: `hcnn`, `bilstm`, or `pooled_embedding` [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `hcnn` |
| enc0\$1cnn\$1filter\$1width | The filter width of the convolutional neural network (CNN) enc0 encoder. **Conditional** Valid values: 1 ≤ integer ≤ 9 Default value: 3 |
| enc0\$1freeze\$1pretrained\$1embedding | Whether to freeze enc0 pretrained embedding weights. **Conditional** Valid values: `True` or `False` Default value: `True` |
| enc0\$1layers | The number of layers in the enc0 encoder. **Conditional** Valid values: `auto` or 1 ≤ integer ≤ 4 [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `auto` |
| enc0\$1pretrained\$1embedding\$1file | The filename of the pretrained enc0 token embedding file in the auxiliary data channel. **Conditional** Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\$1.\$1\$1] Default value: "" (empty string) |
| enc0\$1token\$1embedding\$1dim | The output dimension of the enc0 token embedding layer. **Conditional** Valid values: 2 ≤ integer ≤ 1000 Default value: 300 |
| enc0\$1vocab\$1file | The vocabulary file for mapping pretrained enc0 token embedding vectors to numerical vocabulary IDs. **Conditional** Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\$1.\$1\$1] Default value: "" (empty string) |
| enc1\$1network | The network model for the enc1 encoder. If you want the enc1 encoder to use the same network model as enc0, including the hyperparameter values, set the value to `enc0`. Even when the enc0 and enc1 encoder networks have symmetric architectures, you can't shared parameter values for these networks. **Optional** Valid values: `enc0`, `hcnn`, `bilstm`, or `pooled_embedding` [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `enc0` |
| enc1\$1cnn\$1filter\$1width | The filter width of the CNN enc1 encoder. **Conditional** Valid values: 1 ≤ integer ≤ 9 Default value: 3 |
| enc1\$1freeze\$1pretrained\$1embedding | Whether to freeze enc1 pretrained embedding weights. **Conditional** Valid values: `True` or `False` Default value: `True` |
| enc1\$1layers | The number of layers in the enc1 encoder. **Conditional** Valid values: `auto` or 1 ≤ integer ≤ 4 [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `auto` |
| enc1\$1max\$1seq\$1len | The maximum sequence length for the enc1 encoder. **Conditional** Valid values: 1 ≤ integer ≤ 5000 |
| enc1\$1pretrained\$1embedding\$1file | The name of the enc1 pretrained token embedding file in the auxiliary data channel. **Conditional** Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\$1.\$1\$1] Default value: "" (empty string) |
| enc1\$1token\$1embedding\$1dim | The output dimension of the enc1 token embedding layer. **Conditional** Valid values: 2 ≤ integer ≤ 1000 Default value: 300 |
| enc1\$1vocab\$1file | The vocabulary file for mapping pretrained enc1 token embeddings to vocabulary IDs. **Conditional** Valid values: String with alphanumeric characters, underscore, or period. [A-Za-z0-9\$1.\$1\$1] Default value: "" (empty string) |
| enc1\$1vocab\$1size | The vocabulary size of enc0 tokens. **Conditional** Valid values: 2 ≤ integer ≤ 3000000 |
| epochs | The number of epochs to run for training. **Optional** Valid values: 1 ≤ integer ≤ 100 Default value: 30 |
| learning\$1rate | The learning rate for training. **Optional** Valid values: 1.0E-6 ≤ float ≤ 1.0 Default value: 0.0004 |
| mini\$1batch\$1size | The batch size that the dataset is split into for an `optimizer` during training. **Optional** Valid values: 1 ≤ integer ≤ 10000 Default value: 32 |
| mlp\$1activation | The type of activation function for the multilayer perceptron (MLP) layer. **Optional** Valid values: `tanh`, `relu`, or `linear` [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `linear` |
| mlp\$1dim | The dimension of the output from MLP layers. **Optional** Valid values: 2 ≤ integer ≤ 10000 Default value: 512 |
| mlp\$1layers | The number of MLP layers in the network. **Optional** Valid values: 0 ≤ integer ≤ 10 Default value: 2 |
| negative\$1sampling\$1rate | The ratio of negative samples, generated to assist in training the algorithm, to positive samples that are provided by users. Negative samples represent data that is unlikely to occur in reality and are labelled negatively for training. They facilitate training a model to discriminate between the positive samples observed and the negative samples that are not. To specify the ratio of negative samples to positive samples used for training, set the value to a positive integer. For example, if you train the algorithm on input data in which all of the samples are positive and set `negative_sampling_rate` to 2, the Object2Vec algorithm internally generates two negative samples per positive sample. If you don't want to generate or use negative samples during training, set the value to 0. **Optional** Valid values: 0 ≤ integer Default value: 0 (off) |
| num\$1classes | The number of classes for classification training. Amazon SageMaker AI ignores this hyperparameter for regression problems. **Optional** Valid values: 2 ≤ integer ≤ 30 Default value: 2 |
| optimizer | The optimizer type. **Optional** Valid values: `adadelta`, `adagrad`, `adam`, `sgd`, or `rmsprop`. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `adam` |
| output\$1layer | The type of output layer where you specify that the task is regression or classification. **Optional** Valid values: `softmax` or `mean_squared_error` [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) Default value: `softmax` |
| tied\$1token\$1embedding\$1weight | Whether to use a shared embedding layer for both encoders. If the inputs to both encoders use the same token-level units, use a shared token embedding layer. For example, for a collection of documents, if one encoder encodes sentences and another encodes whole documents, you can use a shared token embedding layer. That's because both sentences and documents are composed of word tokens from the same vocabulary. **Optional** Valid values: `True` or `False` Default value: `False` |
| token\$1embedding\$1storage\$1type | The mode of gradient update used during training: when the `dense` mode is used, the optimizer calculates the full gradient matrix for the token embedding layer even if most rows of the gradient are zero-valued. When `sparse` mode is used, the optimizer only stores rows of the gradient that are actually being used in the mini-batch. If you want the algorithm to perform lazy gradient updates, which calculate the gradients only in the non-zero rows and which speed up training, specify `row_sparse`. Setting the value to `row_sparse` constrains the values available for other hyperparameters, as follows: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/object2vec-hyperparameters.html) **Optional** Valid values: `dense` or `row_sparse` Default value: `dense` |
| weight\$1decay | The weight decay parameter used for optimization. **Optional** Valid values: 0 ≤ float ≤ 10000 Default value: 0 (no decay) |
# Tune an Object2Vec Model
*Automatic model tuning*, also known as hyperparameter tuning, finds the best version of a model by running many jobs that test a range of hyperparameters on your dataset. You choose the tunable hyperparameters, a range of values for each, and an objective metric. For the objective metric, you use one of the metrics that the algorithm computes. Automatic model tuning searches the chosen hyperparameters to find the combination of values that result in the model that optimizes the objective metric.
For more information about model tuning, see [Automatic model tuning with SageMaker AI](automatic-model-tuning.md).
## Metrics Computed by the Object2Vec Algorithm
The Object2Vec algorithm has both classification and regression metrics. The `output_layer` type determines which metric you can use for automatic model tuning.
### Regressor Metrics Computed by the Object2Vec Algorithm
The algorithm reports a mean squared error regressor metric, which is computed during testing and validation. When tuning the model for regression tasks, choose this metric as the objective.
| Metric Name | Description | Optimization Direction |
| --- | --- | --- |
| test:mean\$1squared\$1error | The Mean Square Error | Minimize |
| validation:mean\$1squared\$1error | The Mean Square Error | Minimize |
### Classification Metrics Computed by the Object2Vec Algorithm
The Object2Vec algorithm reports accuracy and cross-entropy classification metrics, which are computed during test and validation. When tuning the model for classification tasks, choose one of these as the objective.
| Metric Name | Description | Optimization Direction |
| --- | --- | --- |
| test:accuracy | Accuracy | Maximize |
| test:cross\$1entropy | Cross-entropy | Minimize |
| validation:accuracy | Accuracy | Maximize |
| validation:cross\$1entropy | Cross-entropy | Minimize |
## Tunable Object2Vec Hyperparameters
You can tune the following hyperparameters for the Object2Vec algorithm.
| Hyperparameter Name | Hyperparameter Type | Recommended Ranges and Values |
| --- | --- | --- |
| dropout | ContinuousParameterRange | MinValue: 0.0, MaxValue: 1.0 |
| early\$1stopping\$1patience | IntegerParameterRange | MinValue: 1, MaxValue: 5 |
| early\$1stopping\$1tolerance | ContinuousParameterRange | MinValue: 0.001, MaxValue: 0.1 |
| enc\$1dim | IntegerParameterRange | MinValue: 4, MaxValue: 4096 |
| enc0\$1cnn\$1filter\$1width | IntegerParameterRange | MinValue: 1, MaxValue: 5 |
| enc0\$1layers | IntegerParameterRange | MinValue: 1, MaxValue: 4 |
| enc0\$1token\$1embedding\$1dim | IntegerParameterRange | MinValue: 5, MaxValue: 300 |
| enc1\$1cnn\$1filter\$1width | IntegerParameterRange | MinValue: 1, MaxValue: 5 |
| enc1\$1layers | IntegerParameterRange | MinValue: 1, MaxValue: 4 |
| enc1\$1token\$1embedding\$1dim | IntegerParameterRange | MinValue: 5, MaxValue: 300 |
| epochs | IntegerParameterRange | MinValue: 4, MaxValue: 20 |
| learning\$1rate | ContinuousParameterRange | MinValue: 1e-6, MaxValue: 1.0 |
| mini\$1batch\$1size | IntegerParameterRange | MinValue: 1, MaxValue: 8192 |
| mlp\$1activation | CategoricalParameterRanges | [`tanh`, `relu`, `linear`] |
| mlp\$1dim | IntegerParameterRange | MinValue: 16, MaxValue: 1024 |
| mlp\$1layers | IntegerParameterRange | MinValue: 1, MaxValue: 4 |
| optimizer | CategoricalParameterRanges | [`adagrad`, `adam`, `rmsprop`, `sgd`, `adadelta`] |
| weight\$1decay | ContinuousParameterRange | MinValue: 0.0, MaxValue: 1.0 |
# Data Formats for Object2Vec Training
When training with the Object2Vec algorithm, make sure that the input data in your request is in JSON Lines format, where each line represents a single data point.
## Input: JSON Lines Request Format
Content-type: application/jsonlines
```
{"label": 0, "in0": [6, 17, 606, 19, 53, 67, 52, 12, 5, 10, 15, 10178, 7, 33, 652, 80, 15, 69, 821, 4], "in1": [16, 21, 13, 45, 14, 9, 80, 59, 164, 4]}
{"label": 1, "in0": [22, 1016, 32, 13, 25, 11, 5, 64, 573, 45, 5, 80, 15, 67, 21, 7, 9, 107, 4], "in1": [22, 32, 13, 25, 1016, 573, 3252, 4]}
{"label": 1, "in0": [774, 14, 21, 206], "in1": [21, 366, 125]}
```
The “in0” and “in1” are the inputs for encoder0 and encoder1, respectively. The same format is valid for both classification and regression problems. For regression, the field `"label"` can accept real valued inputs.
# Data Formats for Object2Vec Inference
The following page describes the input request and output response formats for getting scoring inference from the Amazon SageMaker AI Object2Vec model.
## GPU optimization: Classification or Regression
Due to GPU memory scarcity, the `INFERENCE_PREFERRED_MODE` environment variable can be specified to optimize on whether the classification/regression or the [Output: Encoder Embeddings](object2vec-encoder-embeddings.md#object2vec-out-encoder-embeddings-data) inference network is loaded into GPU. If the majority of your inference is for classification or regression, specify `INFERENCE_PREFERRED_MODE=classification`. The following is a Batch Transform example of using 4 instances of p3.2xlarge that optimizes for classification/regression inference:
```
transformer = o2v.transformer(instance_count=4,
instance_type="ml.p2.xlarge",
max_concurrent_transforms=2,
max_payload=1, # 1MB
strategy='MultiRecord',
env={'INFERENCE_PREFERRED_MODE': 'classification'}, # only useful with GPU
output_path=output_s3_path)
```
## Input: Classification or Regression Request Format
Content-type: application/json
```
{
"instances" : [
{"in0": [6, 17, 606, 19, 53, 67, 52, 12, 5, 10, 15, 10178, 7, 33, 652, 80, 15, 69, 821, 4], "in1": [16, 21, 13, 45, 14, 9, 80, 59, 164, 4]},
{"in0": [22, 1016, 32, 13, 25, 11, 5, 64, 573, 45, 5, 80, 15, 67, 21, 7, 9, 107, 4], "in1": [22, 32, 13, 25, 1016, 573, 3252, 4]},
{"in0": [774, 14, 21, 206], "in1": [21, 366, 125]}
]
}
```
Content-type: application/jsonlines
```
{"in0": [6, 17, 606, 19, 53, 67, 52, 12, 5, 10, 15, 10178, 7, 33, 652, 80, 15, 69, 821, 4], "in1": [16, 21, 13, 45, 14, 9, 80, 59, 164, 4]}
{"in0": [22, 1016, 32, 13, 25, 11, 5, 64, 573, 45, 5, 80, 15, 67, 21, 7, 9, 107, 4], "in1": [22, 32, 13, 25, 1016, 573, 3252, 4]}
{"in0": [774, 14, 21, 206], "in1": [21, 366, 125]}
```
For classification problems, the length of the scores vector corresponds to `num_classes`. For regression problems, the length is 1.
## Output: Classification or Regression Response Format
Accept: application/json
```
{
"predictions": [
{
"scores": [
0.6533935070037842,
0.07582679390907288,
0.2707797586917877
]
},
{
"scores": [
0.026291321963071823,
0.6577019095420837,
0.31600672006607056
]
}
]
}
```
Accept: application/jsonlines
```
{"scores":[0.195667684078216,0.395351558923721,0.408980727195739]}
{"scores":[0.251988261938095,0.258233487606048,0.489778339862823]}
{"scores":[0.280087798833847,0.368331134319305,0.351581096649169]}
```
In both the classification and regression formats, the scores apply to individual labels.
# Encoder Embeddings for Object2Vec
The following page lists the input request and output response formats for getting encoder embedding inference from the Amazon SageMaker AI Object2Vec model.
## GPU optimization: Encoder Embeddings
An embedding is a mapping from discrete objects, such as words, to vectors of real numbers.
Due to GPU memory scarcity, the `INFERENCE_PREFERRED_MODE` environment variable can be specified to optimize on whether the [Data Formats for Object2Vec Inference](object2vec-inference-formats.md) or the encoder embedding inference network is loaded into GPU. If the majority of your inference is for encoder embeddings, specify `INFERENCE_PREFERRED_MODE=embedding`. The following is a Batch Transform example of using 4 instances of p3.2xlarge that optimizes for encoder embedding inference:
```
transformer = o2v.transformer(instance_count=4,
instance_type="ml.p2.xlarge",
max_concurrent_transforms=2,
max_payload=1, # 1MB
strategy='MultiRecord',
env={'INFERENCE_PREFERRED_MODE': 'embedding'}, # only useful with GPU
output_path=output_s3_path)
```
## Input: Encoder Embeddings
Content-type: application/json; infer\$1max\$1seqlens=,
Where and are integers in the range [1,5000] and define the maximum sequence lengths for the forward and backward encoder.
```
{
"instances" : [
{"in0": [6, 17, 606, 19, 53, 67, 52, 12, 5, 10, 15, 10178, 7, 33, 652, 80, 15, 69, 821, 4]},
{"in0": [22, 1016, 32, 13, 25, 11, 5, 64, 573, 45, 5, 80, 15, 67, 21, 7, 9, 107, 4]},
{"in0": [774, 14, 21, 206]}
]
}
```
Content-type: application/jsonlines; infer\$1max\$1seqlens=,
Where and are integers in the range [1,5000] and define the maximum sequence lengths for the forward and backward encoder.
```
{"in0": [6, 17, 606, 19, 53, 67, 52, 12, 5, 10, 15, 10178, 7, 33, 652, 80, 15, 69, 821, 4]}
{"in0": [22, 1016, 32, 13, 25, 11, 5, 64, 573, 45, 5, 80, 15, 67, 21, 7, 9, 107, 4]}
{"in0": [774, 14, 21, 206]}
```
In both of these formats, you specify only one input type: `“in0”` or `“in1.”` The inference service then invokes the corresponding encoder and outputs the embeddings for each of the instances.
## Output: Encoder Embeddings
Content-type: application/json
```
{
"predictions": [
{"embeddings":[0.057368703186511,0.030703511089086,0.099890425801277,0.063688032329082,0.026327300816774,0.003637571120634,0.021305780857801,0.004316598642617,0.0,0.003397724591195,0.0,0.000378780066967,0.0,0.0,0.0,0.007419463712722]},
{"embeddings":[0.150190666317939,0.05145975202322,0.098204270005226,0.064249359071254,0.056249320507049,0.01513972133398,0.047553978860378,0.0,0.0,0.011533712036907,0.011472506448626,0.010696629062294,0.0,0.0,0.0,0.008508535102009]}
]
}
```
Content-type: application/jsonlines
```
{"embeddings":[0.057368703186511,0.030703511089086,0.099890425801277,0.063688032329082,0.026327300816774,0.003637571120634,0.021305780857801,0.004316598642617,0.0,0.003397724591195,0.0,0.000378780066967,0.0,0.0,0.0,0.007419463712722]}
{"embeddings":[0.150190666317939,0.05145975202322,0.098204270005226,0.064249359071254,0.056249320507049,0.01513972133398,0.047553978860378,0.0,0.0,0.011533712036907,0.011472506448626,0.010696629062294,0.0,0.0,0.0,0.008508535102009]}
```
The vector length of the embeddings output by the inference service is equal to the value of one of the following hyperparameters that you specify at training time: `enc0_token_embedding_dim`, `enc1_token_embedding_dim`, or `enc_dim`.