# Use the SageMaker AI DeepAR forecasting algorithm The Amazon SageMaker AI DeepAR forecasting algorithm is a supervised learning algorithm for forecasting scalar (one-dimensional) time series using recurrent neural networks (RNN). Classical forecasting methods, such as autoregressive integrated moving average (ARIMA) or exponential smoothing (ETS), fit a single model to each individual time series. They then use that model to extrapolate the time series into the future. In many applications, however, you have many similar time series across a set of cross-sectional units. For example, you might have time series groupings for demand for different products, server loads, and requests for webpages. For this type of application, you can benefit from training a single model jointly over all of the time series. DeepAR takes this approach. When your dataset contains hundreds of related time series, DeepAR outperforms the standard ARIMA and ETS methods. You can also use the trained model to generate forecasts for new time series that are similar to the ones it has been trained on. The training input for the DeepAR algorithm is one or, preferably, more `target` time series that have been generated by the same process or similar processes. Based on this input dataset, the algorithm trains a model that learns an approximation of this process/processes and uses it to predict how the target time series evolves. Each target time series can be optionally associated with a vector of static (time-independent) categorical features provided by the `cat` field and a vector of dynamic (time-dependent) time series provided by the `dynamic_feat` field. SageMaker AI trains the DeepAR model by randomly sampling training examples from each target time series in the training dataset. Each training example consists of a pair of adjacent context and prediction windows with fixed predefined lengths. To control how far in the past the network can see, use the `context_length` hyperparameter. To control how far in the future predictions can be made, use the `prediction_length` hyperparameter. For more information, see [How the DeepAR Algorithm Works](deepar_how-it-works.md). **Topics** + [Input/Output Interface for the DeepAR Algorithm](#deepar-inputoutput) + [Best Practices for Using the DeepAR Algorithm](#deepar_best_practices) + [EC2 Instance Recommendations for the DeepAR Algorithm](#deepar-instances) + [DeepAR Sample Notebooks](#deepar-sample-notebooks) + [How the DeepAR Algorithm Works](deepar_how-it-works.md) + [DeepAR Hyperparameters](deepar_hyperparameters.md) + [Tune a DeepAR Model](deepar-tuning.md) + [DeepAR Inference Formats](deepar-in-formats.md) ## Input/Output Interface for the DeepAR Algorithm DeepAR supports two data channels. The required `train` channel describes the training dataset. The optional `test` channel describes a dataset that the algorithm uses to evaluate model accuracy after training. You can provide training and test datasets in [JSON Lines](http://jsonlines.org/) format. Files can be in gzip or [Parquet](https://parquet.apache.org/) file format. When specifying the paths for the training and test data, you can specify a single file or a directory that contains multiple files, which can be stored in subdirectories. If you specify a directory, DeepAR uses all files in the directory as inputs for the corresponding channel, except those that start with a period (.) and those named *\$1SUCCESS*. This ensures that you can directly use output folders produced by Spark jobs as input channels for your DeepAR training jobs. By default, the DeepAR model determines the input format from the file extension (`.json`, `.json.gz`, or `.parquet`) in the specified input path. If the path does not end in one of these extensions, you must explicitly specify the format in the SDK for Python. Use the `content_type` parameter of the [s3\$1input](https://sagemaker.readthedocs.io/en/stable/session.html#sagemaker.session.s3_input) class. The records in your input files should contain the following fields: + `start`—A string with the format `YYYY-MM-DD HH:MM:SS`. The start timestamp can't contain time zone information. + `target`—An array of floating-point values or integers that represent the time series. You can encode missing values as `null` literals, or as `"NaN"` strings in JSON, or as `nan` floating-point values in Parquet. + `dynamic_feat` (optional)—An array of arrays of floating-point values or integers that represents the vector of custom feature time series (dynamic features). If you set this field, all records must have the same number of inner arrays (the same number of feature time series). In addition, each inner array must be the same length as the associated `target` value plus `prediction_length`. Missing values are not supported in the features. For example, if target time series represents the demand of different products, an associated `dynamic_feat` might be a boolean time-series which indicates whether a promotion was applied (1) to the particular product or not (0): ``` {"start": ..., "target": [1, 5, 10, 2], "dynamic_feat": [[0, 1, 1, 0]]} ``` + `cat` (optional)—An array of categorical features that can be used to encode the groups that the record belongs to. Categorical features must be encoded as a 0-based sequence of positive integers. For example, the categorical domain \$1R, G, B\$1 can be encoded as \$10, 1, 2\$1. All values from each categorical domain must be represented in the training dataset. That's because the DeepAR algorithm can forecast only for categories that have been observed during training. And, each categorical feature is embedded in a low-dimensional space whose dimensionality is controlled by the `embedding_dimension` hyperparameter. For more information, see [DeepAR Hyperparameters](deepar_hyperparameters.md). If you use a JSON file, it must be in [JSON Lines](http://jsonlines.org/) format. For example: ``` {"start": "2009-11-01 00:00:00", "target": [4.3, "NaN", 5.1, ...], "cat": [0, 1], "dynamic_feat": [[1.1, 1.2, 0.5, ...]]} {"start": "2012-01-30 00:00:00", "target": [1.0, -5.0, ...], "cat": [2, 3], "dynamic_feat": [[1.1, 2.05, ...]]} {"start": "1999-01-30 00:00:00", "target": [2.0, 1.0], "cat": [1, 4], "dynamic_feat": [[1.3, 0.4]]} ``` In this example, each time series has two associated categorical features and one time series features. For Parquet, you use the same three fields as columns. In addition, `"start"` can be the `datetime` type. You can compress Parquet files using gzip (`gzip`) or the Snappy compression library (`snappy`). If the algorithm is trained without `cat` and `dynamic_feat` fields, it learns a "global" model, that is a model that is agnostic to the specific identity of the target time series at inference time and is conditioned only on its shape. If the model is conditioned on the `cat` and `dynamic_feat` feature data provided for each time series, the prediction will probably be influenced by the character of time series with the corresponding `cat` features. For example, if the `target` time series represents the demand of clothing items, you can associate a two-dimensional `cat` vector that encodes the type of item (e.g. 0 = shoes, 1 = dress) in the first component and the color of an item (e.g. 0 = red, 1 = blue) in the second component. A sample input would look as follows: ``` { "start": ..., "target": ..., "cat": [0, 0], ... } # red shoes { "start": ..., "target": ..., "cat": [1, 1], ... } # blue dress ``` At inference time, you can request predictions for targets with `cat` values that are combinations of the `cat` values observed in the training data, for example: ``` { "start": ..., "target": ..., "cat": [0, 1], ... } # blue shoes { "start": ..., "target": ..., "cat": [1, 0], ... } # red dress ``` The following guidelines apply to training data: + The start time and length of the time series can differ. For example, in marketing, products often enter a retail catalog at different dates, so their start dates naturally differ. But all series must have the same frequency, number of categorical features, and number of dynamic features. + Shuffle the training file with respect to the position of the time series in the file. In other words, the time series should occur in random order in the file. + Make sure to set the `start` field correctly. The algorithm uses the `start` timestamp to derive the internal features. + If you use categorical features (`cat`), all time series must have the same number of categorical features. If the dataset contains the `cat` field, the algorithm uses it and extracts the cardinality of the groups from the dataset. By default, `cardinality` is `"auto"`. If the dataset contains the `cat` field, but you don't want to use it, you can disable it by setting `cardinality` to `""`. If a model was trained using a `cat` feature, you must include it for inference. + If your dataset contains the `dynamic_feat` field, the algorithm uses it automatically. All time series have to have the same number of feature time series. The time points in each of the feature time series correspond one-to-one to the time points in the target. In addition, the entry in the `dynamic_feat` field should have the same length as the `target`. If the dataset contains the `dynamic_feat` field, but you don't want to use it, disable it by setting(`num_dynamic_feat` to `""`). If the model was trained with the `dynamic_feat` field, you must provide this field for inference. In addition, each of the features has to have the length of the provided target plus the `prediction_length`. In other words, you must provide the feature value in the future. If you specify optional test channel data, the DeepAR algorithm evaluates the trained model with different accuracy metrics. The algorithm calculates the root mean square error (RMSE) over the test data as follows: ![\[RMSE Formula: Sqrt(1/nT(Sum[i,t](y-hat(i,t)-y(i,t))^2))\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/deepar-1.png) *y**i*,*t* is the true value of time series *i* at the time *t*. *ŷ**i*,*t* is the mean prediction. The sum is over all *n* time series in the test set and over the last Τ time points for each time series, where Τ corresponds to the forecast horizon. You specify the length of the forecast horizon by setting the `prediction_length` hyperparameter. For more information, see [DeepAR Hyperparameters](deepar_hyperparameters.md). In addition, the algorithm evaluates the accuracy of the forecast distribution using weighted quantile loss. For a quantile in the range [0, 1], the weighted quantile loss is defined as follows: ![\[Weighted quantile loss equation.\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/deepar-2.png) *q**i*,*t*(τ) is the τ-quantile of the distribution that the model predicts. To specify which quantiles to calculate loss for, set the `test_quantiles` hyperparameter. In addition to these, the average of the prescribed quantile losses is reported as part of the training logs. For information, see [DeepAR Hyperparameters](deepar_hyperparameters.md). For inference, DeepAR accepts JSON format and the following fields: + `"instances"`, which includes one or more time series in JSON Lines format + A name of `"configuration"`, which includes parameters for generating the forecast For more information, see [DeepAR Inference Formats](deepar-in-formats.md). ## Best Practices for Using the DeepAR Algorithm When preparing your time series data, follow these best practices to achieve the best results: + Except for when splitting your dataset for training and testing, always provide the entire time series for training, testing, and when calling the model for inference. Regardless of how you set `context_length`, don't break up the time series or provide only a part of it. The model uses data points further back than the value set in `context_length` for the lagged values feature. + When tuning a DeepAR model, you can split the dataset to create a training dataset and a test dataset. In a typical evaluation, you would test the model on the same time series used for training, but on the future `prediction_length` time points that follow immediately after the last time point visible during training. You can create training and test datasets that satisfy this criteria by using the entire dataset (the full length of all time series that are available) as a test set and removing the last `prediction_length` points from each time series for training. During training, the model doesn't see the target values for time points on which it is evaluated during testing. During testing, the algorithm withholds the last `prediction_length` points of each time series in the test set and generates a prediction. Then it compares the forecast with the withheld values. You can create more complex evaluations by repeating time series multiple times in the test set, but cutting them at different endpoints. With this approach, accuracy metrics are averaged over multiple forecasts from different time points. For more information, see [Tune a DeepAR Model](deepar-tuning.md). + Avoid using very large values (>400) for the `prediction_length` because it makes the model slow and less accurate. If you want to forecast further into the future, consider aggregating your data at a lower frequency. For example, use `5min` instead of `1min`. + Because lags are used, a model can look further back in the time series than the value specified for `context_length`. Therefore, you don't need to set this parameter to a large value. We recommend starting with the value that you used for `prediction_length`. + We recommend training a DeepAR model on as many time series as are available. Although a DeepAR model trained on a single time series might work well, standard forecasting algorithms, such as ARIMA or ETS, might provide more accurate results. The DeepAR algorithm starts to outperform the standard methods when your dataset contains hundreds of related time series. Currently, DeepAR requires that the total number of observations available across all training time series is at least 300. ## EC2 Instance Recommendations for the DeepAR Algorithm You can train DeepAR on both GPU and CPU instances and in both single and multi-machine settings. We recommend starting with a single CPU instance (for example, ml.c4.2xlarge or ml.c4.4xlarge), and switching to GPU instances and multiple machines only when necessary. Using GPUs and multiple machines improves throughput only for larger models (with many cells per layer and many layers) and for large mini-batch sizes (for example, greater than 512). For inference, DeepAR supports only CPU instances. Specifying large values for `context_length`, `prediction_length`, `num_cells`, `num_layers`, or `mini_batch_size` can create models that are too large for small instances. In this case, use a larger instance type or reduce the values for these parameters. This problem also frequently occurs when running hyperparameter tuning jobs. In that case, use an instance type large enough for the model tuning job and consider limiting the upper values of the critical parameters to avoid job failures. ## DeepAR Sample Notebooks For a sample notebook that shows how to prepare a time series dataset for training the SageMaker AI DeepAR algorithm and how to deploy the trained model for performing inferences, see [DeepAR demo on electricity dataset](https://sagemaker-examples.readthedocs.io/en/latest/introduction_to_amazon_algorithms/deepar_electricity/DeepAR-Electricity.html), which illustrates the advanced features of DeepAR on a real world dataset. For instructions on creating and accessing Jupyter notebook instances that you can use to run the example in SageMaker AI, see [Amazon SageMaker notebook instances](nbi.md). After creating and opening a notebook instance, choose the **SageMaker AI Examples** tab to see a list of all of the SageMaker AI examples. To open a notebook, choose its **Use** tab, and choose **Create copy**. For more information about the Amazon SageMaker AI DeepAR algorithm, see the following blog posts: + [Now available in Amazon SageMaker AI: DeepAR algorithm for more accurate time series forecasting](https://aws.amazon.com/blogs/machine-learning/now-available-in-amazon-sagemaker-deepar-algorithm-for-more-accurate-time-series-forecasting/) + [Deep demand forecasting with Amazon SageMaker AI](https://aws.amazon.com/blogs/machine-learning/deep-demand-forecasting-with-amazon-sagemaker/) # How the DeepAR Algorithm Works During training, DeepAR accepts a training dataset and an optional test dataset. It uses the test dataset to evaluate the trained model. In general, the datasets don't have to contain the same set of time series. You can use a model trained on a given training set to generate forecasts for the future of the time series in the training set, and for other time series. Both the training and the test datasets consist of one or, preferably, more target time series. Each target time series can optionally be associated with a vector of feature time series and a vector of categorical features. For more information, see [Input/Output Interface for the DeepAR Algorithm](deepar.md#deepar-inputoutput). For example, the following is an element of a training set indexed by *i* which consists of a target time series, *Zi,t*, and two associated feature time series, *Xi,1,t* and *Xi,2,t*: ![\[Figure 1: Target time series and associated feature time series\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/ts-full-159.base.png) The target time series might contain missing values, which are represented by line breaks in the time series. DeepAR supports only feature time series that are known in the future. This allows you to run "what if?" scenarios. What happens, for example, if I change the price of a product in some way? Each target time series can also be associated with a number of categorical features. You can use these features to encode which groupings a time series belongs to. Categorical features allow the model to learn typical behavior for groups, which it can use to increase model accuracy. DeepAR implements this by learning an embedding vector for each group that captures the common properties of all time series in the group. ## How Feature Time Series Work in the DeepAR Algorithm To facilitate learning time-dependent patterns, such as spikes during weekends, DeepAR automatically creates feature time series based on the frequency of the target time series. It uses these derived feature time series with the custom feature time series that you provide during training and inference. The following figure shows two of these derived time series features: *ui,1,t* represents the hour of the day and *ui,2,t* the day of the week. ![\[Figure 2: Derived time series\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/ts-full-159.derived.png) The DeepAR algorithm automatically generates these feature time series. The following table lists the derived features for the supported basic time frequencies. | Frequency of the Time Series | Derived Features | | --- | --- | | Minute | `minute-of-hour`, `hour-of-day`, `day-of-week`, `day-of-month`, `day-of-year` | | Hour | `hour-of-day`, `day-of-week`, `day-of-month`, `day-of-year` | | Day | `day-of-week`, `day-of-month`, `day-of-year` | | Week | `day-of-month`, `week-of-year` | | Month | month-of-year | DeepAR trains a model by randomly sampling several training examples from each of the time series in the training dataset. Each training example consists of a pair of adjacent context and prediction windows with fixed predefined lengths. The `context_length` hyperparameter controls how far in the past the network can see, and the `prediction_length` hyperparameter controls how far in the future predictions can be made. During training, the algorithm ignores training set elements containing time series that are shorter than a specified prediction length. The following figure represents five samples with context lengths of 12 hours and prediction lengths of 6 hours drawn from element *i*. For brevity, we've omitted the feature time series *xi,1,t* and *ui,2,t*. ![\[Figure 3: Sampled time series\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/ts-full-159.sampled.png) To capture seasonality patterns, DeepAR also automatically feeds lagged values from the target time series. In the example with hourly frequency, for each time index, *t = T*, the model exposes the *zi,t* values, which occurred approximately one, two, and three days in the past. ![\[Figure 4: Lagged time series\]](http://docs.aws.amazon.com/sagemaker/latest/dg/images/ts-full-159.lags.png) For inference, the trained model takes as input target time series, which might or might not have been used during training, and forecasts a probability distribution for the next `prediction_length` values. Because DeepAR is trained on the entire dataset, the forecast takes into account patterns learned from similar time series. For information on the mathematics behind DeepAR, see [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://arxiv.org/abs/1704.04110). # DeepAR Hyperparameters The following table lists the hyperparameters that you can set when training with the Amazon SageMaker AI DeepAR forecasting algorithm. | Parameter Name | Description | | --- | --- | | context\$1length | The number of time-points that the model gets to see before making the prediction. The value for this parameter should be about the same as the `prediction_length`. The model also receives lagged inputs from the target, so `context_length` can be much smaller than typical seasonalities. For example, a daily time series can have yearly seasonality. The model automatically includes a lag of one year, so the context length can be shorter than a year. The lag values that the model picks depend on the frequency of the time series. For example, lag values for daily frequency are previous week, 2 weeks, 3 weeks, 4 weeks, and year. **Required** Valid values: Positive integer | | epochs | The maximum number of passes over the training data. The optimal value depends on your data size and learning rate. See also `early_stopping_patience`. Typical values range from 10 to 1000. **Required** Valid values: Positive integer | | prediction\$1length | The number of time-steps that the model is trained to predict, also called the forecast horizon. The trained model always generates forecasts with this length. It can't generate longer forecasts. The `prediction_length` is fixed when a model is trained and it cannot be changed later. **Required** Valid values: Positive integer | | time\$1freq | The granularity of the time series in the dataset. Use `time_freq` to select appropriate date features and lags. The model supports the following basic frequencies. It also supports multiples of these basic frequencies. For example, `5min` specifies a frequency of 5 minutes. [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/deepar_hyperparameters.html) **Required** Valid values: An integer followed by *M*, *W*, *D*, *H*, or *min*. For example, `5min`. | | cardinality | When using the categorical features (`cat`), `cardinality` is an array specifying the number of categories (groups) per categorical feature. Set this to `auto` to infer the cardinality from the data. The `auto` mode also works when no categorical features are used in the dataset. This is the recommended setting for the parameter. Set cardinality to `ignore` to force DeepAR to not use categorical features, even it they are present in the data. To perform additional data validation, it is possible to explicitly set this parameter to the actual value. For example, if two categorical features are provided where the first has 2 and the other has 3 possible values, set this to [2, 3]. For more information on how to use categorical feature, see the data-section on the main documentation page of DeepAR. **Optional** Valid values: `auto`, `ignore`, array of positive integers, empty string, or Default value: `auto` | | dropout\$1rate | The dropout rate to use during training. The model uses zoneout regularization. For each iteration, a random subset of hidden neurons are not updated. Typical values are less than 0.2. **Optional** Valid values: float Default value: 0.1 | | early\$1stopping\$1patience | If this parameter is set, training stops when no progress is made within the specified number of `epochs`. The model that has the lowest loss is returned as the final model. **Optional** Valid values: integer | | embedding\$1dimension | Size of embedding vector learned per categorical feature (same value is used for all categorical features). The DeepAR model can learn group-level time series patterns when a categorical grouping feature is provided. To do this, the model learns an embedding vector of size `embedding_dimension` for each group, capturing the common properties of all time series in the group. A larger `embedding_dimension` allows the model to capture more complex patterns. However, because increasing the `embedding_dimension` increases the number of parameters in the model, more training data is required to accurately learn these parameters. Typical values for this parameter are between 10-100. **Optional** Valid values: positive integer Default value: 10 | | learning\$1rate | The learning rate used in training. Typical values range from 1e-4 to 1e-1. **Optional** Valid values: float Default value: 1e-3 | | likelihood | The model generates a probabilistic forecast, and can provide quantiles of the distribution and return samples. Depending on your data, select an appropriate likelihood (noise model) that is used for uncertainty estimates. The following likelihoods can be selected: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/sagemaker/latest/dg/deepar_hyperparameters.html) **Optional** Valid values: One of *gaussian*, *beta*, *negative-binomial*, *student-T*, or *deterministic-L1*. Default value: `student-T` | | mini\$1batch\$1size | The size of mini-batches used during training. Typical values range from 32 to 512. **Optional** Valid values: positive integer Default value: 128 | | num\$1cells | The number of cells to use in each hidden layer of the RNN. Typical values range from 30 to 100. **Optional** Valid values: positive integer Default value: 40 | | num\$1dynamic\$1feat | The number of `dynamic_feat` provided in the data. Set this to `auto` to infer the number of dynamic features from the data. The `auto` mode also works when no dynamic features are used in the dataset. This is the recommended setting for the parameter. To force DeepAR to not use dynamic features, even it they are present in the data, set `num_dynamic_feat` to `ignore`. To perform additional data validation, it is possible to explicitly set this parameter to the actual integer value. For example, if two dynamic features are provided, set this to 2. **Optional** Valid values: `auto`, `ignore`, positive integer, or empty string Default value: `auto` | | num\$1eval\$1samples | The number of samples that are used per time-series when calculating test accuracy metrics. This parameter does not have any influence on the training or the final model. In particular, the model can be queried with a different number of samples. This parameter only affects the reported accuracy scores on the test channel after training. Smaller values result in faster evaluation, but then the evaluation scores are typically worse and more uncertain. When evaluating with higher quantiles, for example 0.95, it may be important to increase the number of evaluation samples. **Optional** Valid values: integer Default value: 100 | | num\$1layers | The number of hidden layers in the RNN. Typical values range from 1 to 4. **Optional** Valid values: positive integer Default value: 2 | | test\$1quantiles | Quantiles for which to calculate quantile loss on the test channel. **Optional** Valid values: array of floats Default value: [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9] | # Tune a DeepAR Model *Automatic model tuning*, also known as hyperparameter tuning, finds the best version of a model by running many jobs that test a range of hyperparameters on your dataset. You choose the tunable hyperparameters, a range of values for each, and an objective metric. You choose the objective metric from the metrics that the algorithm computes. Automatic model tuning searches the hyperparameters chosen to find the combination of values that result in the model that optimizes the objective metric. For more information about model tuning, see [Automatic model tuning with SageMaker AI](automatic-model-tuning.md). ## Metrics Computed by the DeepAR Algorithm The DeepAR algorithm reports three metrics, which are computed during training. When tuning a model, choose one of these as the objective. For the objective, use either the forecast accuracy on a provided test channel (recommended) or the training loss. For recommendations for the training/test split for the DeepAR algorithm, see [Best Practices for Using the DeepAR Algorithm](deepar.md#deepar_best_practices). | Metric Name | Description | Optimization Direction | | --- | --- | --- | | test:RMSE | The root mean square error between the forecast and the actual target computed on the test set. | Minimize | | test:mean\$1wQuantileLoss | The average overall quantile losses computed on the test set. To control which quantiles are used, set the `test_quantiles` hyperparameter. | Minimize | | train:final\$1loss | The training negative log-likelihood loss averaged over the last training epoch for the model. | Minimize | ## Tunable Hyperparameters for the DeepAR Algorithm Tune a DeepAR model with the following hyperparameters. The hyperparameters that have the greatest impact, listed in order from the most to least impactful, on DeepAR objective metrics are: `epochs`, `context_length`, `mini_batch_size`, `learning_rate`, and `num_cells`. | Parameter Name | Parameter Type | Recommended Ranges | | --- | --- | --- | | epochs | `IntegerParameterRanges` | MinValue: 1, MaxValue: 1000 | | context\$1length | `IntegerParameterRanges` | MinValue: 1, MaxValue: 200 | | mini\$1batch\$1size | `IntegerParameterRanges` | MinValue: 32, MaxValue: 1028 | | learning\$1rate | `ContinuousParameterRange` | MinValue: 1e-5, MaxValue: 1e-1 | | num\$1cells | `IntegerParameterRanges` | MinValue: 30, MaxValue: 200 | | num\$1layers | `IntegerParameterRanges` | MinValue: 1, MaxValue: 8 | | dropout\$1rate | `ContinuousParameterRange` | MinValue: 0.00, MaxValue: 0.2 | | embedding\$1dimension | `IntegerParameterRanges` | MinValue: 1, MaxValue: 50 | # DeepAR Inference Formats The following page describes the request and response formats for inference with the Amazon SageMaker AI DeepAR model. ## DeepAR JSON Request Formats Query a trained model by using the model's endpoint. The endpoint takes the following JSON request format. In the request, the `instances` field corresponds to the time series that should be forecast by the model. If the model was trained with categories, you must provide a `cat` for each instance. If the model was trained without the `cat` field, it should be omitted. If the model was trained with a custom feature time series (`dynamic_feat`), you have to provide the same number of `dynamic_feat`values for each instance. Each of them should have a length given by `length(target) + prediction_length`, where the last `prediction_length` values correspond to the time points in the future that will be predicted. If the model was trained without custom feature time series, the field should not be included in the request. ``` { "instances": [ { "start": "2009-11-01 00:00:00", "target": [4.0, 10.0, "NaN", 100.0, 113.0], "cat": [0, 1], "dynamic_feat": [[1.0, 1.1, 2.1, 0.5, 3.1, 4.1, 1.2, 5.0, ...]] }, { "start": "2012-01-30", "target": [1.0], "cat": [2, 1], "dynamic_feat": [[2.0, 3.1, 4.5, 1.5, 1.8, 3.2, 0.1, 3.0, ...]] }, { "start": "1999-01-30", "target": [2.0, 1.0], "cat": [1, 3], "dynamic_feat": [[1.0, 0.1, -2.5, 0.3, 2.0, -1.2, -0.1, -3.0, ...]] } ], "configuration": { "num_samples": 50, "output_types": ["mean", "quantiles", "samples"], "quantiles": ["0.5", "0.9"] } } ``` The `configuration` field is optional. `configuration.num_samples` sets the number of sample paths that the model generates to estimate the mean and quantiles. `configuration.output_types` describes the information that will be returned in the request. Valid values are `"mean"` `"quantiles"` and `"samples"`. If you specify `"quantiles"`, each of the quantile values in `configuration.quantiles` is returned as a time series. If you specify `"samples"`, the model also returns the raw samples used to calculate the other outputs. ## DeepAR JSON Response Formats The following is the format of a response, where `[...]` are arrays of numbers: ``` { "predictions": [ { "quantiles": { "0.9": [...], "0.5": [...] }, "samples": [...], "mean": [...] }, { "quantiles": { "0.9": [...], "0.5": [...] }, "samples": [...], "mean": [...] }, { "quantiles": { "0.9": [...], "0.5": [...] }, "samples": [...], "mean": [...] } ] } ``` DeepAR has a response timeout of 60 seconds. When passing multiple time series in a single request, the forecasts are generated sequentially. Because the forecast for each time series typically takes about 300 to 1000 milliseconds or longer, depending on the model size, passing too many time series in a single request can cause time outs. It's better to send fewer time series per request and send more requests. Because the DeepAR algorithm uses multiple workers per instance, you can achieve much higher throughput by sending multiple requests in parallel. By default, DeepAR uses one worker per CPU for inference, if there is sufficient memory per CPU. If the model is large and there isn't enough memory to run a model on each CPU, the number of workers is reduced. The number of workers used for inference can be overwritten using the environment variable `MODEL_SERVER_WORKERS` For example, by setting `MODEL_SERVER_WORKERS=1`) when calling the SageMaker AI [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModel.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateModel.html) API. ## Batch Transform with the DeepAR Algorithm DeepAR forecasting supports getting inferences by using batch transform from data using the JSON Lines format. In this format, each record is represented on a single line as a JSON object, and lines are separated by newline characters. The format is identical to the JSON Lines format used for model training. For information, see [Input/Output Interface for the DeepAR Algorithm](deepar.md#deepar-inputoutput). For example: ``` {"start": "2009-11-01 00:00:00", "target": [4.3, "NaN", 5.1, ...], "cat": [0, 1], "dynamic_feat": [[1.1, 1.2, 0.5, ..]]} {"start": "2012-01-30 00:00:00", "target": [1.0, -5.0, ...], "cat": [2, 3], "dynamic_feat": [[1.1, 2.05, ...]]} {"start": "1999-01-30 00:00:00", "target": [2.0, 1.0], "cat": [1, 4], "dynamic_feat": [[1.3, 0.4]]} ``` **Note** When creating the transformation job with [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html), set the `BatchStrategy` value to `SingleRecord` and set the `SplitType` value in the [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformInput.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformInput.html) configuration to `Line`, as the default values currently cause runtime failures. Similar to the hosted endpoint inference request format, the `cat` and the `dynamic_feat` fields for each instance are required if both of the following are true: + The model is trained on a dataset that contained both the `cat` and the `dynamic_feat` fields. + The corresponding `cardinality` and `num_dynamic_feat` values used in the training job are not set to `"".` Unlike hosted endpoint inference, the configuration field is set once for the entire batch inference job using an environment variable named `DEEPAR_INFERENCE_CONFIG`. The value of `DEEPAR_INFERENCE_CONFIG` can be passed when the model is created by calling [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html) API. If `DEEPAR_INFERENCE_CONFIG` is missing in the container environment, the inference container uses the following default: ``` { "num_samples": 100, "output_types": ["mean", "quantiles"], "quantiles": ["0.1", "0.2", "0.3", "0.4", "0.5", "0.6", "0.7", "0.8", "0.9"] } ``` The output is also in JSON Lines format, with one line per prediction, in an order identical to the instance order in the corresponding input file. Predictions are encoded as objects identical to the ones returned by responses in online inference mode. For example: ``` { "quantiles": { "0.1": [...], "0.2": [...] }, "samples": [...], "mean": [...] } ``` Note that in the [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformInput.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TransformInput.html) configuration of the SageMaker AI [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html) request clients must explicitly set the `AssembleWith` value to `Line`, as the default value `None` concatenates all JSON objects on the same line. For example, here is a SageMaker AI [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTransformJob.html) request for a DeepAR job with a custom `DEEPAR_INFERENCE_CONFIG`: ``` { "BatchStrategy": "SingleRecord", "Environment": { "DEEPAR_INFERENCE_CONFIG" : "{ \"num_samples\": 200, \"output_types\": [\"mean\"] }", ... }, "TransformInput": { "SplitType": "Line", ... }, "TransformOutput": { "AssembleWith": "Line", ... }, ... } ```