# Getting batch item recommendations with custom resources With custom resources, you can get item recommendations with an asynchronous batch flow. For example, you might get product recommendations for all users on an email list or [item-to-item similarities](native-recipe-similar-items.md) across an inventory. To get batch recommendations for items, you use a batch inference job. A *batch inference job* is a tool that imports your batch input data from an Amazon S3 bucket, uses your custom solution version to generate *item recommendations*, and then exports the item recommendations to an Amazon S3 bucket. Depending on the recipe, your input data is a list of users, or items, or a list of users each with a collection of items. If your solution uses the Similar Items recipe and you have an Items dataset with textual data and item title data, you can generate batch recommendations with themes for each group of items. For more information, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). After you create a custom solution version, how new data influences batch item recommendations depends on its type, the method of import, and the custom recipe you use. For information about how new data influences batch recommendations, see [How new data influences batch recommendations (custom resources)](how-new-data-influences-batch-recommendations.md). **Topics** + [Batch workflow](#batch-worfklow-steps) + [Guidelines and requirements](#batch-permissions-req) + [Batch workflow scoring](#batch-scoring) + [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md) + [Preparing input data for batch recommendations](batch-data-upload.md) + [Creating a batch inference job](creating-batch-inference-job.md) + [Batch inference job output examples](batch-inference-job-output-examples.md) ## Batch workflow The batch workflow is as follows: 1. Prepare and upload your input data in JSON format to an Amazon S3 bucket. The format of your input data depends on the recipe you use. See [Preparing input data for batch recommendations](batch-data-upload.md). 1. Create a separate location for your output data, either a folder or a different Amazon S3 bucket. 1. Create a batch inference job. See [Creating a batch inference job](creating-batch-inference-job.md). 1. When the batch inference is complete, retrieve the item recommendations from your output location in Amazon S3. ## Guidelines and requirements The following are guidelines and requirements for getting batch recommendations: + Your Amazon Personalize IAM service role must have permission to read and add files to your Amazon S3 buckets. For information on granting permissions, see [Service role policy for batch workflows](granting-personalize-s3-access.md#role-policy-for-batch-workflows). For more information on bucket permissions, see [User policy examples](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-policies-s3.html) in the *Amazon Simple Storage Service Developer Guide*. If you use AWS Key Management Service (AWS KMS) for encryption, you must grant Amazon Personalize and your Amazon Personalize IAM service role permission to use your key. For more information, see [Giving Amazon Personalize permission to use your AWS KMS key](granting-personalize-key-access.md). + You must create a custom solution and solution version before you create a batch inference job. However, you don't need to create an Amazon Personalize campaign. If you created a Domain dataset group, you can still create custom resources. + To generate themes with recommendations, you must use the Similar-Items recipe. And you must have an Items dataset with textual data and item title data. For more information about themed recommendations, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). + Your input data must be formatted as described in [Preparing input data for user segments](prepare-input-data-user-segment.md). + You can't get batch recommendations with the Trending-Now or Next-Best-Action recipes. + If you use a filter with placeholder parameters, you must include the values for the parameters in your input data in a `filterValues` object. For more information, see [Providing filter values in your input JSON](filter-batch.md#providing-filter-values). + We recommend that you use a different location for your output data (either a folder or a different Amazon S3 bucket) than your input data. + Batch recommendations might not be exactly the same as real-time recommendations. This is because batch inference jobs take longer to complete and only consider data available 15 minutes before the start of the job. ## Batch workflow scoring Batch recommendations include scores as follows: + With User-Personalization and Personalized-Ranking recipes, Amazon Personalize calculates batch inference job recommendation scores as described in [How recommendation scoring works (custom resources)](recommendations.md#how-recommendation-scoring-works) and [How personalized ranking scoring works](rankings.md#how-ranking-scoring-works). You can view scores in the batch inference job's output JSON file. + With the Similar-Items recipe, if you get themed batch recommendations, Amazon Personalize ranks each set of related items based on how relevant the theme is for each item. Each item includes a score from 0 to 1. The higher the score, the more closely related the item is to the theme. For more information about recommendations with themes, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). # Batch recommendations with themes from Content Generator **Important** When you get batch recommendations with themes, you incur additional costs. For more information, see [Amazon Personalize pricing](https://aws.amazon.com/personalize/pricing/). If you use the [Similar-Items recipe](native-recipe-similar-items.md), Amazon Personalize Content Generator can add descriptive themes to batch recommendations. *Content Generator* is a generative artificial intelligence (generative AI) capability managed by Amazon Personalize. When you get batch recommendations with themes, Amazon Personalize Content Generator adds a descriptive theme for each set of similar items. The theme is based on the item description and item name data in your Items dataset. Amazon Personalize includes the themes in the output of the batch inference job. You can use the themes to make the text in your application or marketing messages more compelling. For example, if you get related items recommendations for a breakfast food item, Amazon Personalize might generate a theme like *Rise and shine* or *Morning essentials*. You might use the theme to replace a generic carousel title, like *Frequently bought together*. Or you might incorporate the theme in a promotional email or marketing campaign for new menu options. AWS doesn't monitor themes from Content Generator. To confirm the theme quality, you can use the scores produced for each recommended item. For more information, see [Ranking and scoring for batch recommendations with themes](#themed-batch-rec-scoring). **Topics** + [Supported regions](#themes-regions) + [Guidelines and requirements](#themed-batch-requirements) + [Ranking and scoring for batch recommendations with themes](#themed-batch-rec-scoring) + [Generating batch recommendations with themes](#getting-themed-batch-rec) ## Supported regions Amazon Personalize Content Generator is only available in the following AWS Regions: + US East (N. Virginia) + US West (Oregon) + Asia Pacific (Tokyo) ## Guidelines and requirements The following are guidelines and requirements for generating recommendations with themes: + Your input file can have up to 100 items. For information about input data for batch recommendations, see [Preparing input data for batch recommendations](batch-data-upload.md). + Your solution must use the [Similar-Items recipe](native-recipe-similar-items.md). + You must have an Items dataset with the following data. This data can help generate more relevant themes. + It must have a textual field, such as a DESCRIPTION field. For information about textual data, see [Unstructured text metadata](items-datasets.md#text-data). + It must have a string column with item name data, such as a TITLE field. If your Items dataset doesn't have this data, you can add it. For information about updating existing data, see [Updating data in datasets after training](updating-datasets.md). ## Ranking and scoring for batch recommendations with themes When you get batch recommendations with themes, Amazon Personalize ranks each set of items based on how relevant the theme is for each item. Each item includes a score in a rough range of -0.1 and 0.6. The higher the score, the more closely related the item is to the theme. You might use the scores to set a threshold to show only items that are strongly related to the theme. For example, Amazon Personalize might return a theme of `For your sweet tooth`, and the related items and their scores might be: hard candy (score 0.19884521), chocolate (score .17664525), apple (score .08994528), popsicle (score .14294521), sweet potato (score .07794527), and carrot (score .04994523). In your application, you might add a rule to include only items with a score of `.10` or greater, eliminating the fruits and vegetables. The following example shows the format of the output of a batch inference job that generates movie recommendations with themes. ``` {"input":{"itemId":"40"},"output":{"recommendedItems":["36","50","44","22","21","29","3","1","2","39"],"theme":"Movies with a strong female lead","itemsThemeRelevanceScores":[0.19994527,0.183059963,0.17478035,0.1618133,0.1574806,0.15468733,0.1499242,0.14353688,0.13531424,0.10291852]}} {"input":{"itemId":"43"},"output":{"recommendedItems":["50","21","36","3","17","2","39","1","10","5"],"theme":"The best movies of 1995","itemsThemeRelevanceScores":[0.184988,0.1795761,0.11143453,0.0989443,0.08258403,0.07952615,0.07115086,0.0621634,-0.138913,-0.188913]}} ... ``` ## Generating batch recommendations with themes To generate batch recommendations with themes, you complete the batch workflow as described in [Batch workflow](getting-batch-recommendations.md#batch-worfklow-steps). You prepare your input data in the same way you would for a `RELATED_ITEMS` recipe. For an example, see [RELATED\$1ITEMS recipes](batch-data-upload.md#batch-input-related-items). When you create the batch inference job, you enable theme generation and specify the item title column of your Items dataset. + For information about using the Amazon Personalize console to create a batch inference job that generates themes, see [Creating a batch inference job](creating-batch-inference-job.md). + For a code sample that shows how to use the SDK for Python (Boto3) to create a batch inference job that generates themes, see [Creating a batch inference job that generates themes](creating-batch-inference-job.md#batch-sdk-themes). # Preparing input data for batch recommendations A batch inference job imports your batch input JSON data from an Amazon S3 bucket, uses your custom solution version to generate recommendations, and then exports the item recommendations to an Amazon S3 bucket. Before you can get batch recommendations, you must prepare and upload your JSON file to an Amazon S3 bucket. We recommend that you create an output folder in your Amazon S3 bucket or use a separate output Amazon S3 bucket. You can then run multiple batch inference jobs using the same input data location. If you use a filter with placeholder parameters, such as `$GENRE`, you must provide the values for the parameters in a `filterValues` object in your input JSON. For more information, see [Providing filter values in your input JSON](filter-batch.md#providing-filter-values). **To prepare and import data** 1. Format your batch input data depending on your recipe. You can't get batch recommendations with the Trending-Now recipe. + For USER\$1PERSONALIZATION recipes and the Popularity-Count recipe, your input data is a JSON file with a list of userIds + For RELATED\$1ITEMS recipes, your input data is a list of itemIds + For PERSONALIZED\$1RANKING recipes, your input data is a list of userIds, each paired with a collection of itemIds Separate each row with a new line. For input data examples, see [Batch inference job input and output JSON examples](#batch-inference-job-json-examples). 1. Upload your input JSON to an input folder in your Amazon S3 bucket. For more information, see [Uploading files and folders by using drag and drop](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/upload-objects.html) in the *Amazon Simple Storage Service User Guide* 1. Create a separate location for your output data, either a folder or a different Amazon S3 bucket. By creating a separate location for the output JSON, you can run multiple batch inference jobs with the same input data location. 1. Create a batch inference job. Amazon Personalize outputs the recommendations from your solution version to your output data location. ## Batch inference job input and output JSON examples How you format your input data the recipe you use. If you use a filter with placeholder parameters, such as `$GENRE`, you must provide the values for the parameters in a `filterValues` object in your input JSON. For more information, see [Providing filter values in your input JSON](filter-batch.md#providing-filter-values). The following sections list correctly formatted JSON input and output examples for batch inference jobs. You can't get batch recommendations with the Trending-Now recipe. **Topics** + [USER\$1PERSONALIZATION recipes](#batch-input-user-personalization) + [POPULAR\$1ITEMS recipes (Popularity-Count only)](#batch-input-popular-items) + [PERSONALIZED\$1RANKING recipes](#batch-input-ranking) + [RELATED\$1ITEMS recipes](#batch-input-related-items) ### USER\$1PERSONALIZATION recipes The following shows correctly formatted JSON input and output examples for the USER\$1PERSONALIZATION recipes. If you use User-Personalization-v2, each recommended item includes a list of reasons for why the item was included in recommendations. This list can be empty. For information about possible reasons, see [Recommendation reasons with User-Personalization-v2](recommendations.md#recommendation-reasons). ------ #### [ Input ] Separate each `userId` with a new line as follows. ``` {"userId": "4638"} {"userId": "663"} {"userId": "3384"} ... ``` ------ #### [ Output ] ``` {"input":{"userId":"4638"},"output":{"recommendedItems":["63992","115149","110102","148626","148888","31685","102445","69526","92535","143355","62374","7451","56171","122882","66097","91542","142488","139385","40583","71530","39292","111360","34048","47099","135137"],"scores":[0.0152238,0.0069081,0.0068222,0.006394,0.0059746,0.0055851,0.0049357,0.0044644,0.0042968,0.004015,0.0038805,0.0037476,0.0036563,0.0036178,0.00341,0.0033467,0.0033258,0.0032454,0.0032076,0.0031996,0.0029558,0.0029021,0.0029007,0.0028837,0.0028316]},"error":null} {"input":{"userId":"663"},"output":{"recommendedItems":["368","377","25","780","1610","648","1270","6","165","1196","1097","300","1183","608","104","474","736","293","141","2987","1265","2716","223","733","2028"],"scores":[0.0406197,0.0372557,0.0254077,0.0151975,0.014991,0.0127175,0.0124547,0.0116712,0.0091098,0.0085492,0.0079035,0.0078995,0.0075598,0.0074876,0.0072006,0.0071775,0.0068923,0.0066552,0.0066232,0.0062504,0.0062386,0.0061121,0.0060942,0.0060781,0.0059263]},"error":null} {"input":{"userId":"3384"},"output":{"recommendedItems":["597","21","223","2144","208","2424","594","595","920","104","520","367","2081","39","1035","2054","160","1370","48","1092","158","2671","500","474","1907"],"scores":[0.0241061,0.0119394,0.0118012,0.010662,0.0086972,0.0079428,0.0073218,0.0071438,0.0069602,0.0056961,0.0055999,0.005577,0.0054387,0.0051787,0.0051412,0.0050493,0.0047126,0.0045393,0.0042159,0.0042098,0.004205,0.0042029,0.0040778,0.0038897,0.0038809]},"error":null} ... ``` ------ ### POPULAR\$1ITEMS recipes (Popularity-Count only) The following shows correctly formatted JSON input and output examples for the Popularity-Count recipe. You can't get batch recommendations with the Trending-Now recipe. ------ #### [ Input ] Separate each `userId` with a new line as follows. ``` {"userId": "12"} {"userId": "105"} {"userId": "41"} ... ``` ------ #### [ Output ] ``` {"input": {"userId": "12"}, "output": {"recommendedItems": ["105", "106", "441"]}} {"input": {"userId": "105"}, "output": {"recommendedItems": ["105", "106", "441"]}} {"input": {"userId": "41"}, "output": {"recommendedItems": ["105", "106", "441"]}} ... ``` ------ ### PERSONALIZED\$1RANKING recipes The following shows correctly formatted JSON input and output examples for PERSONALIZED\$1RANKING recipes. ------ #### [ Input ] Separate each `userId` and list of `itemIds` to be ranked with a new line as follows. ``` {"userId": "891", "itemList": ["27", "886", "101"]} {"userId": "445", "itemList": ["527", "55", "901"]} {"userId": "71", "itemList": ["27", "351", "101"]} ... ``` ------ #### [ Output ] ``` {"input":{"userId":"891","itemList":["27","886","101"]},"output":{"recommendedItems":["27","101","886"],"scores":[0.48421,0.28133,0.23446]}} {"input":{"userId":"445","itemList":["527","55","901"]},"output":{"recommendedItems":["901","527","55"],"scores":[0.46972,0.31011,0.22017]}} {"input":{"userId":"71","itemList":["29","351","199"]},"output":{"recommendedItems":["351","29","199"],"scores":[0.68937,0.24829,0.06232]}} ... ``` ------ ### RELATED\$1ITEMS recipes The following shows correctly formatted JSON input and output examples for RELATED\$1ITEMS recipes. ------ #### [ Input ] Separate each `itemId` with a new line as follows. ``` {"itemId": "105"} {"itemId": "106"} {"itemId": "441"} ... ``` ------ #### [ Output ] ``` {"input": {"itemId": "105"}, "output": {"recommendedItems": ["106", "107", "49"]}} {"input": {"itemId": "106"}, "output": {"recommendedItems": ["105", "107", "49"]}} {"input": {"itemId": "441"}, "output": {"recommendedItems": ["2", "442", "435"]}} ... ``` ------ The following shows correctly formatted JSON input and output examples for the Similar-Items recipe with themes. ------ #### [ Input ] Separate each `itemId` with a new line as follows. ``` {"itemId": "40"} {"itemId": "43"} ... ``` ------ #### [ Output ] ``` {"input":{"itemId":"40"},"output":{"recommendedItems":["36","50","44","22","21","29","3","1","2","39"],"theme":"Movies with a strong female lead","itemsThemeRelevanceScores":[0.19994527,0.183059963,0.17478035,0.1618133,0.1574806,0.15468733,0.1499242,0.14353688,0.13531424,0.10291852]}} {"input":{"itemId":"43"},"output":{"recommendedItems":["50","21","36","3","17","2","39","1","10","5"],"theme":"The best movies of 1995","itemsThemeRelevanceScores":[0.184988,0.1795761,0.11143453,0.0989443,0.08258403,0.07952615,0.07115086,0.0621634,-0.138913,-0.188913]}} ... ``` ------ # Creating a batch inference job Create a batch inference job to get batch item recommendations for users based on input data from Amazon S3. The input data can be a list of users or items (or both) in JSON format. You can create a batch inference job with the Amazon Personalize console, the AWS Command Line Interface (AWS CLI), or AWS SDKs. When you create a batch inference job, you specify the Amazon S3 paths to your input and output locations. Amazon S3 is prefix based. If you provide a prefix for the input data location, Amazon Personalize uses all files matching that prefix as input data. For example, if you provide `s3://amzn-s3-demo-bucket/folderName` and your bucket also has a folder with a path of `s3://amzn-s3-demo-bucket/folderName_test`, Amazon Personalize uses all files in both folders as input data. To use only the files within a specific folder as input data, end the Amazon S3 path with a prefix delimiter, such as `/`: `s3://amzn-s3-demo-bucket/folderName/` For more information about how Amazon S3 organizes objects, see [Organizing, listing, and working with your objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/organizing-objects.html). For more information about the batch workflow in Amazon Personalize, including permissions requirements, recommendation scoring, and preparing and importing input data, see [Getting batch item recommendations with custom resources](getting-batch-recommendations.md). **Topics** + [Creating a batch inference job (console)](#batch-console) + [Creating a batch inference job (AWS CLI)](#batch-cli) + [Creating a batch inference job (AWS SDKs)](#batch-sdk) ## Creating a batch inference job (console) After you have completed [Preparing input data for batch recommendations](batch-data-upload.md), you are ready to create a batch inference job. This procedure assumes that you have already created a solution and a solution version (trained model). **To create a batch inference job (console)** 1. Open the Amazon Personalize console at [https://console.aws.amazon.com/personalize/home](https://console.aws.amazon.com/personalize/home) and sign in to your account. 1. On the **Dataset groups** page, choose your dataset group. 1. From the navigation pane, under **Custom resources**, choose **Batch inference jobs**. 1. Choose **Create batch inference job**. 1. Choose the batch inference job type. + To generate item recommendations without themes, choose **Item recommendations**. + If you use the Similar-Items recipe and want to add descriptive themes to groups of similar items, choose **Themed recommendations with Content Generator**. To generate themes, you must have an Items dataset with item name data and textual data. For more information, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). 1. In **Batch inference job details**, in **Batch inference job name**, specify a name for your batch inference job. 1. For **Solution**, choose the solution and then choose the **Solution version ID** that you want to use to generate the recommendations. 1. For **Number of results**, optionally specify the number of recommendations for each line of input data. The default is 25. 1. If your batch job generates recommendations with themes, in **Themed recommendations details**, choose the column containing names or titles for the items in your Items dataset. This data can help generate more relevant themes. For more information, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). 1. In **Input source**, specify the Amazon S3 path to your input file. Use the following syntax: **s3://amzn-s3-demo-bucket//.json** Your input data must be in the correct format for the recipe your solution uses. For input data examples see [Batch inference job input and output JSON examples](batch-data-upload.md#batch-inference-job-json-examples). 1. For **Decryption key**, if you use your own AWS KMS key for bucket encryption, specify the Amazon Resource Name (ARN) of your key. Amazon Personalize must have permission to use your key. For information about granting permissions, see [Giving Amazon Personalize permission to use your AWS KMS key](granting-personalize-key-access.md). 1. In **Output destination**, specify the path to your output location. We recommend using a different location for your output data (either a folder or a different Amazon S3 bucket). Use the following syntax: **s3://amzn-s3-demo-bucket//** 1. For **Encryption key**, if you use your own AWS KMS key for encryption, specify the ARN of your key. Amazon Personalize must have permission to use your key. For information about granting permissions, see [Giving Amazon Personalize permission to use your AWS KMS key](granting-personalize-key-access.md). 1. For **IAM service role**, choose the IAM service role you created for Amazon Personalize during set up. This role must have read and write access to your input and output Amazon S3 buckets respectively. 1. In **Filters** optionally choose a filter to apply a filter to the batch recommendations. If your filter uses placeholder parameters, make sure the values for the parameters are included in your input JSON. For more information, see [Providing filter values in your input JSON](filter-batch.md#providing-filter-values). 1. For **Tags**, optionally add any tags. For more information about tagging Amazon Personalize resources, see [Tagging Amazon Personalize resources](tagging-resources.md). 1. Choose **Create batch inference job**. Batch inference job creation starts and the **Batch inference jobs** page appears with the **Batch inference job detail** section displayed. When the batch inference job's status changes to **Active**, you can retrieve the job's output from the designated output Amazon S3 bucket. The output file's name will be of the format `input-name.out`. ## Creating a batch inference job (AWS CLI) After you have completed [Preparing input data for batch recommendations](batch-data-upload.md), you are ready to create a batch inference job with the [CreateBatchInferenceJob](API_CreateBatchInferenceJob.md) operation. **Topics** + [Creating a batch inference job](#batch-cli-code-sample) + [Creating a batch inference job that generates themes](#batch-cli-themes) ### Creating a batch inference job You can use the `create-batch-inference-job` command to create a batch inference job. Specify a job name, replace `Solution version ARN` with the Amazon Resource Name (ARN) of your solution version, and replace the `IAM service role ARN` with the ARN of the IAM service role you created for Amazon Personalize during set up. This role must have read and write access to your input and output Amazon S3 buckets respectively. Optionally provide a filter ARN to filter recommendations. If your filter uses placeholder parameters, make sure the values for the parameters are included in your input JSON. For more information, see [Filtering batch recommendations and user segments (custom resources)](filter-batch.md). Replace `S3 input path` and `S3 output path` with the Amazon S3 path to your input file and output locations. We recommend using a different location for your output data (either a folder or a different Amazon S3 bucket). Use the following syntax for input and output locations: **s3://amzn-s3-demo-bucket//.json** and **s3://amzn-s3-demo-bucket//**. The example includes optional User-Personalization recipe specific `itemExplorationConfig` hyperparameters: `explorationWeight` and `explorationItemAgeCutOff`. Optionally include `explorationWeight` and `explorationItemAgeCutOff` values to configure exploration. For more information, see [User-Personalization recipe](native-recipe-new-item-USER_PERSONALIZATION.md). ``` aws personalize create-batch-inference-job \ --job-name Batch job name \ --solution-version-arn Solution version ARN \ --filter-arn Filter ARN \ --job-input s3DataSource={path=s3://S3 input path} \ --job-output s3DataDestination={path=s3://S3 output path} \ --role-arn IAM service role ARN \ --batch-inference-job-config "{\"itemExplorationConfig\":{\"explorationWeight\":\"0.3\",\"explorationItemAgeCutOff\":\"30\"}}" ``` ### Creating a batch inference job that generates themes To generate themes for similar items, you must use the Similar-Items recipe and your Items dataset must have a textual field and a column of item name data. For more information about recommendations with themes, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). The following code creates a batch inference job that generates recommendations with themes. Leave the `batch-inference-job-mode` set to `THEME_GENERATION`. Replace `COLUMN_NAME` with the name of the column that stores your item name data. ``` aws personalize create-batch-inference-job \ --job-name Themed batch job name \ --solution-version-arn Solution version ARN \ --filter-arn Filter ARN \ --job-input s3DataSource={path=s3://S3 input path} \ --job-output s3DataDestination={path=s3://S3 output path} \ --role-arn IAM service role ARN \ --batch-inference-job-mode THEME_GENERATION \ --theme-generation-config "{\"fieldsForThemeGeneration\": {\"itemName\":\"COLUMN_NAME\"}}" ``` ## Creating a batch inference job (AWS SDKs) After you have completed [Preparing input data for batch recommendations](batch-data-upload.md), you are ready to create a batch inference job with the [CreateBatchInferenceJob](API_CreateBatchInferenceJob.md) operation. **Topics** + [Creating a batch inference job](#batch-sdk-code-sample) + [Creating a batch inference job that generates themes](#batch-sdk-themes) ### Creating a batch inference job You can use the following code to create a batch inference job. Specify a job name, the Amazon Resource Name (ARN) of your solution version, and the ARN of the IAM service role you created for Amazon Personalize during set up. This role must have read and write access to your input and output Amazon S3 buckets. We recommend using a different location for your output data (either a folder or a different Amazon S3 bucket). Use the following syntax for input and output locations: **s3:/amzn-s3-demo-bucket//.json** and **s3://amzn-s3-demo-bucket//**. For `numResults`, specify the number of items you want Amazon Personalize to predict for each line of input data. Optionally provide a filter ARN to filter recommendations. If your filter uses placeholder parameters, make sure the values for the parameters are included in your input JSON. For more information, see [Filtering batch recommendations and user segments (custom resources)](filter-batch.md). ------ #### [ SDK for Python (Boto3) ] The example includes optional User-Personalization recipe specific `itemExplorationConfig` hyperparameters: `explorationWeight` and `explorationItemAgeCutOff`. Optionally include `explorationWeight` and `explorationItemAgeCutOff` values to configure exploration. For more information, see [User-Personalization recipe](native-recipe-new-item-USER_PERSONALIZATION.md). ``` import boto3 personalize_rec = boto3.client(service_name='personalize') personalize_rec.create_batch_inference_job ( solutionVersionArn = "Solution version ARN", jobName = "Batch job name", roleArn = "IAM service role ARN", filterArn = "Filter ARN", batchInferenceJobConfig = { # optional USER_PERSONALIZATION recipe hyperparameters "itemExplorationConfig": { "explorationWeight": "0.3", "explorationItemAgeCutOff": "30" } }, jobInput = {"s3DataSource": {"path": "s3://amzn-s3-demo-bucket//.json"}}, jobOutput = {"s3DataDestination": {"path": "s3:/amzn-s3-demo-bucket//"}} ) ``` ------ #### [ SDK for Java 2.x ] The example includes optional User-Personalization recipe specific `itemExplorationConfig` fields: `explorationWeight` and `explorationItemAgeCutOff`. Optionally include `explorationWeight` and `explorationItemAgeCutOff` values to configure exploration. For more information, see [User-Personalization recipe](native-recipe-new-item-USER_PERSONALIZATION.md). ``` public static String createPersonalizeBatchInferenceJob(PersonalizeClient personalizeClient, String solutionVersionArn, String jobName, String filterArn, String s3InputDataSourcePath, String s3DataDestinationPath, String roleArn, String explorationWeight, String explorationItemAgeCutOff) { long waitInMilliseconds = 60 * 1000; String status; String batchInferenceJobArn; try { // Set up data input and output parameters. S3DataConfig inputSource = S3DataConfig.builder() .path(s3InputDataSourcePath) .build(); S3DataConfig outputDestination = S3DataConfig.builder() .path(s3DataDestinationPath) .build(); BatchInferenceJobInput jobInput = BatchInferenceJobInput.builder() .s3DataSource(inputSource) .build(); BatchInferenceJobOutput jobOutputLocation = BatchInferenceJobOutput.builder() .s3DataDestination(outputDestination) .build(); // Optional code to build the User-Personalization specific item exploration config. HashMap explorationConfig = new HashMap<>(); explorationConfig.put("explorationWeight", explorationWeight); explorationConfig.put("explorationItemAgeCutOff", explorationItemAgeCutOff); BatchInferenceJobConfig jobConfig = BatchInferenceJobConfig.builder() .itemExplorationConfig(explorationConfig) .build(); // End optional User-Personalization recipe specific code. CreateBatchInferenceJobRequest createBatchInferenceJobRequest = CreateBatchInferenceJobRequest.builder() .solutionVersionArn(solutionVersionArn) .jobInput(jobInput) .jobOutput(jobOutputLocation) .jobName(jobName) .filterArn(filterArn) .roleArn(roleArn) .batchInferenceJobConfig(jobConfig) // Optional .build(); batchInferenceJobArn = personalizeClient.createBatchInferenceJob(createBatchInferenceJobRequest) .batchInferenceJobArn(); DescribeBatchInferenceJobRequest describeBatchInferenceJobRequest = DescribeBatchInferenceJobRequest.builder() .batchInferenceJobArn(batchInferenceJobArn) .build(); long maxTime = Instant.now().getEpochSecond() + 3 * 60 * 60; // wait until the batch inference job is complete. while (Instant.now().getEpochSecond() < maxTime) { BatchInferenceJob batchInferenceJob = personalizeClient .describeBatchInferenceJob(describeBatchInferenceJobRequest) .batchInferenceJob(); status = batchInferenceJob.status(); System.out.println("Batch inference job status: " + status); if (status.equals("ACTIVE") || status.equals("CREATE FAILED")) { break; } try { Thread.sleep(waitInMilliseconds); } catch (InterruptedException e) { System.out.println(e.getMessage()); } } return batchInferenceJobArn; } catch (PersonalizeException e) { System.out.println(e.awsErrorDetails().errorMessage()); } return ""; } ``` ------ #### [ SDK for JavaScript v3 ] ``` // Get service clients module and commands using ES6 syntax. import { CreateBatchInferenceJobCommand } from "@aws-sdk/client-personalize"; import { personalizeClient } from "./libs/personalizeClients.js"; // Or, create the client here. // const personalizeClient = new PersonalizeClient({ region: "REGION"}); // Set the batch inference job's parameters. export const createBatchInferenceJobParam = { jobName: "JOB_NAME", jobInput: { s3DataSource: { path: "INPUT_PATH", }, }, jobOutput: { s3DataDestination: { path: "OUTPUT_PATH", }, }, roleArn: "ROLE_ARN", solutionVersionArn: "SOLUTION_VERSION_ARN", numResults: 20, }; export const run = async () => { try { const response = await personalizeClient.send( new CreateBatchInferenceJobCommand(createBatchInferenceJobParam), ); console.log("Success", response); return response; // For unit tests. } catch (err) { console.log("Error", err); } }; run(); ``` ------ Processing the batch job might take a while to complete. You can check a job's status by calling [DescribeBatchInferenceJob](API_DescribeBatchInferenceJob.md) and passing a `batchRecommendationsJobArn` as the input parameter. You can also list all Amazon Personalize batch inference jobs in your AWS environment by calling [ListBatchInferenceJobs](API_ListBatchInferenceJobs.md). ### Creating a batch inference job that generates themes To generate themes for similar items, you must use the Similar-Items recipe and your Items dataset must have a textual field and a column of item name data. For more information about recommendations with themes, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). The following code creates a batch inference job that generates recommendations with themes. Leave the `batchInferenceJobMode` set to `"THEME_GENERATION"`. Replace `COLUMNN_NAME` with the name of the column that stores your item name data. ``` import boto3 personalize_rec = boto3.client(service_name='personalize') personalize_rec.create_batch_inference_job ( solutionVersionArn = "Solution version ARN", jobName = "Batch job name", roleArn = "IAM service role ARN", filterArn = "Filter ARN", batchInferenceJobMode = "THEME_GENERATION", themeGenerationConfig = { "fieldsForThemeGeneration": { "itemName": "COLUMN_NAME" } }, jobInput = {"s3DataSource": {"path": "s3://amzn-s3-demo-bucket//.json"}}, jobOutput = {"s3DataDestination": {"path": "s3://amzn-s3-demo-bucket//"}} ) ``` # Batch inference job output examples When you create a batch inference job, the job imports your batch input data from an Amazon S3 bucket, uses your solution version to generate *item recommendations*, and exports the recommendations to an Amazon S3 bucket in JSON format. The following sections list output file examples for batch inference jobs by recipe type. You can't get batch recommendations with the Trending-Now or Next-Best-Action recipes. **Topics** + [USER\$1PERSONALIZATION recipes](#batch-inference-output-user-personalization) + [POPULAR\$1ITEMS recipes](#batch-inference-output-popular-items) + [PERSONALIZED\$1RANKING recipes](#batch-inference-output-personalized-ranking) + [RELATED\$1ITEMS recipes](#batch-inference-output-related-items) ## USER\$1PERSONALIZATION recipes The following is an example of the output JSON file for a USER\$1PERSONALIZATION recipe. ``` {"input":{"userId":"4638"},"output":{"recommendedItems":["63992","115149","110102","148626","148888","31685","102445","69526","92535","143355","62374","7451","56171","122882","66097","91542","142488","139385","40583","71530","39292","111360","34048","47099","135137"],"scores":[0.0152238,0.0069081,0.0068222,0.006394,0.0059746,0.0055851,0.0049357,0.0044644,0.0042968,0.004015,0.0038805,0.0037476,0.0036563,0.0036178,0.00341,0.0033467,0.0033258,0.0032454,0.0032076,0.0031996,0.0029558,0.0029021,0.0029007,0.0028837,0.0028316]},"error":null} {"input":{"userId":"663"},"output":{"recommendedItems":["368","377","25","780","1610","648","1270","6","165","1196","1097","300","1183","608","104","474","736","293","141","2987","1265","2716","223","733","2028"],"scores":[0.0406197,0.0372557,0.0254077,0.0151975,0.014991,0.0127175,0.0124547,0.0116712,0.0091098,0.0085492,0.0079035,0.0078995,0.0075598,0.0074876,0.0072006,0.0071775,0.0068923,0.0066552,0.0066232,0.0062504,0.0062386,0.0061121,0.0060942,0.0060781,0.0059263]},"error":null} {"input":{"userId":"3384"},"output":{"recommendedItems":["597","21","223","2144","208","2424","594","595","920","104","520","367","2081","39","1035","2054","160","1370","48","1092","158","2671","500","474","1907"],"scores":[0.0241061,0.0119394,0.0118012,0.010662,0.0086972,0.0079428,0.0073218,0.0071438,0.0069602,0.0056961,0.0055999,0.005577,0.0054387,0.0051787,0.0051412,0.0050493,0.0047126,0.0045393,0.0042159,0.0042098,0.004205,0.0042029,0.0040778,0.0038897,0.0038809]},"error":null} ... ``` ## POPULAR\$1ITEMS recipes The following example shows the format of the output JSON file for the Popularity-Count recipe. You can't get batch recommendations with the Trending-Now recipe. ``` {"input": {"userId": "12"}, "output": {"recommendedItems": ["105", "106", "441"]}} {"input": {"userId": "105"}, "output": {"recommendedItems": ["105", "106", "441"]}} {"input": {"userId": "41"}, "output": {"recommendedItems": ["105", "106", "441"]}} ... ``` ## PERSONALIZED\$1RANKING recipes The following example shows the format of the output JSON file for a PERSONALIZED\$1RANKING recipe. ``` {"input":{"userId":"891","itemList":["27","886","101"]},"output":{"recommendedItems":["27","101","886"],"scores":[0.48421,0.28133,0.23446]}} {"input":{"userId":"445","itemList":["527","55","901"]},"output":{"recommendedItems":["901","527","55"],"scores":[0.46972,0.31011,0.22017]}} {"input":{"userId":"71","itemList":["29","351","199"]},"output":{"recommendedItems":["351","29","199"],"scores":[0.68937,0.24829,0.06232]}} ... ``` ## RELATED\$1ITEMS recipes The following example shows the format of the output JSON file for a RELATED\$1ITEMS recipe. ``` {"input": {"itemId": "105"}, "output": {"recommendedItems": ["106", "107", "49"]}} {"input": {"itemId": "106"}, "output": {"recommendedItems": ["105", "107", "49"]}} {"input": {"itemId": "441"}, "output": {"recommendedItems": ["2", "442", "435"]}} ... ``` The following example shows the format of the output JSON file for the Similar-Items recipe with themes. For more information about recommendations with themes, see [Batch recommendations with themes from Content Generator](themed-batch-recommendations.md). ``` {"input":{"itemId":"40"},"output":{"recommendedItems":["36","50","44","22","21","29","3","1","2","39"],"theme":"Movies with a strong female lead","itemsThemeRelevanceScores":[0.19994527,0.183059963,0.17478035,0.1618133,0.1574806,0.15468733,0.1499242,0.14353688,0.13531424,0.10291852]}} {"input":{"itemId":"43"},"output":{"recommendedItems":["50","21","36","3","17","2","39","1","10","5"],"theme":"The best movies of 1995","itemsThemeRelevanceScores":[0.184988,0.1795761,0.11143453,0.0989443,0.08258403,0.07952615,0.07115086,0.0621634,-0.138913,-0.188913]}} ... ```