Creating custom document fields

You can create custom attributes or fields for your documents in your Amazon Kendra index. For example, you can create a custom field or attribute called "Department" with the values of "HR", "Sales", and "Manufacturing". If you map these custom fields or attributes to your Amazon Kendra index, you can use them to filter the search results to include documents by the "HR" department attribute, for example.

Before you can use a custom field or attribute, you must first create the field in the index. Use the console to edit the data source field mappings to add a custom field or use the UpdateIndex API to create the index field. You cannot change the field data type once you have created the field.

For most data sources, you map fields in the external data source to the corresponding fields in Amazon Kendra. For more information, see Mapping data source fields. For S3 data sources, you can create custom fields or attributes using a JSON metadata file.

You can create up to 500 custom fields or attributes.

You can also use Amazon Kendra reserved or common fields. For more information, see Document attributes or fields.

Updating custom document fields

With the UpdateIndex API, you add custom fields or attributes using the DocumentMetadataConfigurationUpdates parameter.

The following JSON example uses DocumentMetadataConfigurationUpdates to add a field called "Department" to the index.

"DocumentmetadataConfigurationUpdates": [
   {
       "Name": "Department",
       "Type": "STRING_VALUE"
   }
]

The following sections include examples for adding custom attributes or fields using the BatchPutDocument and for an Amazon S3 data source.

Adding custom attributes or fields with the BatchPutDocument API

When you use the BatchPutDocument API to add a document to your index, you specify custom fields or attributes as part of Attributes. You can add multiple fields or attributes when you call the API. You can create up to 500 custom fields or attributes. The following example is a custom field or attribute that adds "Department" to a document.

"Attributes": 
    {
        "Department": "HR",
        "_category": "Vacation policy"
    }

Adding custom attributes or fields to an Amazon S3 data source

When you use an S3 bucket as a data source for your index, you add metadata to the documents with companion metadata files. You place the metadata JSON files in a directory structure that is parallel to your documents. For more information, see S3 document metadata.

You specify custom fields or attributes in the Attributes JSON structure. You can create up to 500 custom fields or attributes. For example, the following example uses Attributes to define three custom fields or attributes and one reserved field.

"Attributes": {
        "brand": "Amazon Basics",
        "price": 1595,
        "_category": "sports",
        "subcategories": ["outdoors", "electronics"]
    }

The following steps walk you through adding custom attributes to an Amazon S3 data source.

Step 1: Create a Amazon Kendra index

Follow the steps in Creating an index to create your Amazon Kendra index.

Step 2: Update index to add custom document fields

After creating an index, you add fields to it. The following procedure shows how to add fields to an index using the console and the CLI.

Console

To create index fields

1. Make sure you've created an index.

2. Then, from the left navigation menu, from Data management, choose Facet definition.

3. In Index field settings guide, from Index fields, choose Add field to add custom fields.

4. In the Add index field dialog box, do the following:

   • Field name – Add a field name.

   • Data type – Select data type, whether String, String list, or Date.

   • Usage types – Select usage types, whether Facetable, Searchable, Displayable, and Sortable.

     Then, select Add.

   Repeat the last step for any other fields you want to map.

CLI

aws kendra update-index  \
--region $region \
--endpoint-url $endpoint \
--application-id $applicationId \
--index-id $indexId  \
--document-metadata-configuration-updates \
"[
    {
        "Name": "string",
        "Type": "STRING_VALUE"|"STRING_LIST_VALUE"|"LONG_VALUE"|"DATE_VALUE",
        "Relevance": {
            "Freshness": true|false,
            "Importance": integer,
            "Duration": "string",
            "RankOrder": "ASCENDING"|"DESCENDING",
            "ValueImportanceMap": {"string": integer
            ...}
    },
    "Search": {
        "Facetable": true|false,
        "Searchable": true|false,
        "Displayable": true|false,
        "Sortable": true|false
        }
    }
...
]"

Step 3: Create an Amazon S3 data source and map data source fields to custom attributes

To create an Amazon S3 data source and map fields to it, follow the instructions in Amazon S3.

If you're using the API, use the fieldMappings attribute under configuration when you use the CreateDataSource API.

For an overview of how data source fields are mapped, see Mapping data source fields.