Connecting Amazon Q Business to Amazon S3 using APIs - Amazon Q Business
Amazon S3 configuration propertiesAmazon S3 JSON schemaAmazon S3 JSON schema example

Connecting Amazon Q Business to Amazon S3 using APIs

You use the CreateDataSource action to connect a data source to your Amazon Q application.

Then, you use the configuration parameter to provide a JSON schema with all other configuration information specific to your data source connector.

For an example of the API request, see CreateDataSource in the Amazon Q API Reference.

Amazon S3 configuration properties

The following provides information about important configuration properties required in the schema.

Configuration Description Type Required

type

The type of data source. Specify S3 as your data source type.

string

The only allowed value is S3

.		 Yes

syncMode

Specify whether Amazon Q should update your index by syncing all documents or only new, modified, and deleted documents.

string

You can choose from the following options:

  • Use FORCED_FULL_CRAWL to freshly re-crawl all content and replace existing content each time your data source syncs with your index

  • Use FULL_CRAWL to incrementally crawl only new, modified, and deleted content each time your data source syncs with your index

 Yes

connectionConfiguration

Configuration information for the endpoint for the data source.

object

This property has a sub-property called repositoryEndpointMetadata.

 Yes

repositoryEndpointMetadata

This is the endpoint information for the data source. This is a sub-property for the connectionConfiguration.

object

This property has a sub-property called BucketName.

 Yes

BucketName

The name of your Amazon S3 bucket. This is a sub-property for the repositoryEndpointMetadata.

string

 Yes

repositoryConfigurations

Configuration information for the content of the data source. For example, configuring specific types of content and field mappings

.

object

This property has a sub-property called document.

 Yes

document

This property has information related to the document.

object

This property has a sub-property called fieldMappings.

 Yes

fieldMappings

This property has information related to the document.

array

This property has the following sub-properties.

  • indexFieldName

  • indexFieldType

  • dataSourceFieldName

 Yes

indexFieldName

The name of the index field. This is a sub-property for the fieldMappings.

string

 Yes

indexFieldType

The type of the index field. This is a sub-property for the fieldMappings.

string

The only allowed value is STRING.

 Yes

dataSourceFieldName

The field name of the data source. This is a sub-property for the fieldMappings.

string

 Yes

additionalProperties

Additional configuration options for your content in your data source.

object

This property has the following sub-properties that are not required

  • aclConfigurationFilePath

  • metadataFilesPrefix.

  • maxFileSizeInMegaBytes

  • inclusionPatterns

  • exclusionPatterns

  • inclusionPrefixes

  • exclusionPrefixes

 No

aclConfigurationFilePath

The path to the file that controls access control information for your documents in an Amazon Q index. This is a sub-property of additionalProperties.

string

 No

metadataFilesPrefix

The location, in your Amazon S3 bucket, of your document metadata files. This is a sub-property of additionalProperties.

string

 No

maxFileSizeInMegaBytes

Specify the maximum single file size limit in MBs that Amazon Q will crawl. Amazon Q will crawl only the files within the size limit you define.

The default file size is 50MB. The maximum file size should be greater than 0MB and less than or equal to 50MB.

string

You can enter a value between 0 and 50.

 No

All of these following are sub-properties of additionalProperties

  • inclusionPatterns

  • exclusionPatterns

  • inclusionPrefixes

  • exclusionPrefixes

A list of regular expression patterns to include or exclude specific files in your Amazon S3 data source. Files that match the patterns are included in the index. Files that don't match the patterns are excluded from the index. If a file matches both an inclusion and exclusion pattern, the exclusion pattern takes precedence and the file isn't included in the index.

array

 No

version

The version of the template that's supported.

string

The default value is 1.0.0.

 No

Amazon S3 JSON schema

The following is the Amazon S3 JSON schema:

{
  "type": "object",
  "properties": {
    "type": {
      "type": "string",
      "pattern": "S3"
    },
    "syncMode": {
      "type": "string",
      "enum": ["FULL_CRAWL", "FORCED_FULL_CRAWL"]
    },
    "connectionConfiguration": {
      "type": "object",
      "properties": {
        "repositoryEndpointMetadata": {
          "type": "object",
          "properties": {
            "BucketName": {
              "type": "string"
            }
          },
          "required": ["BucketName"]
        }
      },
      "required": ["repositoryEndpointMetadata"]
    },
    "repositoryConfigurations": {
      "type": "object",
      "properties": {
        "document": {
          "type": "object",
          "properties": {
            "fieldMappings": {
              "type": "array",
              "items": [
                {
                  "type": "object",
                  "properties": {
                    "indexFieldName": {
                      "type": "string"
                    },
                    "indexFieldType": {
                      "type": "string",
                      "enum": ["STRING"]
                    },
                    "dataSourceFieldName": {
                      "type": "string"
                    }
                  },
                  "required": [
                    "indexFieldName",
                    "indexFieldType",
                    "dataSourceFieldName"
                  ]
                }
              ]
            }
          },
          "required": ["fieldMappings"]
        }
      },
      "required": ["document"]
    },
    "additionalProperties": {
      "type": "object",
      "properties": {
        "inclusionPatterns": {
          "type": "array"
        },
        "exclusionPatterns": {
          "type": "array"
        },
        "inclusionPrefixes": {
          "type": "array"
        },
        "exclusionPrefixes": {
          "type": "array"
        },
        "aclConfigurationFilePath": {
          "type": "string"
        },
        "metadataFilesPrefix": {
          "type": "string"
        },
        "maxFileSizeInMegaBytes": {
          "type": "string"
        },
        "enableDeletionProtection": {
          "anyOf": [
            {
              "type": "boolean"
            },
            {
              "type": "string",
              "enum": ["true", "false"]
            }
          ]
        },
        "deletionProtectionThreshold": {
          "type": "string",
          "default": "15"
        }
      }
    }
  },
  "version": {
    "type": "string",
    "anyOf": [
      {
        "pattern": "1.0.0"
      }
    ]
  },
  "required": [
    "type",
    "syncMode",
    "connectionConfiguration",
    "repositoryConfigurations"
  ]
}
Show moreShow less

Amazon S3 JSON schema example

The following is the Amazon S3 JSON schema example:

{
  "type": "S3",
  "syncMode": "FULL_CRAWL",
  "connectionConfiguration": {
    "repositoryEndpointMetadata": {
      "BucketName": "my-company-data-bucket"
    }
  },
  "repositoryConfigurations": {
    "document": {
      "fieldMappings": [
        {
          "dataSourceFieldName": "content",
          "indexFieldName": "document_content",
          "indexFieldType": "STRING"
        }
      ]
    }
  },
  "additionalProperties": {
    "inclusionPatterns": ["*.pdf", "*.docx"],
    "exclusionPatterns": ["*.tmp"],
    "inclusionPrefixes": ["/important-docs/"],
    "exclusionPrefixes": ["/temporary/"],
    "aclConfigurationFilePath": "/configs/acl.json",
    "metadataFilesPrefix": "/metadata/",
    "maxFileSizeInMegaBytes": "50"
  }
}
Show moreShow less