You use the CreateDataSource action to connect a data source to your Amazon Q application.
Then, you use the
configuration parameter to provide a JSON blob that conforms the AWS-defined JSON schema.
For an example of the API request, see CreateDataSource in the Amazon Q API Reference.
Amazon S3 configuration properties
The following provides information about important configuration properties required in the schema.
| Configuration | Description | Type | Required |
|---|---|---|---|
|
The type of data source. Specify |
The only allowed value is |
Yes |
|
Specify whether Amazon Q should update your index by syncing all documents or only new, modified, and deleted documents. |
You can choose from the following options:
|
Yes |
|
Configuration information for the endpoint for the data source. |
This property has a sub-property called
|
Yes |
|
This is the endpoint information for the data source. This is a sub-property
for the |
This property has a sub-property called |
Yes |
|
The name of your Amazon S3 bucket. This is a sub-property for the
|
|
Yes |
|
Configuration information for the content of the data source. For example, configuring specific types of content and field mappings . |
This property has a sub-property called |
Yes |
|
This property has information related to the document. |
This property has a sub-property called
|
Yes |
|
This property has information related to the document. |
This property has the following sub-properties.
|
Yes |
|
The name of the index field. This is a sub-property for the
|
|
Yes |
|
The type of the index field. This is a sub-property for the
|
The only allowed value is |
Yes |
|
The field name of the data source. This is a sub-property for the
|
|
Yes |
|
Additional configuration options for your content in your data source. |
This property has the following sub-properties that are not required
|
No |
|
The path to the file that controls access control information for your
documents in an Amazon Q index. This is a sub-property of
|
|
No |
|
The location, in your Amazon S3 bucket, of your document metadata
files. This is a sub-property of |
|
No |
|
Specify the maximum single file size limit in MBs that Amazon Q will crawl. Amazon Q will crawl only the files within the size limit you define. The default file size is 50MB. The maximum file size should be greater than 0MB and less than or equal to 50MB. You can go up to 10 GB (10240 MB) if you enable Video files in Multi-media content configuration, and up to 2 GB (2048 MB) if you enable Audio files in Multi-media content configuration. |
You can enter a value between |
No |
All of these following are sub-properties of
|
A list of regular expression patterns to include or exclude specific files in your Amazon S3 data source. Files that match the patterns are included in the index. Files that don't match the patterns are excluded from the index. If a file matches both an inclusion and exclusion pattern, the exclusion pattern takes precedence and the file isn't included in the index. |
|
No |
|
The version of the template that's supported. |
The default value is |
No |
Amazon S3 JSON schema
The following is the Amazon S3 JSON schema:
{
"type": "object",
"properties": {
"type": {
"type": "string",
"pattern": "S3"
},
"syncMode": {
"type": "string",
"enum": ["FULL_CRAWL", "FORCED_FULL_CRAWL"]
},
"connectionConfiguration": {
"type": "object",
"properties": {
"repositoryEndpointMetadata": {
"type": "object",
"properties": {
"BucketName": {
"type": "string"
}
},
"required": ["BucketName"]
}
},
"required": ["repositoryEndpointMetadata"]
},
"repositoryConfigurations": {
"type": "object",
"properties": {
"document": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING"]
},
"dataSourceFieldName": {
"type": "string"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
}
},
"required": ["document"]
},
"additionalProperties": {
"type": "object",
"properties": {
"inclusionPatterns": {
"type": "array"
},
"exclusionPatterns": {
"type": "array"
},
"inclusionPrefixes": {
"type": "array"
},
"exclusionPrefixes": {
"type": "array"
},
"aclConfigurationFilePath": {
"type": "string"
},
"metadataFilesPrefix": {
"type": "string"
},
"maxFileSizeInMegaBytes": {
"type": "string"
},
"enableDeletionProtection": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"deletionProtectionThreshold": {
"type": "string",
"default": "15"
}
}
}
},
"version": {
"type": "string",
"anyOf": [
{
"pattern": "1.0.0"
}
]
},
"required": [
"type",
"syncMode",
"connectionConfiguration",
"repositoryConfigurations"
]
}Amazon S3 JSON schema example
The following is the Amazon S3 JSON schema example:
{
"type": "S3",
"syncMode": "FULL_CRAWL",
"connectionConfiguration": {
"repositoryEndpointMetadata": {
"BucketName": "my-company-data-bucket"
}
},
"repositoryConfigurations": {
"document": {
"fieldMappings": [
{
"dataSourceFieldName": "content",
"indexFieldName": "document_content",
"indexFieldType": "STRING"
}
]
}
},
"additionalProperties": {
"inclusionPatterns": ["*.pdf", "*.docx"],
"exclusionPatterns": ["*.tmp"],
"inclusionPrefixes": ["/important-docs/"],
"exclusionPrefixes": ["/temporary/"],
"aclConfigurationFilePath": "/configs/acl.json",
"metadataFilesPrefix": "/metadata/",
"maxFileSizeInMegaBytes": "10240"
}
}