You use the AWS::QBusiness::DataSource resource to connect a data source to
your Amazon Q application.
Use the configuration property to provide a JSON or YAML schema with the necessary
configuration details specific to your data source connector.
To learn more about AWS CloudFormation, see What is AWS CloudFormation? in the AWS CloudFormation User Guide.
Topics
GitHub (Cloud) configuration
properties
The following provides information about important configuration properties required in the schema.
| Configuration | Description | Type | Required |
|---|---|---|---|
|
Configuration information for the endpoint for the data source. |
This property has a sub-property called
|
Yes |
|
The endpoint information for the data source. |
This property has the following sub-properties.
|
Yes |
|
The GitHub (Cloud) host URL. For example, if you use GitHub (Cloud) Enterprise Cloud: https://api.github.com. |
|
Yes |
|
The hosting method for your GitHub instance. |
The only allowed value is |
Yes |
|
You can find your organization name when you log in to GitHub (Cloud) desktop and go to Your organizations under your profile picture dropdown. |
|
Yes |
|
Configuration information for the content of the data source. For example, configuring specific types of content and field mappings. |
This propery has the following sub-properties:
|
Yes |
|
A list of objects that map the attributes or field names of your GitHub (Cloud) pages and assets to Amazon Q index field names. |
These properties have the following sub-properties.
|
No |
|
The field name of your GitHub (Cloud) pages and assets. |
|
Yes |
|
The field type of your GitHub (Cloud) pages and assets. |
The allowed values are |
Yes |
|
The data source field name of your GitHub (Cloud) pages and assets. |
|
Yes |
|
The date format of your GitHub (Cloud) pages and assets. |
Specify the date format in the form |
No |
|
Additional configuration options for your content in your data source. |
This property has the following sub-properties.
|
Yes |
|
Specify true to crawl access control information from
documents. |
|
No |
|
Specify the maximum single file size limit in MBs that Amazon Q will crawl. Amazon Q will crawl only the files within the size limit you define. The default file size is 50MB. The maximum file size should be greater than 0MB and less than or equal to 50MB. |
The allowed values are numbers between greater than 0 and less than or equal to 50. |
No |
|
Specify field to use for UserId for ACL crawling. |
|
No |
| repositoryFilter | A list of names of the specific repositories and branch names you want to index. |
This property has the following sub-properties: |
No |
|
The list of repository names that you want to index. |
|
No |
|
The list of branch names that you want to index. |
|
No |
|
Specify true to crawl repositories. |
|
No |
|
Specify true to crawl repository documents. |
|
No |
|
Specify true to crawl issues. |
|
No |
|
Specify true to crawl issue comments. |
|
No |
|
Specify true to crawl issue comment attachments. |
|
No |
|
Specify true to crawl pull requests. |
|
No |
|
Specify true to crawl pull request comments. |
|
No |
|
Specify true to crawl pull request comment attachments. |
|
No |
|
A list of regular expression patterns to include specific content in your GitHub (Cloud) data source. Content that matches the patterns are included in the index. Content that doesn't match the patterns are excluded from the index. If any content matches both an inclusion and exclusion pattern, the exclusion pattern takes precedence, and the content isn't included in the index. |
|
No |
|
A list of regular expression patterns to exclude specific content in your GitHub (Cloud) data source. Content that matches the patterns are included in the index. Content that doesn't match the patterns are excluded from the index. If any content matches both an inclusion and exclusion pattern, the exclusion pattern takes precedence, and the content isn't included in the index. |
|
No |
|
The type of data source. Specify GITHUB as your data source
type. |
|
Yes |
|
Specify true to use the Amazon Q identity crawler to sync
identity/principal information on users and groups with access to specific
documents. |
|
Yes |
|
Specify whether Amazon Q should update your index by syncing all documents or only new, modified, and deleted documents. |
You can choose between the following options:
|
Yes |
|
The Amazon Resource Name (ARN) of an AWS Secrets Manager secret that contains the key-value pairs required to connect to your GitHub (Cloud). |
The secret must contain a JSON structure with the following keys: |
No |
|
The version of this template that's currently supported. |
|
No |
GitHub (Cloud) JSON schema for using the
configuration property with AWS CloudFormation
The following is the GitHub (Cloud) JSON schema and examples for the configuration property for AWS CloudFormation.
Topics
GitHub (Cloud) JSON schema for using the configuration property with AWS CloudFormation
The following is the GitHub (Cloud) JSON schema for the configuration property for AWS CloudFormation
{
"type": "object",
"properties": {
"type": {
"type": "string",
"pattern": "GITHUB"
},
"syncMode": {
"type": "string",
"enum": ["FULL_CRAWL", "FORCED_FULL_CRAWL", "CHANGE_LOG"]
},
"secretArn": {
"type": "string",
"minLength": 20,
"maxLength": 2048
},
"enableIdentityCrawler": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"connectionConfiguration": {
"type": "object",
"properties": {
"repositoryEndpointMetadata": {
"type": "object",
"properties": {
"type": {
"type": "string"
},
"hostUrl": {
"type": "string",
"pattern": "https://.*"
},
"organizationName": {
"type": "string"
}
},
"required": ["type", "hostUrl", "organizationName"]
}
},
"required": ["repositoryEndpointMetadata"]
},
"repositoryConfigurations": {
"type": "object",
"properties": {
"ghRepository": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghCommit": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghIssueDocument": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghIssueComment": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghIssueAttachment": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghPRDocument": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghPRComment": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
},
"ghPRAttachment": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": ["STRING", "STRING_LIST", "DATE"]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": ["fieldMappings"]
}
}
},
"additionalProperties": {
"type": "object",
"properties": {
"isCrawlAcl": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"maxFileSizeInMegaBytes": {
"type": "string"
},
"fieldForUserId": {
"type": "string"
},
"crawlRepository": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlRepositoryDocuments": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlIssue": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlIssueComment": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlIssueCommentAttachment": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlPullRequest": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlPullRequestComment": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"crawlPullRequestCommentAttachment": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
]
},
"repositoryFilter": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"repositoryName": {
"type": "string"
},
"branchNameList": {
"type": "array",
"items": {
"type": "string"
}
}
}
}
]
},
"inclusionFolderNamePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"inclusionFileTypePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"inclusionFileNamePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"exclusionFolderNamePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"exclusionFileTypePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"exclusionFileNamePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"enableDeletionProtection": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "string",
"enum": ["true", "false"]
}
],
"default": false
},
"deletionProtectionThreshold": {
"type": "string",
"default": "15"
}
},
"required": []
},
"version": {
"type": "string",
"anyOf": [
{
"pattern": "1.0.0"
}
]
}
},
"required": [
"syncMode",
"enableIdentityCrawler",
"connectionConfiguration",
"repositoryConfigurations",
"additionalProperties"
]
}GitHub (Cloud) JSON schema example for using the configuration property with AWS CloudFormation
The following is the GitHub (Cloud) JSON schema example for the configuration property for AWS CloudFormation
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "CloudFormation GITHUB Data Source Template",
"Resources": {
"DataSourceGitHub": {
"Type": "AWS::QBusiness::DataSource",
"Properties": {
"ApplicationId": "app12345-1234-1234-1234-123456789012",
"IndexId": "indx1234-1234-1234-1234-123456789012",
"DisplayName": "MyGitHubDataSource",
"RoleArn": "arn:aws:iam::123456789012:role/qbusiness-data-source-role",
"Configuration": {
"type": "GITHUB",
"syncMode": "FULL_CRAWL",
"secretArn": "arn:aws:secretsmanager:us-west-2:123456789012:secret:my-github-secret",
"enableIdentityCrawler": "true",
"sslCertificatePath": {
"bucket": "my-github-bucket",
"key": "certificates/my-cert.pem"
},
"connectionConfiguration": {
"repositoryEndpointMetadata": {
"type": "GitHub",
"hostUrl": "https://api.github.com",
"organizationName": "my-org"
}
},
"repositoryConfigurations": {
"ghRepository": {
"fieldMappings": [
{
"indexFieldName": "repo_name",
"indexFieldType": "STRING",
"dataSourceFieldName": "name",
"dateFieldFormat": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
]
},
"ghCommit": {
"fieldMappings": [
{
"indexFieldName": "commit_id",
"indexFieldType": "STRING",
"dataSourceFieldName": "id",
"dateFieldFormat": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
]
}
},
"additionalProperties": {
"isCrawlAcl": "true",
"maxFileSizeInMegaBytes": "50",
"crawlRepository": "true",
"crawlIssue": "true",
"repositoryFilter": [
{
"repositoryName": "my-repo",
"branchNameList": ["main", "develop"]
}
],
"inclusionFileTypePatterns": ["*.md", "*.txt"],
"exclusionFileNamePatterns": ["*draft*"],
"enableDeletionProtection": "false",
"deletionProtectionThreshold": "15"
}
}
}
}
}
}GitHub (Cloud) YAML schema for using the
configuration property with AWS CloudFormation
The following is the GitHub (Cloud) YAML schema and examples for the configuration property for AWS CloudFormation:
Topics
GitHub (Cloud) YAML schema for using the configuration property with AWS CloudFormation
The following is the GitHub (Cloud) YAML schema for the configuration property for AWS CloudFormation.
type: object
properties:
type:
type: string
pattern: GITHUB
syncMode:
type: string
enum:
- FULL_CRAWL
- FORCED_FULL_CRAWL
- CHANGE_LOG
secretArn:
type: string
minLength: 20
maxLength: 2048
enableIdentityCrawler:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
connectionConfiguration:
type: object
properties:
repositoryEndpointMetadata:
type: object
properties:
type:
type: string
hostUrl:
type: string
pattern: "https://.*"
organizationName:
type: string
required:
- type
- hostUrl
- organizationName
required:
- repositoryEndpointMetadata
repositoryConfigurations:
type: object
properties:
ghRepository:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghCommit:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghIssueDocument:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghIssueComment:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghIssueAttachment:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghPRDocument:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghPRComment:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
ghPRAttachment:
type: object
properties:
fieldMappings:
type: array
items:
type: object
properties:
indexFieldName:
type: string
indexFieldType:
type: string
enum:
- STRING
- STRING_LIST
- DATE
dataSourceFieldName:
type: string
dateFieldFormat:
type: string
pattern: "yyyy-MM-dd'T'HH:mm:ss'Z'"
required:
- indexFieldName
- indexFieldType
- dataSourceFieldName
required:
- fieldMappings
additionalProperties:
type: object
properties:
isCrawlAcl:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
maxFileSizeInMegaBytes:
type: string
fieldForUserId:
type: string
crawlRepository:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlRepositoryDocuments:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlIssue:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlIssueComment:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlIssueCommentAttachment:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlPullRequest:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlPullRequestComment:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
crawlPullRequestCommentAttachment:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
repositoryFilter:
type: array
items:
type: object
properties:
repositoryName:
type: string
branchNameList:
type: array
items:
type: string
inclusionFolderNamePatterns:
type: array
items:
type: string
inclusionFileTypePatterns:
type: array
items:
type: string
inclusionFileNamePatterns:
type: array
items:
type: string
exclusionFolderNamePatterns:
type: array
items:
type: string
exclusionFileTypePatterns:
type: array
items:
type: string
exclusionFileNamePatterns:
type: array
items:
type: string
enableDeletionProtection:
anyOf:
- type: boolean
- type: string
enum:
- true
- false
default: false
deletionProtectionThreshold:
type: string
default: "15"
required: []
version:
type: string
anyOf:
- pattern: 1.0.0
required:
- syncMode
- enableIdentityCrawler
- connectionConfiguration
- repositoryConfigurations
- additionalPropertiesGitHub (Cloud) YAML schema example for using the configuration property with AWS CloudFormation
The following is the GitHub (Cloud) YAML example for the Configuration property for AWS CloudFormation:
AWSTemplateFormatVersion: "2010-09-09"
Description: CloudFormation GITHUB Data Source Template
Resources:
DataSourceGitHub:
Type: AWS::QBusiness::DataSource
Properties:
ApplicationId: app12345-1234-1234-1234-123456789012
IndexId: indx1234-1234-1234-1234-123456789012
DisplayName: MyGitHubDataSource
RoleArn: arn:aws:iam::123456789012:role/qbusiness-data-source-role
Configuration:
type: GITHUB
syncMode: FULL_CRAWL
secretArn: arn:aws:secretsmanager:us-west-2:123456789012:secret:my-github-secret
enableIdentityCrawler: "true"
sslCertificatePath:
bucket: my-github-bucket
key: certificates/my-cert.pem
connectionConfiguration:
repositoryEndpointMetadata:
type: GitHub
hostUrl: https://api.github.com
organizationName: my-org
repositoryConfigurations:
ghRepository:
fieldMappings:
- indexFieldName: repo_name
indexFieldType: STRING
dataSourceFieldName: name
dateFieldFormat: yyyy-MM-dd'T'HH:mm:ss'Z'
ghCommit:
fieldMappings:
- indexFieldName: commit_id
indexFieldType: STRING
dataSourceFieldName: id
dateFieldFormat: yyyy-MM-dd'T'HH:mm:ss'Z'
additionalProperties:
isCrawlAcl: "true"
maxFileSizeInMegaBytes: "50"
crawlRepository: "true"
crawlIssue: "true"
repositoryFilter:
- repositoryName: my-repo
branchNameList:
- main
- develop
inclusionFileTypePatterns:
- "*.md"
- "*.txt"
exclusionFileNamePatterns:
- "*draft*"
enableDeletionProtection: "false"
deletionProtectionThreshold: "15"