Creating an Amazon S3 data source
The following examples demonstrate creating an Amazon S3 data source. The
examples assume that you have already created an index and an IAM role
with permission to read the data from the index. For more information about the IAM role, see IAM access
roles. For more information about creating an index, see Creating an
index.
- CLI
-
aws kendra create-data-source \
--index-id index ID \
--name example-data-source \
--type S3 \
--configuration '{"S3Configuration":{"BucketName":"bucket name"}}'
--role-arn 'arn:aws:iam::account id:role:/role name
- Python
-
The following snippet of Python code creates an Amazon S3 data
source. For the complete example, see Getting started (AWS SDK for Python (Boto3)).
print("Create an Amazon S3 data source.")
name = "getting-started-data-source"
description = "Getting started data source."
role_arn = "arn:aws:iam::${accountID}:role/${roleName}"
s3_bucket_name = "S3-bucket-name"
type = "S3"
configuration = {"S3DataSourceConfiguration":
{
"BucketName": s3_bucket_name
}
}
data_source_response = kendra.create_data_source(
Configuration = configuration,
Name = name,
Description = description,
RoleArn = role_arn,
Type = type,
IndexId = index_id
)
It can take some time to create your data source. You can monitor the progress by
using the DescribeDataSource API. When the data source status is ACTIVE
the data source is ready to use.
The following examples demonstrate getting the status of a data source.
- CLI
-
aws kendra describe-data-source \
--index-id index ID \
--id data source ID
- Python
-
The following snippet of Python code gets information about an S3 data
source. For the complete example, see Getting started (AWS SDK for Python (Boto3)).
print("Wait for Amazon Kendra to create the data source.")
while True:
data_source_description = kendra.describe_data_source(
Id = "data-source-id",
IndexId = "index-id"
)
status = data_source_description["Status"]
print(" Creating data source. Status: "+status)
time.sleep(60)
if status != "CREATING":
break
This data source doesn't have a schedule, so it doesn't run automatically. To index
the data source, you call StartDataSourceSyncJob to synchronize the index with the data
source.
The following examples demonstrate synchronizing a data source.
- CLI
-
aws kendra start-data-source-sync-job \
--index-id index ID \
--id data source ID
- Python
-
The following snippet of Python code synchronizes an Amazon S3
data source. For the complete example, see Getting started (AWS SDK for Python (Boto3)).
print("Synchronize the data source.")
sync_response = kendra.start_data_source_sync_job(
Id = "data-source-id",
IndexId = "index-id"
)
