There are more AWS SDK examples available in the [AWS Doc SDK Examples](https://github.com/awsdocs/aws-doc-sdk-examples) GitHub repo.
# Code examples for Amazon Comprehend using AWS SDKs
The following code examples show you how to use Amazon Comprehend with an AWS software development kit (SDK).
*Actions* are code excerpts from larger programs and must be run in context. While actions show you how to call individual service functions, you can see actions in context in their related scenarios.
*Scenarios* are code examples that show you how to accomplish specific tasks by calling multiple functions within a service or combined with other AWS services.
**More resources**
+ **[ Amazon Comprehend Developer Guide](https://docs.aws.amazon.com/comprehend/latest/dg/what-is.html)** – More information about Amazon Comprehend.
+ **[Amazon Comprehend API Reference](https://docs.aws.amazon.com/comprehend/latest/APIReference/welcome.html)** – Details about all available Amazon Comprehend actions.
+ **[AWS Developer Center](https://aws.amazon.com/developer/code-examples/?awsf.sdk-code-examples-product=product%23comprehend)** – Code examples that you can filter by category or full-text search.
+ **[AWS SDK Examples](https://github.com/awsdocs/aws-doc-sdk-examples)** – GitHub repo with complete code in preferred languages. Includes instructions for setting up and running the code.
**Contents**
+ [Basics](comprehend_code_examples_basics.md)
+ [Actions](comprehend_code_examples_actions.md)
+ [`CreateDocumentClassifier`](comprehend_example_comprehend_CreateDocumentClassifier_section.md)
+ [`DeleteDocumentClassifier`](comprehend_example_comprehend_DeleteDocumentClassifier_section.md)
+ [`DescribeDocumentClassificationJob`](comprehend_example_comprehend_DescribeDocumentClassificationJob_section.md)
+ [`DescribeDocumentClassifier`](comprehend_example_comprehend_DescribeDocumentClassifier_section.md)
+ [`DescribeTopicsDetectionJob`](comprehend_example_comprehend_DescribeTopicsDetectionJob_section.md)
+ [`DetectDominantLanguage`](comprehend_example_comprehend_DetectDominantLanguage_section.md)
+ [`DetectEntities`](comprehend_example_comprehend_DetectEntities_section.md)
+ [`DetectKeyPhrases`](comprehend_example_comprehend_DetectKeyPhrases_section.md)
+ [`DetectPiiEntities`](comprehend_example_comprehend_DetectPiiEntities_section.md)
+ [`DetectSentiment`](comprehend_example_comprehend_DetectSentiment_section.md)
+ [`DetectSyntax`](comprehend_example_comprehend_DetectSyntax_section.md)
+ [`ListDocumentClassificationJobs`](comprehend_example_comprehend_ListDocumentClassificationJobs_section.md)
+ [`ListDocumentClassifiers`](comprehend_example_comprehend_ListDocumentClassifiers_section.md)
+ [`ListTopicsDetectionJobs`](comprehend_example_comprehend_ListTopicsDetectionJobs_section.md)
+ [`StartDocumentClassificationJob`](comprehend_example_comprehend_StartDocumentClassificationJob_section.md)
+ [`StartTopicsDetectionJob`](comprehend_example_comprehend_StartTopicsDetectionJob_section.md)
+ [Scenarios](comprehend_code_examples_scenarios.md)
+ [Build an Amazon Transcribe streaming app](comprehend_example_cross_TranscriptionStreamingApp_section.md)
+ [Building an Amazon Lex chatbot](comprehend_example_cross_LexChatbotLanguages_section.md)
+ [Create a messaging application](comprehend_example_cross_SQSMessageApp_section.md)
+ [Create an application to analyze customer feedback](comprehend_example_cross_FSA_section.md)
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
+ [Detect entities in text extracted from an image](comprehend_example_cross_TextractComprehendDetectEntities_section.md)
+ [Run a topic modeling job on sample data](comprehend_example_comprehend_Usage_TopicModeler_section.md)
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
# Basic examples for Amazon Comprehend using AWS SDKs
The following code examples show how to use the basics of Amazon Comprehend with AWS SDKs.
**Contents**
+ [Actions](comprehend_code_examples_actions.md)
+ [`CreateDocumentClassifier`](comprehend_example_comprehend_CreateDocumentClassifier_section.md)
+ [`DeleteDocumentClassifier`](comprehend_example_comprehend_DeleteDocumentClassifier_section.md)
+ [`DescribeDocumentClassificationJob`](comprehend_example_comprehend_DescribeDocumentClassificationJob_section.md)
+ [`DescribeDocumentClassifier`](comprehend_example_comprehend_DescribeDocumentClassifier_section.md)
+ [`DescribeTopicsDetectionJob`](comprehend_example_comprehend_DescribeTopicsDetectionJob_section.md)
+ [`DetectDominantLanguage`](comprehend_example_comprehend_DetectDominantLanguage_section.md)
+ [`DetectEntities`](comprehend_example_comprehend_DetectEntities_section.md)
+ [`DetectKeyPhrases`](comprehend_example_comprehend_DetectKeyPhrases_section.md)
+ [`DetectPiiEntities`](comprehend_example_comprehend_DetectPiiEntities_section.md)
+ [`DetectSentiment`](comprehend_example_comprehend_DetectSentiment_section.md)
+ [`DetectSyntax`](comprehend_example_comprehend_DetectSyntax_section.md)
+ [`ListDocumentClassificationJobs`](comprehend_example_comprehend_ListDocumentClassificationJobs_section.md)
+ [`ListDocumentClassifiers`](comprehend_example_comprehend_ListDocumentClassifiers_section.md)
+ [`ListTopicsDetectionJobs`](comprehend_example_comprehend_ListTopicsDetectionJobs_section.md)
+ [`StartDocumentClassificationJob`](comprehend_example_comprehend_StartDocumentClassificationJob_section.md)
+ [`StartTopicsDetectionJob`](comprehend_example_comprehend_StartTopicsDetectionJob_section.md)
# Actions for Amazon Comprehend using AWS SDKs
The following code examples demonstrate how to perform individual Amazon Comprehend actions with AWS SDKs. Each example includes a link to GitHub, where you can find instructions for setting up and running the code.
These excerpts call the Amazon Comprehend API and are code excerpts from larger programs that must be run in context. You can see actions in context in [Scenarios for Amazon Comprehend using AWS SDKs](comprehend_code_examples_scenarios.md).
The following examples include only the most commonly used actions. For a complete list, see the [Amazon Comprehend API Reference](https://docs.aws.amazon.com/comprehend/latest/APIReference/welcome.html).
**Topics**
+ [`CreateDocumentClassifier`](comprehend_example_comprehend_CreateDocumentClassifier_section.md)
+ [`DeleteDocumentClassifier`](comprehend_example_comprehend_DeleteDocumentClassifier_section.md)
+ [`DescribeDocumentClassificationJob`](comprehend_example_comprehend_DescribeDocumentClassificationJob_section.md)
+ [`DescribeDocumentClassifier`](comprehend_example_comprehend_DescribeDocumentClassifier_section.md)
+ [`DescribeTopicsDetectionJob`](comprehend_example_comprehend_DescribeTopicsDetectionJob_section.md)
+ [`DetectDominantLanguage`](comprehend_example_comprehend_DetectDominantLanguage_section.md)
+ [`DetectEntities`](comprehend_example_comprehend_DetectEntities_section.md)
+ [`DetectKeyPhrases`](comprehend_example_comprehend_DetectKeyPhrases_section.md)
+ [`DetectPiiEntities`](comprehend_example_comprehend_DetectPiiEntities_section.md)
+ [`DetectSentiment`](comprehend_example_comprehend_DetectSentiment_section.md)
+ [`DetectSyntax`](comprehend_example_comprehend_DetectSyntax_section.md)
+ [`ListDocumentClassificationJobs`](comprehend_example_comprehend_ListDocumentClassificationJobs_section.md)
+ [`ListDocumentClassifiers`](comprehend_example_comprehend_ListDocumentClassifiers_section.md)
+ [`ListTopicsDetectionJobs`](comprehend_example_comprehend_ListTopicsDetectionJobs_section.md)
+ [`StartDocumentClassificationJob`](comprehend_example_comprehend_StartDocumentClassificationJob_section.md)
+ [`StartTopicsDetectionJob`](comprehend_example_comprehend_StartTopicsDetectionJob_section.md)
# Use `CreateDocumentClassifier` with an AWS SDK or CLI
The following code examples show how to use `CreateDocumentClassifier`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To create a document classifier to categorize documents**
The following `create-document-classifier` example begins the training process for a document classifier model. The training data file, `training.csv`, is located at the `--input-data-config` tag. `training.csv` is a two column document where the labels, or, classifications are provided in the first column and the documents are provided in the second column.
```
aws comprehend create-document-classifier \
--document-classifier-name example-classifier \
--data-access-arn arn:aws:comprehend:us-west-2:111122223333:pii-entities-detection-job/123456abcdeb0e11022f22a11EXAMPLE \
--input-data-config "S3Uri=s3://amzn-s3-demo-bucket/" \
--language-code en
```
Output:
```
{
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:111122223333:document-classifier/example-classifier"
}
```
For more information, see [Custom Classification](https://docs.aws.amazon.com/comprehend/latest/dg/how-document-classification.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [CreateDocumentClassifier](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/create-document-classifier.html) in *AWS CLI Command Reference*.
------
#### [ Java ]
**SDK for Java 2.x**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/comprehend#code-examples).
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.comprehend.ComprehendClient;
import software.amazon.awssdk.services.comprehend.model.ComprehendException;
import software.amazon.awssdk.services.comprehend.model.CreateDocumentClassifierRequest;
import software.amazon.awssdk.services.comprehend.model.CreateDocumentClassifierResponse;
import software.amazon.awssdk.services.comprehend.model.DocumentClassifierInputDataConfig;
/**
* Before running this code example, you can setup the necessary resources, such
* as the CSV file and IAM Roles, by following this document:
* https://aws.amazon.com/blogs/machine-learning/building-a-custom-classifier-using-amazon-comprehend/
*
* Also, set up your development environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DocumentClassifierDemo {
public static void main(String[] args) {
final String usage = """
Usage:
Where:
dataAccessRoleArn - The ARN value of the role used for this operation.
s3Uri - The Amazon S3 bucket that contains the CSV file.
documentClassifierName - The name of the document classifier.
""";
if (args.length != 3) {
System.out.println(usage);
System.exit(1);
}
String dataAccessRoleArn = args[0];
String s3Uri = args[1];
String documentClassifierName = args[2];
Region region = Region.US_EAST_1;
ComprehendClient comClient = ComprehendClient.builder()
.region(region)
.build();
createDocumentClassifier(comClient, dataAccessRoleArn, s3Uri, documentClassifierName);
comClient.close();
}
public static void createDocumentClassifier(ComprehendClient comClient, String dataAccessRoleArn, String s3Uri,
String documentClassifierName) {
try {
DocumentClassifierInputDataConfig config = DocumentClassifierInputDataConfig.builder()
.s3Uri(s3Uri)
.build();
CreateDocumentClassifierRequest createDocumentClassifierRequest = CreateDocumentClassifierRequest.builder()
.documentClassifierName(documentClassifierName)
.dataAccessRoleArn(dataAccessRoleArn)
.languageCode("en")
.inputDataConfig(config)
.build();
CreateDocumentClassifierResponse createDocumentClassifierResult = comClient
.createDocumentClassifier(createDocumentClassifierRequest);
String documentClassifierArn = createDocumentClassifierResult.documentClassifierArn();
System.out.println("Document Classifier ARN: " + documentClassifierArn);
} catch (ComprehendException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ For API details, see [CreateDocumentClassifier](https://docs.aws.amazon.com/goto/SdkForJavaV2/comprehend-2017-11-27/CreateDocumentClassifier) in *AWS SDK for Java 2.x API Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def create(
self,
name,
language_code,
training_bucket,
training_key,
data_access_role_arn,
mode,
):
"""
Creates a custom classifier. After the classifier is created, it immediately
starts training on the data found in the specified Amazon S3 bucket. Training
can take 30 minutes or longer. The `describe_document_classifier` function
can be used to get training status and returns a status of TRAINED when the
classifier is ready to use.
:param name: The name of the classifier.
:param language_code: The language the classifier can operate on.
:param training_bucket: The Amazon S3 bucket that contains the training data.
:param training_key: The prefix used to find training data in the training
bucket. If multiple objects have the same prefix, all
of them are used.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
training bucket.
:return: The ARN of the newly created classifier.
"""
try:
response = self.comprehend_client.create_document_classifier(
DocumentClassifierName=name,
LanguageCode=language_code,
InputDataConfig={"S3Uri": f"s3://{training_bucket}/{training_key}"},
DataAccessRoleArn=data_access_role_arn,
Mode=mode.value,
)
self.classifier_arn = response["DocumentClassifierArn"]
logger.info("Started classifier creation. Arn is: %s.", self.classifier_arn)
except ClientError:
logger.exception("Couldn't create classifier %s.", name)
raise
else:
return self.classifier_arn
```
+ For API details, see [CreateDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/CreateDocumentClassifier) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->createdocumentclassifier(
iv_documentclassifiername = iv_classifier_name
iv_languagecode = iv_language_code
io_inputdataconfig = NEW /aws1/cl_cpddocclifierinpdat00(
iv_s3uri = iv_training_s3_uri
)
iv_dataaccessrolearn = iv_data_access_role_arn
iv_mode = iv_mode
).
MESSAGE 'Document classifier creation started.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdresrclimitexcdex.
MESSAGE 'Resource limit exceeded.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanytagsex.
MESSAGE 'Too many tags.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [CreateDocumentClassifier](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DeleteDocumentClassifier` with an AWS SDK or CLI
The following code examples show how to use `DeleteDocumentClassifier`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To delete a custom document classifier**
The following `delete-document-classifier` example deletes a custom document classifier model.
```
aws comprehend delete-document-classifier \
--document-classifier-arn arn:aws:comprehend:us-west-2:111122223333:document-classifier/example-classifier-1
```
This command produces no output.
For more information, see [Managing Amazon Comprehend endpoints](https://docs.aws.amazon.com/comprehend/latest/dg/manage-endpoints.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DeleteDocumentClassifier](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/delete-document-classifier.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def delete(self):
"""
Deletes the classifier.
"""
try:
self.comprehend_client.delete_document_classifier(
DocumentClassifierArn=self.classifier_arn
)
logger.info("Deleted classifier %s.", self.classifier_arn)
self.classifier_arn = None
except ClientError:
logger.exception("Couldn't deleted classifier %s.", self.classifier_arn)
raise
```
+ For API details, see [DeleteDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DeleteDocumentClassifier) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->deletedocumentclassifier(
iv_documentclassifierarn = iv_classifier_arn
).
MESSAGE 'Document classifier deleted.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdresourcenotfoundex.
MESSAGE 'Resource not found.' TYPE 'E'.
CATCH /aws1/cx_cpdresourceinuseex.
MESSAGE 'Resource in use.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DeleteDocumentClassifier](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DescribeDocumentClassificationJob` with an AWS SDK or CLI
The following code examples show how to use `DescribeDocumentClassificationJob`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To describe a document classification job**
The following `describe-document-classification-job` example gets the properties of an asynchronous document classification job.
```
aws comprehend describe-document-classification-job \
--job-id 123456abcdeb0e11022f22a11EXAMPLE
```
Output:
```
{
"DocumentClassificationJobProperties": {
"JobId": "123456abcdeb0e11022f22a11EXAMPLE",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:document-classification-job/123456abcdeb0e11022f22a11EXAMPLE",
"JobName": "exampleclassificationjob",
"JobStatus": "COMPLETED",
"SubmitTime": "2023-06-14T17:09:51.788000+00:00",
"EndTime": "2023-06-14T17:15:58.582000+00:00",
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:111122223333:document-classifier/mymodel/version/1",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket/jobdata/",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/testfolder/111122223333-CLN-123456abcdeb0e11022f22a11EXAMPLE/output/output.tar.gz"
},
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-servicerole"
}
}
```
For more information, see [Custom Classification](https://docs.aws.amazon.com/comprehend/latest/dg/how-document-classification.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DescribeDocumentClassificationJob](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/describe-document-classification-job.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def describe_job(self, job_id):
"""
Gets metadata about a classification job.
:param job_id: The ID of the job to look up.
:return: Metadata about the job.
"""
try:
response = self.comprehend_client.describe_document_classification_job(
JobId=job_id
)
job = response["DocumentClassificationJobProperties"]
logger.info("Got classification job %s.", job["JobName"])
except ClientError:
logger.exception("Couldn't get classification job %s.", job_id)
raise
else:
return job
```
+ For API details, see [DescribeDocumentClassificationJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeDocumentClassificationJob) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->describedocclassificationjob(
iv_jobid = iv_job_id
).
MESSAGE 'Document classification job described.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdjobnotfoundex.
MESSAGE 'Job not found.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DescribeDocumentClassificationJob](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DescribeDocumentClassifier` with an AWS SDK or CLI
The following code examples show how to use `DescribeDocumentClassifier`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To describe a document classifier**
The following `describe-document-classifier` example gets the properties of a custom document classifier model.
```
aws comprehend describe-document-classifier \
--document-classifier-arn arn:aws:comprehend:us-west-2:111122223333:document-classifier/example-classifier-1
```
Output:
```
{
"DocumentClassifierProperties": {
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:111122223333:document-classifier/example-classifier-1",
"LanguageCode": "en",
"Status": "TRAINED",
"SubmitTime": "2023-06-13T19:04:15.735000+00:00",
"EndTime": "2023-06-13T19:42:31.752000+00:00",
"TrainingStartTime": "2023-06-13T19:08:20.114000+00:00",
"TrainingEndTime": "2023-06-13T19:41:35.080000+00:00",
"InputDataConfig": {
"DataFormat": "COMPREHEND_CSV",
"S3Uri": "s3://amzn-s3-demo-bucket/trainingdata"
},
"OutputDataConfig": {},
"ClassifierMetadata": {
"NumberOfLabels": 3,
"NumberOfTrainedDocuments": 5016,
"NumberOfTestDocuments": 557,
"EvaluationMetrics": {
"Accuracy": 0.9856,
"Precision": 0.9919,
"Recall": 0.9459,
"F1Score": 0.9673,
"MicroPrecision": 0.9856,
"MicroRecall": 0.9856,
"MicroF1Score": 0.9856,
"HammingLoss": 0.0144
}
},
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-example-role",
"Mode": "MULTI_CLASS"
}
}
```
For more information, see [Creating and managing custom models](https://docs.aws.amazon.com/comprehend/latest/dg/manage-models.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DescribeDocumentClassifier](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/describe-document-classifier.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def describe(self, classifier_arn=None):
"""
Gets metadata about a custom classifier, including its current status.
:param classifier_arn: The ARN of the classifier to look up.
:return: Metadata about the classifier.
"""
if classifier_arn is not None:
self.classifier_arn = classifier_arn
try:
response = self.comprehend_client.describe_document_classifier(
DocumentClassifierArn=self.classifier_arn
)
classifier = response["DocumentClassifierProperties"]
logger.info("Got classifier %s.", self.classifier_arn)
except ClientError:
logger.exception("Couldn't get classifier %s.", self.classifier_arn)
raise
else:
return classifier
```
+ For API details, see [DescribeDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeDocumentClassifier) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->describedocumentclassifier(
iv_documentclassifierarn = iv_classifier_arn
).
MESSAGE 'Document classifier described.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdresourcenotfoundex.
MESSAGE 'Resource not found.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DescribeDocumentClassifier](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DescribeTopicsDetectionJob` with an AWS SDK or CLI
The following code examples show how to use `DescribeTopicsDetectionJob`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Run a topic modeling job on sample data](comprehend_example_comprehend_Usage_TopicModeler_section.md)
------
#### [ CLI ]
**AWS CLI**
**To describe a topics detection job**
The following `describe-topics-detection-job` example gets the properties of an asynchronous topics detection job.
```
aws comprehend describe-topics-detection-job \
--job-id 123456abcdeb0e11022f22a11EXAMPLE
```
Output:
```
{
"TopicsDetectionJobProperties": {
"JobId": "123456abcdeb0e11022f22a11EXAMPLE",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:topics-detection-job/123456abcdeb0e11022f22a11EXAMPLE",
"JobName": "example_topics_detection",
"JobStatus": "IN_PROGRESS",
"SubmitTime": "2023-06-09T18:44:43.414000+00:00",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/testfolder/111122223333-TOPICS-123456abcdeb0e11022f22a11EXAMPLE/output/output.tar.gz"
},
"NumberOfTopics": 10,
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-examplerole"
}
}
```
For more information, see [Async analysis for Amazon Comprehend insights](https://docs.aws.amazon.com/comprehend/latest/dg/api-async-insights.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DescribeTopicsDetectionJob](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/describe-topics-detection-job.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendTopicModeler:
"""Encapsulates a Comprehend topic modeler."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def describe_job(self, job_id):
"""
Gets metadata about a topic modeling job.
:param job_id: The ID of the job to look up.
:return: Metadata about the job.
"""
try:
response = self.comprehend_client.describe_topics_detection_job(
JobId=job_id
)
job = response["TopicsDetectionJobProperties"]
logger.info("Got topic detection job %s.", job_id)
except ClientError:
logger.exception("Couldn't get topic detection job %s.", job_id)
raise
else:
return job
```
+ For API details, see [DescribeTopicsDetectionJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeTopicsDetectionJob) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->describetopicsdetectionjob(
iv_jobid = iv_job_id
).
MESSAGE 'Topics detection job described.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdjobnotfoundex.
MESSAGE 'Job not found.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DescribeTopicsDetectionJob](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DetectDominantLanguage` with an AWS SDK or CLI
The following code examples show how to use `DetectDominantLanguage`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example calls the Amazon Comprehend service to determine the
/// dominant language.
///
public static class DetectDominantLanguage
{
///
/// Calls Amazon Comprehend to determine the dominant language used in
/// the sample text.
///
public static async Task Main()
{
string text = "It is raining today in Seattle.";
var comprehendClient = new AmazonComprehendClient(Amazon.RegionEndpoint.USWest2);
Console.WriteLine("Calling DetectDominantLanguage\n");
var detectDominantLanguageRequest = new DetectDominantLanguageRequest()
{
Text = text,
};
var detectDominantLanguageResponse = await comprehendClient.DetectDominantLanguageAsync(detectDominantLanguageRequest);
foreach (var dl in detectDominantLanguageResponse.Languages)
{
Console.WriteLine($"Language Code: {dl.LanguageCode}, Score: {dl.Score}");
}
Console.WriteLine("Done");
}
}
```
+ For API details, see [DetectDominantLanguage](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/DetectDominantLanguage) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To detect the dominant language of input text**
The following `detect-dominant-language` analyzes the input text and identifies the dominant language. The pre-trained model's confidence score is also output.
```
aws comprehend detect-dominant-language \
--text "It is a beautiful day in Seattle."
```
Output:
```
{
"Languages": [
{
"LanguageCode": "en",
"Score": 0.9877256155014038
}
]
}
```
For more information, see [Dominant Language](https://docs.aws.amazon.com/comprehend/latest/dg/how-languages.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DetectDominantLanguage](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/detect-dominant-language.html) in *AWS CLI Command Reference*.
------
#### [ Java ]
**SDK for Java 2.x**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/comprehend#code-examples).
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.comprehend.ComprehendClient;
import software.amazon.awssdk.services.comprehend.model.ComprehendException;
import software.amazon.awssdk.services.comprehend.model.DetectDominantLanguageRequest;
import software.amazon.awssdk.services.comprehend.model.DetectDominantLanguageResponse;
import software.amazon.awssdk.services.comprehend.model.DominantLanguage;
import java.util.List;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DetectLanguage {
public static void main(String[] args) {
// Specify French text - "It is raining today in Seattle".
String text = "Il pleut aujourd'hui à Seattle";
Region region = Region.US_EAST_1;
ComprehendClient comClient = ComprehendClient.builder()
.region(region)
.build();
System.out.println("Calling DetectDominantLanguage");
detectTheDominantLanguage(comClient, text);
comClient.close();
}
public static void detectTheDominantLanguage(ComprehendClient comClient, String text) {
try {
DetectDominantLanguageRequest request = DetectDominantLanguageRequest.builder()
.text(text)
.build();
DetectDominantLanguageResponse resp = comClient.detectDominantLanguage(request);
List allLanList = resp.languages();
for (DominantLanguage lang : allLanList) {
System.out.println("Language is " + lang.languageCode());
}
} catch (ComprehendException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ For API details, see [DetectDominantLanguage](https://docs.aws.amazon.com/goto/SdkForJavaV2/comprehend-2017-11-27/DetectDominantLanguage) in *AWS SDK for Java 2.x API Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_languages(self, text):
"""
Detects languages used in a document.
:param text: The document to inspect.
:return: The list of languages along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_dominant_language(Text=text)
languages = response["Languages"]
logger.info("Detected %s languages.", len(languages))
except ClientError:
logger.exception("Couldn't detect languages.")
raise
else:
return languages
```
+ For API details, see [DetectDominantLanguage](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectDominantLanguage) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->detectdominantlanguage( iv_text = iv_text ).
MESSAGE 'Languages detected.' TYPE 'I'.
CATCH /aws1/cx_cpdtextsizelmtexcdex.
MESSAGE 'Text size exceeds limit.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DetectDominantLanguage](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DetectEntities` with an AWS SDK or CLI
The following code examples show how to use `DetectEntities`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example shows how to use the AmazonComprehend service detect any
/// entities in submitted text.
///
public static class DetectEntities
{
///
/// The main method calls the DetectEntitiesAsync method to find any
/// entities in the sample code.
///
public static async Task Main()
{
string text = "It is raining today in Seattle";
var comprehendClient = new AmazonComprehendClient();
Console.WriteLine("Calling DetectEntities\n");
var detectEntitiesRequest = new DetectEntitiesRequest()
{
Text = text,
LanguageCode = "en",
};
var detectEntitiesResponse = await comprehendClient.DetectEntitiesAsync(detectEntitiesRequest);
foreach (var e in detectEntitiesResponse.Entities)
{
Console.WriteLine($"Text: {e.Text}, Type: {e.Type}, Score: {e.Score}, BeginOffset: {e.BeginOffset}, EndOffset: {e.EndOffset}");
}
Console.WriteLine("Done");
}
}
```
+ For API details, see [DetectEntities](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/DetectEntities) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To detect named entities in input text**
The following `detect-entities` example analyzes the input text and returns the named entities. The pre-trained model's confidence score is also output for each prediction.
```
aws comprehend detect-entities \
--language-code en \
--text "Hello Zhang Wei, I am John. Your AnyCompany Financial Services, LLC credit card \
account 1111-XXXX-1111-XXXX has a minimum payment of $24.53 that is due by July 31st. Based on your autopay settings, \
we will withdraw your payment on the due date from your bank account number XXXXXX1111 with the routing number XXXXX0000. \
Customer feedback for Sunshine Spa, 123 Main St, Anywhere. Send comments to Alice at AnySpa@example.com."
```
Output:
```
{
"Entities": [
{
"Score": 0.9994556307792664,
"Type": "PERSON",
"Text": "Zhang Wei",
"BeginOffset": 6,
"EndOffset": 15
},
{
"Score": 0.9981022477149963,
"Type": "PERSON",
"Text": "John",
"BeginOffset": 22,
"EndOffset": 26
},
{
"Score": 0.9986887574195862,
"Type": "ORGANIZATION",
"Text": "AnyCompany Financial Services, LLC",
"BeginOffset": 33,
"EndOffset": 67
},
{
"Score": 0.9959119558334351,
"Type": "OTHER",
"Text": "1111-XXXX-1111-XXXX",
"BeginOffset": 88,
"EndOffset": 107
},
{
"Score": 0.9708039164543152,
"Type": "QUANTITY",
"Text": ".53",
"BeginOffset": 133,
"EndOffset": 136
},
{
"Score": 0.9987268447875977,
"Type": "DATE",
"Text": "July 31st",
"BeginOffset": 152,
"EndOffset": 161
},
{
"Score": 0.9858865737915039,
"Type": "OTHER",
"Text": "XXXXXX1111",
"BeginOffset": 271,
"EndOffset": 281
},
{
"Score": 0.9700471758842468,
"Type": "OTHER",
"Text": "XXXXX0000",
"BeginOffset": 306,
"EndOffset": 315
},
{
"Score": 0.9591118693351746,
"Type": "ORGANIZATION",
"Text": "Sunshine Spa",
"BeginOffset": 340,
"EndOffset": 352
},
{
"Score": 0.9797496795654297,
"Type": "LOCATION",
"Text": "123 Main St",
"BeginOffset": 354,
"EndOffset": 365
},
{
"Score": 0.994929313659668,
"Type": "PERSON",
"Text": "Alice",
"BeginOffset": 394,
"EndOffset": 399
},
{
"Score": 0.9949769377708435,
"Type": "OTHER",
"Text": "AnySpa@example.com",
"BeginOffset": 403,
"EndOffset": 418
}
]
}
```
For more information, see [Entities](https://docs.aws.amazon.com/comprehend/latest/dg/how-entities.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DetectEntities](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/detect-entities.html) in *AWS CLI Command Reference*.
------
#### [ Java ]
**SDK for Java 2.x**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/comprehend#code-examples).
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.comprehend.ComprehendClient;
import software.amazon.awssdk.services.comprehend.model.DetectEntitiesRequest;
import software.amazon.awssdk.services.comprehend.model.DetectEntitiesResponse;
import software.amazon.awssdk.services.comprehend.model.Entity;
import software.amazon.awssdk.services.comprehend.model.ComprehendException;
import java.util.List;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DetectEntities {
public static void main(String[] args) {
String text = "Amazon.com, Inc. is located in Seattle, WA and was founded July 5th, 1994 by Jeff Bezos, allowing customers to buy everything from books to blenders. Seattle is north of Portland and south of Vancouver, BC. Other notable Seattle - based companies are Starbucks and Boeing.";
Region region = Region.US_EAST_1;
ComprehendClient comClient = ComprehendClient.builder()
.region(region)
.build();
System.out.println("Calling DetectEntities");
detectAllEntities(comClient, text);
comClient.close();
}
public static void detectAllEntities(ComprehendClient comClient, String text) {
try {
DetectEntitiesRequest detectEntitiesRequest = DetectEntitiesRequest.builder()
.text(text)
.languageCode("en")
.build();
DetectEntitiesResponse detectEntitiesResult = comClient.detectEntities(detectEntitiesRequest);
List entList = detectEntitiesResult.entities();
for (Entity entity : entList) {
System.out.println("Entity text is " + entity.text());
}
} catch (ComprehendException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ For API details, see [DetectEntities](https://docs.aws.amazon.com/goto/SdkForJavaV2/comprehend-2017-11-27/DetectEntities) in *AWS SDK for Java 2.x API Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_entities(self, text, language_code):
"""
Detects entities in a document. Entities can be things like people and places
or other common terms.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of entities along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_entities(
Text=text, LanguageCode=language_code
)
entities = response["Entities"]
logger.info("Detected %s entities.", len(entities))
except ClientError:
logger.exception("Couldn't detect entities.")
raise
else:
return entities
```
+ For API details, see [DetectEntities](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectEntities) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->detectentities(
iv_text = iv_text
iv_languagecode = iv_language_code
).
MESSAGE 'Entities detected.' TYPE 'I'.
CATCH /aws1/cx_cpdtextsizelmtexcdex.
MESSAGE 'Text size exceeds limit.' TYPE 'E'.
CATCH /aws1/cx_cpdunsuppedlanguageex.
MESSAGE 'Unsupported language.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DetectEntities](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DetectKeyPhrases` with an AWS SDK or CLI
The following code examples show how to use `DetectKeyPhrases`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example shows how to use the Amazon Comprehend service to
/// search text for key phrases.
///
public static class DetectKeyPhrase
{
///
/// This method calls the Amazon Comprehend method DetectKeyPhrasesAsync
/// to detect any key phrases in the sample text.
///
public static async Task Main()
{
string text = "It is raining today in Seattle";
var comprehendClient = new AmazonComprehendClient(Amazon.RegionEndpoint.USWest2);
// Call DetectKeyPhrases API
Console.WriteLine("Calling DetectKeyPhrases");
var detectKeyPhrasesRequest = new DetectKeyPhrasesRequest()
{
Text = text,
LanguageCode = "en",
};
var detectKeyPhrasesResponse = await comprehendClient.DetectKeyPhrasesAsync(detectKeyPhrasesRequest);
foreach (var kp in detectKeyPhrasesResponse.KeyPhrases)
{
Console.WriteLine($"Text: {kp.Text}, Score: {kp.Score}, BeginOffset: {kp.BeginOffset}, EndOffset: {kp.EndOffset}");
}
Console.WriteLine("Done");
}
}
```
+ For API details, see [DetectKeyPhrases](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/DetectKeyPhrases) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To detect key phrases in input text**
The following `detect-key-phrases` example analyzes the input text and identifies the key noun phrases. The pre-trained model's confidence score is also output for each prediction.
```
aws comprehend detect-key-phrases \
--language-code en \
--text "Hello Zhang Wei, I am John. Your AnyCompany Financial Services, LLC credit card \
account 1111-XXXX-1111-XXXX has a minimum payment of $24.53 that is due by July 31st. Based on your autopay settings, \
we will withdraw your payment on the due date from your bank account number XXXXXX1111 with the routing number XXXXX0000. \
Customer feedback for Sunshine Spa, 123 Main St, Anywhere. Send comments to Alice at AnySpa@example.com."
```
Output:
```
{
"KeyPhrases": [
{
"Score": 0.8996376395225525,
"Text": "Zhang Wei",
"BeginOffset": 6,
"EndOffset": 15
},
{
"Score": 0.9992469549179077,
"Text": "John",
"BeginOffset": 22,
"EndOffset": 26
},
{
"Score": 0.988385021686554,
"Text": "Your AnyCompany Financial Services",
"BeginOffset": 28,
"EndOffset": 62
},
{
"Score": 0.8740853071212769,
"Text": "LLC credit card account 1111-XXXX-1111-XXXX",
"BeginOffset": 64,
"EndOffset": 107
},
{
"Score": 0.9999437928199768,
"Text": "a minimum payment",
"BeginOffset": 112,
"EndOffset": 129
},
{
"Score": 0.9998900890350342,
"Text": ".53",
"BeginOffset": 133,
"EndOffset": 136
},
{
"Score": 0.9979453086853027,
"Text": "July 31st",
"BeginOffset": 152,
"EndOffset": 161
},
{
"Score": 0.9983011484146118,
"Text": "your autopay settings",
"BeginOffset": 172,
"EndOffset": 193
},
{
"Score": 0.9996572136878967,
"Text": "your payment",
"BeginOffset": 211,
"EndOffset": 223
},
{
"Score": 0.9995037317276001,
"Text": "the due date",
"BeginOffset": 227,
"EndOffset": 239
},
{
"Score": 0.9702621698379517,
"Text": "your bank account number XXXXXX1111",
"BeginOffset": 245,
"EndOffset": 280
},
{
"Score": 0.9179925918579102,
"Text": "the routing number XXXXX0000.Customer feedback",
"BeginOffset": 286,
"EndOffset": 332
},
{
"Score": 0.9978160858154297,
"Text": "Sunshine Spa",
"BeginOffset": 337,
"EndOffset": 349
},
{
"Score": 0.9706913232803345,
"Text": "123 Main St",
"BeginOffset": 351,
"EndOffset": 362
},
{
"Score": 0.9941995143890381,
"Text": "comments",
"BeginOffset": 379,
"EndOffset": 387
},
{
"Score": 0.9759287238121033,
"Text": "Alice",
"BeginOffset": 391,
"EndOffset": 396
},
{
"Score": 0.8376792669296265,
"Text": "AnySpa@example.com",
"BeginOffset": 400,
"EndOffset": 415
}
]
}
```
For more information, see [Key Phrases](https://docs.aws.amazon.com/comprehend/latest/dg/how-key-phrases.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DetectKeyPhrases](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/detect-key-phrases.html) in *AWS CLI Command Reference*.
------
#### [ Java ]
**SDK for Java 2.x**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/comprehend#code-examples).
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.comprehend.ComprehendClient;
import software.amazon.awssdk.services.comprehend.model.DetectKeyPhrasesRequest;
import software.amazon.awssdk.services.comprehend.model.DetectKeyPhrasesResponse;
import software.amazon.awssdk.services.comprehend.model.KeyPhrase;
import software.amazon.awssdk.services.comprehend.model.ComprehendException;
import java.util.List;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DetectKeyPhrases {
public static void main(String[] args) {
String text = "Amazon.com, Inc. is located in Seattle, WA and was founded July 5th, 1994 by Jeff Bezos, allowing customers to buy everything from books to blenders. Seattle is north of Portland and south of Vancouver, BC. Other notable Seattle - based companies are Starbucks and Boeing.";
Region region = Region.US_EAST_1;
ComprehendClient comClient = ComprehendClient.builder()
.region(region)
.build();
System.out.println("Calling DetectKeyPhrases");
detectAllKeyPhrases(comClient, text);
comClient.close();
}
public static void detectAllKeyPhrases(ComprehendClient comClient, String text) {
try {
DetectKeyPhrasesRequest detectKeyPhrasesRequest = DetectKeyPhrasesRequest.builder()
.text(text)
.languageCode("en")
.build();
DetectKeyPhrasesResponse detectKeyPhrasesResult = comClient.detectKeyPhrases(detectKeyPhrasesRequest);
List phraseList = detectKeyPhrasesResult.keyPhrases();
for (KeyPhrase keyPhrase : phraseList) {
System.out.println("Key phrase text is " + keyPhrase.text());
}
} catch (ComprehendException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ For API details, see [DetectKeyPhrases](https://docs.aws.amazon.com/goto/SdkForJavaV2/comprehend-2017-11-27/DetectKeyPhrases) in *AWS SDK for Java 2.x API Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_key_phrases(self, text, language_code):
"""
Detects key phrases in a document. A key phrase is typically a noun and its
modifiers.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of key phrases along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_key_phrases(
Text=text, LanguageCode=language_code
)
phrases = response["KeyPhrases"]
logger.info("Detected %s phrases.", len(phrases))
except ClientError:
logger.exception("Couldn't detect phrases.")
raise
else:
return phrases
```
+ For API details, see [DetectKeyPhrases](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectKeyPhrases) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->detectkeyphrases(
iv_text = iv_text
iv_languagecode = iv_language_code
).
MESSAGE 'Key phrases detected.' TYPE 'I'.
CATCH /aws1/cx_cpdtextsizelmtexcdex.
MESSAGE 'Text size exceeds limit.' TYPE 'E'.
CATCH /aws1/cx_cpdunsuppedlanguageex.
MESSAGE 'Unsupported language.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DetectKeyPhrases](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DetectPiiEntities` with an AWS SDK or CLI
The following code examples show how to use `DetectPiiEntities`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example shows how to use the Amazon Comprehend service to find
/// personally identifiable information (PII) within text submitted to the
/// DetectPiiEntitiesAsync method.
///
public class DetectingPII
{
///
/// This method calls the DetectPiiEntitiesAsync method to locate any
/// personally dientifiable information within the supplied text.
///
public static async Task Main()
{
var comprehendClient = new AmazonComprehendClient();
var text = @"Hello Paul Santos. The latest statement for your
credit card account 1111-0000-1111-0000 was
mailed to 123 Any Street, Seattle, WA 98109.";
var request = new DetectPiiEntitiesRequest
{
Text = text,
LanguageCode = "EN",
};
var response = await comprehendClient.DetectPiiEntitiesAsync(request);
if (response.Entities.Count > 0)
{
foreach (var entity in response.Entities)
{
var entityValue = text.Substring(entity.BeginOffset, entity.EndOffset - entity.BeginOffset);
Console.WriteLine($"{entity.Type}: {entityValue}");
}
}
}
}
```
+ For API details, see [DetectPiiEntities](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/DetectPiiEntities) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To detect pii entities in input text**
The following `detect-pii-entities` example analyzes the input text and identifies entities that contain personally identifiable information (PII). The pre-trained model's confidence score is also output for each prediction.
```
aws comprehend detect-pii-entities \
--language-code en \
--text "Hello Zhang Wei, I am John. Your AnyCompany Financial Services, LLC credit card \
account 1111-XXXX-1111-XXXX has a minimum payment of $24.53 that is due by July 31st. Based on your autopay settings, \
we will withdraw your payment on the due date from your bank account number XXXXXX1111 with the routing number XXXXX0000. \
Customer feedback for Sunshine Spa, 123 Main St, Anywhere. Send comments to Alice at AnySpa@example.com."
```
Output:
```
{
"Entities": [
{
"Score": 0.9998322129249573,
"Type": "NAME",
"BeginOffset": 6,
"EndOffset": 15
},
{
"Score": 0.9998878240585327,
"Type": "NAME",
"BeginOffset": 22,
"EndOffset": 26
},
{
"Score": 0.9994089603424072,
"Type": "CREDIT_DEBIT_NUMBER",
"BeginOffset": 88,
"EndOffset": 107
},
{
"Score": 0.9999760985374451,
"Type": "DATE_TIME",
"BeginOffset": 152,
"EndOffset": 161
},
{
"Score": 0.9999449253082275,
"Type": "BANK_ACCOUNT_NUMBER",
"BeginOffset": 271,
"EndOffset": 281
},
{
"Score": 0.9999847412109375,
"Type": "BANK_ROUTING",
"BeginOffset": 306,
"EndOffset": 315
},
{
"Score": 0.999925434589386,
"Type": "ADDRESS",
"BeginOffset": 354,
"EndOffset": 365
},
{
"Score": 0.9989161491394043,
"Type": "NAME",
"BeginOffset": 394,
"EndOffset": 399
},
{
"Score": 0.9994171857833862,
"Type": "EMAIL",
"BeginOffset": 403,
"EndOffset": 418
}
]
}
```
For more information, see [Personally Identifiable Information (PII)](https://docs.aws.amazon.com/comprehend/latest/dg/pii.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DetectPiiEntities](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/detect-pii-entities.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_pii(self, text, language_code):
"""
Detects personally identifiable information (PII) in a document. PII can be
things like names, account numbers, or addresses.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of PII entities along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_pii_entities(
Text=text, LanguageCode=language_code
)
entities = response["Entities"]
logger.info("Detected %s PII entities.", len(entities))
except ClientError:
logger.exception("Couldn't detect PII entities.")
raise
else:
return entities
```
+ For API details, see [DetectPiiEntities](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectPiiEntities) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->detectpiientities(
iv_text = iv_text
iv_languagecode = iv_language_code
).
MESSAGE 'PII entities detected.' TYPE 'I'.
CATCH /aws1/cx_cpdtextsizelmtexcdex.
MESSAGE 'Text size exceeds limit.' TYPE 'E'.
CATCH /aws1/cx_cpdunsuppedlanguageex.
MESSAGE 'Unsupported language.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DetectPiiEntities](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DetectSentiment` with an AWS SDK or CLI
The following code examples show how to use `DetectSentiment`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example shows how to detect the overall sentiment of the supplied
/// text using the Amazon Comprehend service.
///
public static class DetectSentiment
{
///
/// This method calls the DetetectSentimentAsync method to analyze the
/// supplied text and determine the overal sentiment.
///
public static async Task Main()
{
string text = "It is raining today in Seattle";
var comprehendClient = new AmazonComprehendClient(Amazon.RegionEndpoint.USWest2);
// Call DetectKeyPhrases API
Console.WriteLine("Calling DetectSentiment");
var detectSentimentRequest = new DetectSentimentRequest()
{
Text = text,
LanguageCode = "en",
};
var detectSentimentResponse = await comprehendClient.DetectSentimentAsync(detectSentimentRequest);
Console.WriteLine($"Sentiment: {detectSentimentResponse.Sentiment}");
Console.WriteLine("Done");
}
}
```
+ For API details, see [DetectSentiment](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/DetectSentiment) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To detect the sentiment of an input text**
The following `detect-sentiment` example analyzes the input text and returns an inference of the prevailing sentiment (`POSITIVE`, `NEUTRAL`, `MIXED`, or `NEGATIVE`).
```
aws comprehend detect-sentiment \
--language-code en \
--text "It is a beautiful day in Seattle"
```
Output:
```
{
"Sentiment": "POSITIVE",
"SentimentScore": {
"Positive": 0.9976957440376282,
"Negative": 9.653854067437351e-05,
"Neutral": 0.002169104292988777,
"Mixed": 3.857641786453314e-05
}
}
```
For more information, see [Sentiment](https://docs.aws.amazon.com/comprehend/latest/dg/how-sentiment.html) in the *Amazon Comprehend Developer Guide*
+ For API details, see [DetectSentiment](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/detect-sentiment.html) in *AWS CLI Command Reference*.
------
#### [ Java ]
**SDK for Java 2.x**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/comprehend#code-examples).
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.comprehend.ComprehendClient;
import software.amazon.awssdk.services.comprehend.model.ComprehendException;
import software.amazon.awssdk.services.comprehend.model.DetectSentimentRequest;
import software.amazon.awssdk.services.comprehend.model.DetectSentimentResponse;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DetectSentiment {
public static void main(String[] args) {
String text = "Amazon.com, Inc. is located in Seattle, WA and was founded July 5th, 1994 by Jeff Bezos, allowing customers to buy everything from books to blenders. Seattle is north of Portland and south of Vancouver, BC. Other notable Seattle - based companies are Starbucks and Boeing.";
Region region = Region.US_EAST_1;
ComprehendClient comClient = ComprehendClient.builder()
.region(region)
.build();
System.out.println("Calling DetectSentiment");
detectSentiments(comClient, text);
comClient.close();
}
public static void detectSentiments(ComprehendClient comClient, String text) {
try {
DetectSentimentRequest detectSentimentRequest = DetectSentimentRequest.builder()
.text(text)
.languageCode("en")
.build();
DetectSentimentResponse detectSentimentResult = comClient.detectSentiment(detectSentimentRequest);
System.out.println("The Neutral value is " + detectSentimentResult.sentimentScore().neutral());
} catch (ComprehendException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ For API details, see [DetectSentiment](https://docs.aws.amazon.com/goto/SdkForJavaV2/comprehend-2017-11-27/DetectSentiment) in *AWS SDK for Java 2.x API Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_sentiment(self, text, language_code):
"""
Detects the overall sentiment expressed in a document. Sentiment can
be positive, negative, neutral, or a mixture.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The sentiments along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_sentiment(
Text=text, LanguageCode=language_code
)
logger.info("Detected primary sentiment %s.", response["Sentiment"])
except ClientError:
logger.exception("Couldn't detect sentiment.")
raise
else:
return response
```
+ For API details, see [DetectSentiment](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectSentiment) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->detectsentiment(
iv_text = iv_text
iv_languagecode = iv_language_code
).
MESSAGE 'Sentiment detected.' TYPE 'I'.
CATCH /aws1/cx_cpdtextsizelmtexcdex.
MESSAGE 'Text size exceeds limit.' TYPE 'E'.
CATCH /aws1/cx_cpdunsuppedlanguageex.
MESSAGE 'Unsupported language.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DetectSentiment](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `DetectSyntax` with an AWS SDK or CLI
The following code examples show how to use `DetectSyntax`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example shows how to use Amazon Comprehend to detect syntax
/// elements by calling the DetectSyntaxAsync method.
///
public class DetectingSyntax
{
///
/// This method calls DetectSynaxAsync to identify the syntax elements
/// in the sample text.
///
public static async Task Main()
{
string text = "It is raining today in Seattle";
var comprehendClient = new AmazonComprehendClient();
// Call DetectSyntax API
Console.WriteLine("Calling DetectSyntaxAsync\n");
var detectSyntaxRequest = new DetectSyntaxRequest()
{
Text = text,
LanguageCode = "en",
};
DetectSyntaxResponse detectSyntaxResponse = await comprehendClient.DetectSyntaxAsync(detectSyntaxRequest);
foreach (SyntaxToken s in detectSyntaxResponse.SyntaxTokens)
{
Console.WriteLine($"Text: {s.Text}, PartOfSpeech: {s.PartOfSpeech.Tag}, BeginOffset: {s.BeginOffset}, EndOffset: {s.EndOffset}");
}
Console.WriteLine("Done");
}
}
```
+ For API details, see [DetectSyntax](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/DetectSyntax) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To detect the parts of speech in an input text**
The following `detect-syntax` example analyzes the syntax of the input text and returns the different parts of speech. The pre-trained model's confidence score is also output for each prediction.
```
aws comprehend detect-syntax \
--language-code en \
--text "It is a beautiful day in Seattle."
```
Output:
```
{
"SyntaxTokens": [
{
"TokenId": 1,
"Text": "It",
"BeginOffset": 0,
"EndOffset": 2,
"PartOfSpeech": {
"Tag": "PRON",
"Score": 0.9999740719795227
}
},
{
"TokenId": 2,
"Text": "is",
"BeginOffset": 3,
"EndOffset": 5,
"PartOfSpeech": {
"Tag": "VERB",
"Score": 0.999901294708252
}
},
{
"TokenId": 3,
"Text": "a",
"BeginOffset": 6,
"EndOffset": 7,
"PartOfSpeech": {
"Tag": "DET",
"Score": 0.9999938607215881
}
},
{
"TokenId": 4,
"Text": "beautiful",
"BeginOffset": 8,
"EndOffset": 17,
"PartOfSpeech": {
"Tag": "ADJ",
"Score": 0.9987351894378662
}
},
{
"TokenId": 5,
"Text": "day",
"BeginOffset": 18,
"EndOffset": 21,
"PartOfSpeech": {
"Tag": "NOUN",
"Score": 0.9999796748161316
}
},
{
"TokenId": 6,
"Text": "in",
"BeginOffset": 22,
"EndOffset": 24,
"PartOfSpeech": {
"Tag": "ADP",
"Score": 0.9998047947883606
}
},
{
"TokenId": 7,
"Text": "Seattle",
"BeginOffset": 25,
"EndOffset": 32,
"PartOfSpeech": {
"Tag": "PROPN",
"Score": 0.9940530061721802
}
}
]
}
```
For more information, see [Syntax Analysis](https://docs.aws.amazon.com/comprehend/latest/dg/how-syntax.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [DetectSyntax](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/detect-syntax.html) in *AWS CLI Command Reference*.
------
#### [ Java ]
**SDK for Java 2.x**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/comprehend#code-examples).
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.comprehend.ComprehendClient;
import software.amazon.awssdk.services.comprehend.model.ComprehendException;
import software.amazon.awssdk.services.comprehend.model.DetectSyntaxRequest;
import software.amazon.awssdk.services.comprehend.model.DetectSyntaxResponse;
import software.amazon.awssdk.services.comprehend.model.SyntaxToken;
import java.util.List;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DetectSyntax {
public static void main(String[] args) {
String text = "Amazon.com, Inc. is located in Seattle, WA and was founded July 5th, 1994 by Jeff Bezos, allowing customers to buy everything from books to blenders. Seattle is north of Portland and south of Vancouver, BC. Other notable Seattle - based companies are Starbucks and Boeing.";
Region region = Region.US_EAST_1;
ComprehendClient comClient = ComprehendClient.builder()
.region(region)
.build();
System.out.println("Calling DetectSyntax");
detectAllSyntax(comClient, text);
comClient.close();
}
public static void detectAllSyntax(ComprehendClient comClient, String text) {
try {
DetectSyntaxRequest detectSyntaxRequest = DetectSyntaxRequest.builder()
.text(text)
.languageCode("en")
.build();
DetectSyntaxResponse detectSyntaxResult = comClient.detectSyntax(detectSyntaxRequest);
List syntaxTokens = detectSyntaxResult.syntaxTokens();
for (SyntaxToken token : syntaxTokens) {
System.out.println("Language is " + token.text());
System.out.println("Part of speech is " + token.partOfSpeech().tagAsString());
}
} catch (ComprehendException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ For API details, see [DetectSyntax](https://docs.aws.amazon.com/goto/SdkForJavaV2/comprehend-2017-11-27/DetectSyntax) in *AWS SDK for Java 2.x API Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_syntax(self, text, language_code):
"""
Detects syntactical elements of a document. Syntax tokens are portions of
text along with their use as parts of speech, such as nouns, verbs, and
interjections.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of syntax tokens along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_syntax(
Text=text, LanguageCode=language_code
)
tokens = response["SyntaxTokens"]
logger.info("Detected %s syntax tokens.", len(tokens))
except ClientError:
logger.exception("Couldn't detect syntax.")
raise
else:
return tokens
```
+ For API details, see [DetectSyntax](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectSyntax) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->detectsyntax(
iv_text = iv_text
iv_languagecode = iv_language_code
).
MESSAGE 'Syntax tokens detected.' TYPE 'I'.
CATCH /aws1/cx_cpdtextsizelmtexcdex.
MESSAGE 'Text size exceeds limit.' TYPE 'E'.
CATCH /aws1/cx_cpdunsuppedlanguageex.
MESSAGE 'Unsupported language.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [DetectSyntax](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `ListDocumentClassificationJobs` with an AWS SDK or CLI
The following code examples show how to use `ListDocumentClassificationJobs`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To list of all document classification jobs**
The following `list-document-classification-jobs` example lists all document classification jobs.
```
aws comprehend list-document-classification-jobs
```
Output:
```
{
"DocumentClassificationJobPropertiesList": [
{
"JobId": "123456abcdeb0e11022f22a11EXAMPLE",
"JobArn": "arn:aws:comprehend:us-west-2:1234567890101:document-classification-job/123456abcdeb0e11022f22a11EXAMPLE",
"JobName": "exampleclassificationjob",
"JobStatus": "COMPLETED",
"SubmitTime": "2023-06-14T17:09:51.788000+00:00",
"EndTime": "2023-06-14T17:15:58.582000+00:00",
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:1234567890101:document-classifier/mymodel/version/12",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket/jobdata/",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/thefolder/1234567890101-CLN-e758dd56b824aa717ceab551f11749fb/output/output.tar.gz"
},
"DataAccessRoleArn": "arn:aws:iam::1234567890101:role/service-role/AmazonComprehendServiceRole-example-role"
},
{
"JobId": "123456abcdeb0e11022f22a1EXAMPLE2",
"JobArn": "arn:aws:comprehend:us-west-2:1234567890101:document-classification-job/123456abcdeb0e11022f22a1EXAMPLE2",
"JobName": "exampleclassificationjob2",
"JobStatus": "COMPLETED",
"SubmitTime": "2023-06-14T17:22:39.829000+00:00",
"EndTime": "2023-06-14T17:28:46.107000+00:00",
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:1234567890101:document-classifier/mymodel/version/12",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket/jobdata/",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/thefolder/1234567890101-CLN-123456abcdeb0e11022f22a1EXAMPLE2/output/output.tar.gz"
},
"DataAccessRoleArn": "arn:aws:iam::1234567890101:role/service-role/AmazonComprehendServiceRole-example-role"
}
]
}
```
For more information, see [Custom Classification](https://docs.aws.amazon.com/comprehend/latest/dg/how-document-classification.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [ListDocumentClassificationJobs](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/list-document-classification-jobs.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def list_jobs(self):
"""
Lists the classification jobs for the current account.
:return: The list of jobs.
"""
try:
response = self.comprehend_client.list_document_classification_jobs()
jobs = response["DocumentClassificationJobPropertiesList"]
logger.info("Got %s document classification jobs.", len(jobs))
except ClientError:
logger.exception(
"Couldn't get document classification jobs.",
)
raise
else:
return jobs
```
+ For API details, see [ListDocumentClassificationJobs](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListDocumentClassificationJobs) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->listdocclassificationjobs( ).
MESSAGE 'Document classification jobs listed.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidfilterex.
MESSAGE 'Invalid filter.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [ListDocumentClassificationJobs](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `ListDocumentClassifiers` with an AWS SDK or CLI
The following code examples show how to use `ListDocumentClassifiers`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To list of all document classifiers**
The following `list-document-classifiers` example lists all trained and in-training document classifier models.
```
aws comprehend list-document-classifiers
```
Output:
```
{
"DocumentClassifierPropertiesList": [
{
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:111122223333:document-classifier/exampleclassifier1",
"LanguageCode": "en",
"Status": "TRAINED",
"SubmitTime": "2023-06-13T19:04:15.735000+00:00",
"EndTime": "2023-06-13T19:42:31.752000+00:00",
"TrainingStartTime": "2023-06-13T19:08:20.114000+00:00",
"TrainingEndTime": "2023-06-13T19:41:35.080000+00:00",
"InputDataConfig": {
"DataFormat": "COMPREHEND_CSV",
"S3Uri": "s3://amzn-s3-demo-bucket/trainingdata"
},
"OutputDataConfig": {},
"ClassifierMetadata": {
"NumberOfLabels": 3,
"NumberOfTrainedDocuments": 5016,
"NumberOfTestDocuments": 557,
"EvaluationMetrics": {
"Accuracy": 0.9856,
"Precision": 0.9919,
"Recall": 0.9459,
"F1Score": 0.9673,
"MicroPrecision": 0.9856,
"MicroRecall": 0.9856,
"MicroF1Score": 0.9856,
"HammingLoss": 0.0144
}
},
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-testorle",
"Mode": "MULTI_CLASS"
},
{
"DocumentClassifierArn": "arn:aws:comprehend:us-west-2:111122223333:document-classifier/exampleclassifier2",
"LanguageCode": "en",
"Status": "TRAINING",
"SubmitTime": "2023-06-13T21:20:28.690000+00:00",
"InputDataConfig": {
"DataFormat": "COMPREHEND_CSV",
"S3Uri": "s3://amzn-s3-demo-bucket/trainingdata"
},
"OutputDataConfig": {},
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-testorle",
"Mode": "MULTI_CLASS"
}
]
}
```
For more information, see [Creating and managing custom models](https://docs.aws.amazon.com/comprehend/latest/dg/manage-models.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [ListDocumentClassifiers](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/list-document-classifiers.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def list(self):
"""
Lists custom classifiers for the current account.
:return: The list of classifiers.
"""
try:
response = self.comprehend_client.list_document_classifiers()
classifiers = response["DocumentClassifierPropertiesList"]
logger.info("Got %s classifiers.", len(classifiers))
except ClientError:
logger.exception(
"Couldn't get classifiers.",
)
raise
else:
return classifiers
```
+ For API details, see [ListDocumentClassifiers](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListDocumentClassifiers) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->listdocumentclassifiers( ).
MESSAGE 'Document classifiers listed.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidfilterex.
MESSAGE 'Invalid filter.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [ListDocumentClassifiers](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `ListTopicsDetectionJobs` with an AWS SDK or CLI
The following code examples show how to use `ListTopicsDetectionJobs`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Run a topic modeling job on sample data](comprehend_example_comprehend_Usage_TopicModeler_section.md)
------
#### [ CLI ]
**AWS CLI**
**To list all topic detection jobs**
The following `list-topics-detection-jobs` example lists all in-progress and completed asynchronous topics detection jobs.
```
aws comprehend list-topics-detection-jobs
```
Output:
```
{
"TopicsDetectionJobPropertiesList": [
{
"JobId": "123456abcdeb0e11022f22a11EXAMPLE",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:topics-detection-job/123456abcdeb0e11022f22a11EXAMPLE",
"JobName" "topic-analysis-1"
"JobStatus": "IN_PROGRESS",
"SubmitTime": "2023-06-09T18:40:35.384000+00:00",
"EndTime": "2023-06-09T18:46:41.936000+00:00",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/thefolder/111122223333-TOPICS-123456abcdeb0e11022f22a11EXAMPLE/output/output.tar.gz"
},
"NumberOfTopics": 10,
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-example-role"
},
{
"JobId": "123456abcdeb0e11022f22a1EXAMPLE2",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:topics-detection-job/123456abcdeb0e11022f22a1EXAMPLE2",
"JobName": "topic-analysis-2",
"JobStatus": "COMPLETED",
"SubmitTime": "2023-06-09T18:44:43.414000+00:00",
"EndTime": "2023-06-09T18:50:50.872000+00:00",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/thefolder/111122223333-TOPICS-123456abcdeb0e11022f22a1EXAMPLE2/output/output.tar.gz"
},
"NumberOfTopics": 10,
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-example-role"
},
{
"JobId": "123456abcdeb0e11022f22a1EXAMPLE3",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:topics-detection-job/123456abcdeb0e11022f22a1EXAMPLE3",
"JobName": "topic-analysis-2",
"JobStatus": "IN_PROGRESS",
"SubmitTime": "2023-06-09T18:50:56.737000+00:00",
"InputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-bucket",
"InputFormat": "ONE_DOC_PER_LINE"
},
"OutputDataConfig": {
"S3Uri": "s3://amzn-s3-demo-destination-bucket/thefolder/111122223333-TOPICS-123456abcdeb0e11022f22a1EXAMPLE3/output/output.tar.gz"
},
"NumberOfTopics": 10,
"DataAccessRoleArn": "arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-example-role"
}
]
}
```
For more information, see [Async analysis for Amazon Comprehend insights](https://docs.aws.amazon.com/comprehend/latest/dg/api-async-insights.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [ListTopicsDetectionJobs](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/list-topics-detection-jobs.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendTopicModeler:
"""Encapsulates a Comprehend topic modeler."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def list_jobs(self):
"""
Lists topic modeling jobs for the current account.
:return: The list of jobs.
"""
try:
response = self.comprehend_client.list_topics_detection_jobs()
jobs = response["TopicsDetectionJobPropertiesList"]
logger.info("Got %s topic detection jobs.", len(jobs))
except ClientError:
logger.exception("Couldn't get topic detection jobs.")
raise
else:
return jobs
```
+ For API details, see [ListTopicsDetectionJobs](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListTopicsDetectionJobs) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->listtopicsdetectionjobs( ).
MESSAGE 'Topics detection jobs listed.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdinvalidfilterex.
MESSAGE 'Invalid filter.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [ListTopicsDetectionJobs](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `StartDocumentClassificationJob` with an AWS SDK or CLI
The following code examples show how to use `StartDocumentClassificationJob`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
------
#### [ CLI ]
**AWS CLI**
**To start document classification job**
The following `start-document-classification-job` example starts a document classification job with a custom model on all of the files at the address specified by the `--input-data-config` tag. In this example, the input S3 bucket contains `SampleSMStext1.txt`, `SampleSMStext2.txt`, and `SampleSMStext3.txt`. The model was previously trained on document classifications of spam and non-spam, or, "ham", SMS messages. When the job is complete, `output.tar.gz` is put at the location specified by the `--output-data-config` tag. `output.tar.gz` contains `predictions.jsonl` which lists the classification of each document. The Json output is printed on one line per file, but is formatted here for readability.
```
aws comprehend start-document-classification-job \
--job-name exampleclassificationjob \
--input-data-config "S3Uri=s3://amzn-s3-demo-bucket-INPUT/jobdata/" \
--output-data-config "S3Uri=s3://amzn-s3-demo-destination-bucket/testfolder/" \
--data-access-role-arn arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-example-role \
--document-classifier-arn arn:aws:comprehend:us-west-2:111122223333:document-classifier/mymodel/version/12
```
Contents of `SampleSMStext1.txt`:
```
"CONGRATULATIONS! TXT 2155550100 to win $5000"
```
Contents of `SampleSMStext2.txt`:
```
"Hi, when do you want me to pick you up from practice?"
```
Contents of `SampleSMStext3.txt`:
```
"Plz send bank account # to 2155550100 to claim prize!!"
```
Output:
```
{
"JobId": "e758dd56b824aa717ceab551fEXAMPLE",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:document-classification-job/e758dd56b824aa717ceab551fEXAMPLE",
"JobStatus": "SUBMITTED"
}
```
Contents of `predictions.jsonl`:
```
{"File": "SampleSMSText1.txt", "Line": "0", "Classes": [{"Name": "spam", "Score": 0.9999}, {"Name": "ham", "Score": 0.0001}]}
{"File": "SampleSMStext2.txt", "Line": "0", "Classes": [{"Name": "ham", "Score": 0.9994}, {"Name": "spam", "Score": 0.0006}]}
{"File": "SampleSMSText3.txt", "Line": "0", "Classes": [{"Name": "spam", "Score": 0.9999}, {"Name": "ham", "Score": 0.0001}]}
```
For more information, see [Custom Classification](https://docs.aws.amazon.com/comprehend/latest/dg/how-document-classification.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [StartDocumentClassificationJob](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/start-document-classification-job.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def start_job(
self,
job_name,
input_bucket,
input_key,
input_format,
output_bucket,
output_key,
data_access_role_arn,
):
"""
Starts a classification job. The classifier must be trained or the job
will fail. Input is read from the specified Amazon S3 input bucket and
written to the specified output bucket. Output data is stored in a tar
archive compressed in gzip format. The job runs asynchronously, so you can
call `describe_document_classification_job` to get job status until it
returns a status of SUCCEEDED.
:param job_name: The name of the job.
:param input_bucket: The Amazon S3 bucket that contains input data.
:param input_key: The prefix used to find input data in the input
bucket. If multiple objects have the same prefix, all
of them are used.
:param input_format: The format of the input data, either one document per
file or one document per line.
:param output_bucket: The Amazon S3 bucket where output data is written.
:param output_key: The prefix prepended to the output data.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
input bucket and write to the output bucket.
:return: Information about the job, including the job ID.
"""
try:
response = self.comprehend_client.start_document_classification_job(
DocumentClassifierArn=self.classifier_arn,
JobName=job_name,
InputDataConfig={
"S3Uri": f"s3://{input_bucket}/{input_key}",
"InputFormat": input_format.value,
},
OutputDataConfig={"S3Uri": f"s3://{output_bucket}/{output_key}"},
DataAccessRoleArn=data_access_role_arn,
)
logger.info(
"Document classification job %s is %s.", job_name, response["JobStatus"]
)
except ClientError:
logger.exception("Couldn't start classification job %s.", job_name)
raise
else:
return response
```
+ For API details, see [StartDocumentClassificationJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/StartDocumentClassificationJob) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->startdocclassificationjob(
iv_jobname = iv_job_name
iv_documentclassifierarn = iv_classifier_arn
io_inputdataconfig = NEW /aws1/cl_cpdinputdataconfig(
iv_s3uri = iv_input_s3_uri
iv_inputformat = iv_input_format
)
io_outputdataconfig = NEW /aws1/cl_cpdoutputdataconfig(
iv_s3uri = iv_output_s3_uri
)
iv_dataaccessrolearn = iv_data_access_role_arn
).
MESSAGE 'Document classification job started.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdresourcenotfoundex.
MESSAGE 'Resource not found.' TYPE 'E'.
CATCH /aws1/cx_cpdresourceunavailex.
MESSAGE 'Resource unavailable.' TYPE 'E'.
CATCH /aws1/cx_cpdkmskeyvalidationex.
MESSAGE 'KMS key validation error.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanytagsex.
MESSAGE 'Too many tags.' TYPE 'E'.
CATCH /aws1/cx_cpdresrclimitexcdex.
MESSAGE 'Resource limit exceeded.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [StartDocumentClassificationJob](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Use `StartTopicsDetectionJob` with an AWS SDK or CLI
The following code examples show how to use `StartTopicsDetectionJob`.
Action examples are code excerpts from larger programs and must be run in context. You can see this action in context in the following code example:
+ [Run a topic modeling job on sample data](comprehend_example_comprehend_Usage_TopicModeler_section.md)
------
#### [ .NET ]
**SDK for .NET**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/Comprehend/#code-examples).
```
using System;
using System.Threading.Tasks;
using Amazon.Comprehend;
using Amazon.Comprehend.Model;
///
/// This example scans the documents in an Amazon Simple Storage Service
/// (Amazon S3) bucket and analyzes it for topics. The results are stored
/// in another bucket and then the resulting job properties are displayed
/// on the screen. This example was created using the AWS SDK for .NEt
/// version 3.7 and .NET Core version 5.0.
///
public static class TopicModeling
{
///
/// This methos calls a topic detection job by calling the Amazon
/// Comprehend StartTopicsDetectionJobRequest.
///
public static async Task Main()
{
var comprehendClient = new AmazonComprehendClient();
string inputS3Uri = "s3://input bucket/input path";
InputFormat inputDocFormat = InputFormat.ONE_DOC_PER_FILE;
string outputS3Uri = "s3://output bucket/output path";
string dataAccessRoleArn = "arn:aws:iam::account ID:role/data access role";
int numberOfTopics = 10;
var startTopicsDetectionJobRequest = new StartTopicsDetectionJobRequest()
{
InputDataConfig = new InputDataConfig()
{
S3Uri = inputS3Uri,
InputFormat = inputDocFormat,
},
OutputDataConfig = new OutputDataConfig()
{
S3Uri = outputS3Uri,
},
DataAccessRoleArn = dataAccessRoleArn,
NumberOfTopics = numberOfTopics,
};
var startTopicsDetectionJobResponse = await comprehendClient.StartTopicsDetectionJobAsync(startTopicsDetectionJobRequest);
var jobId = startTopicsDetectionJobResponse.JobId;
Console.WriteLine("JobId: " + jobId);
var describeTopicsDetectionJobRequest = new DescribeTopicsDetectionJobRequest()
{
JobId = jobId,
};
var describeTopicsDetectionJobResponse = await comprehendClient.DescribeTopicsDetectionJobAsync(describeTopicsDetectionJobRequest);
PrintJobProperties(describeTopicsDetectionJobResponse.TopicsDetectionJobProperties);
var listTopicsDetectionJobsResponse = await comprehendClient.ListTopicsDetectionJobsAsync(new ListTopicsDetectionJobsRequest());
foreach (var props in listTopicsDetectionJobsResponse.TopicsDetectionJobPropertiesList)
{
PrintJobProperties(props);
}
}
///
/// This method is a helper method that displays the job properties
/// from the call to StartTopicsDetectionJobRequest.
///
/// A list of properties from the call to
/// StartTopicsDetectionJobRequest.
private static void PrintJobProperties(TopicsDetectionJobProperties props)
{
Console.WriteLine($"JobId: {props.JobId}, JobName: {props.JobName}, JobStatus: {props.JobStatus}");
Console.WriteLine($"NumberOfTopics: {props.NumberOfTopics}\nInputS3Uri: {props.InputDataConfig.S3Uri}");
Console.WriteLine($"InputFormat: {props.InputDataConfig.InputFormat}, OutputS3Uri: {props.OutputDataConfig.S3Uri}");
}
}
```
+ For API details, see [StartTopicsDetectionJob](https://docs.aws.amazon.com/goto/DotNetSDKV3/comprehend-2017-11-27/StartTopicsDetectionJob) in *AWS SDK for .NET API Reference*.
------
#### [ CLI ]
**AWS CLI**
**To start a topics detection analysis job**
The following `start-topics-detection-job` example starts an asynchronous topics detection job for all files located at the address specified by the `--input-data-config` tag. When the job is complete, the folder, `output`, is placed at the location specified by the `--ouput-data-config` tag. `output` contains topic-terms.csv and doc-topics.csv. The first output file, topic-terms.csv, is a list of topics in the collection. For each topic, the list includes, by default, the top terms by topic according to their weight. The second file, `doc-topics.csv`, lists the documents associated with a topic and the proportion of the document that is concerned with the topic.
```
aws comprehend start-topics-detection-job \
--job-name example_topics_detection_job \
--language-code en \
--input-data-config "S3Uri=s3://amzn-s3-demo-bucket/" \
--output-data-config "S3Uri=s3://amzn-s3-demo-destination-bucket/testfolder/" \
--data-access-role-arn arn:aws:iam::111122223333:role/service-role/AmazonComprehendServiceRole-example-role \
--language-code en
```
Output:
```
{
"JobId": "123456abcdeb0e11022f22a11EXAMPLE",
"JobArn": "arn:aws:comprehend:us-west-2:111122223333:key-phrases-detection-job/123456abcdeb0e11022f22a11EXAMPLE",
"JobStatus": "SUBMITTED"
}
```
For more information, see [Topic Modeling](https://docs.aws.amazon.com/comprehend/latest/dg/topic-modeling.html) in the *Amazon Comprehend Developer Guide*.
+ For API details, see [StartTopicsDetectionJob](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/comprehend/start-topics-detection-job.html) in *AWS CLI Command Reference*.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
```
class ComprehendTopicModeler:
"""Encapsulates a Comprehend topic modeler."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def start_job(
self,
job_name,
input_bucket,
input_key,
input_format,
output_bucket,
output_key,
data_access_role_arn,
):
"""
Starts a topic modeling job. Input is read from the specified Amazon S3
input bucket and written to the specified output bucket. Output data is stored
in a tar archive compressed in gzip format. The job runs asynchronously, so you
can call `describe_topics_detection_job` to get job status until it
returns a status of SUCCEEDED.
:param job_name: The name of the job.
:param input_bucket: An Amazon S3 bucket that contains job input.
:param input_key: The prefix used to find input data in the input
bucket. If multiple objects have the same prefix, all
of them are used.
:param input_format: The format of the input data, either one document per
file or one document per line.
:param output_bucket: The Amazon S3 bucket where output data is written.
:param output_key: The prefix prepended to the output data.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
input bucket and write to the output bucket.
:return: Information about the job, including the job ID.
"""
try:
response = self.comprehend_client.start_topics_detection_job(
JobName=job_name,
DataAccessRoleArn=data_access_role_arn,
InputDataConfig={
"S3Uri": f"s3://{input_bucket}/{input_key}",
"InputFormat": input_format.value,
},
OutputDataConfig={"S3Uri": f"s3://{output_bucket}/{output_key}"},
)
logger.info("Started topic modeling job %s.", response["JobId"])
except ClientError:
logger.exception("Couldn't start topic modeling job.")
raise
else:
return response
```
+ For API details, see [StartTopicsDetectionJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/StartTopicsDetectionJob) in *AWS SDK for Python (Boto3) API Reference*.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cpd#code-examples).
```
TRY.
oo_result = lo_cpd->starttopicsdetectionjob(
iv_jobname = iv_job_name
io_inputdataconfig = NEW /aws1/cl_cpdinputdataconfig(
iv_s3uri = iv_input_s3_uri
iv_inputformat = iv_input_format
)
io_outputdataconfig = NEW /aws1/cl_cpdoutputdataconfig(
iv_s3uri = iv_output_s3_uri
)
iv_dataaccessrolearn = iv_data_access_role_arn
).
MESSAGE 'Topics detection job started.' TYPE 'I'.
CATCH /aws1/cx_cpdinvalidrequestex.
MESSAGE 'Invalid request.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanyrequestsex.
MESSAGE 'Too many requests.' TYPE 'E'.
CATCH /aws1/cx_cpdkmskeyvalidationex.
MESSAGE 'KMS key validation error.' TYPE 'E'.
CATCH /aws1/cx_cpdtoomanytagsex.
MESSAGE 'Too many tags.' TYPE 'E'.
CATCH /aws1/cx_cpdresrclimitexcdex.
MESSAGE 'Resource limit exceeded.' TYPE 'E'.
CATCH /aws1/cx_cpdinternalserverex.
MESSAGE 'Internal server error occurred.' TYPE 'E'.
ENDTRY.
```
+ For API details, see [StartTopicsDetectionJob](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html) in *AWS SDK for SAP ABAP API reference*.
------
# Scenarios for Amazon Comprehend using AWS SDKs
The following code examples show you how to implement common scenarios in Amazon Comprehend with AWS SDKs. These scenarios show you how to accomplish specific tasks by calling multiple functions within Amazon Comprehend or combined with other AWS services. Each scenario includes a link to the complete source code, where you can find instructions on how to set up and run the code.
Scenarios target an intermediate level of experience to help you understand service actions in context.
**Topics**
+ [Build an Amazon Transcribe streaming app](comprehend_example_cross_TranscriptionStreamingApp_section.md)
+ [Building an Amazon Lex chatbot](comprehend_example_cross_LexChatbotLanguages_section.md)
+ [Create a messaging application](comprehend_example_cross_SQSMessageApp_section.md)
+ [Create an application to analyze customer feedback](comprehend_example_cross_FSA_section.md)
+ [Detect document elements](comprehend_example_comprehend_Usage_DetectApis_section.md)
+ [Detect entities in text extracted from an image](comprehend_example_cross_TextractComprehendDetectEntities_section.md)
+ [Run a topic modeling job on sample data](comprehend_example_comprehend_Usage_TopicModeler_section.md)
+ [Train a custom classifier and classify documents](comprehend_example_comprehend_Usage_ComprehendClassifier_section.md)
# Build an Amazon Transcribe streaming app
The following code example shows how to build an app that records, transcribes, and translates live audio in real-time, and emails the results.
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
Shows how to use Amazon Transcribe to build an app that records, transcribes, and translates live audio in real-time, and emails the results using Amazon Simple Email Service (Amazon SES).
For complete source code and instructions on how to set up and run, see the full example on [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cross-services/transcribe-streaming-app).
**Services used in this example**
+ Amazon Comprehend
+ Amazon SES
+ Amazon Transcribe
+ Amazon Translate
------
# Create an Amazon Lex chatbot to engage your website visitors
The following code examples show how to create a chatbot to engage your website visitors.
------
#### [ Java ]
**SDK for Java 2.x**
Shows how to use the Amazon Lex API to create a Chatbot within a web application to engage your web site visitors.
For complete source code and instructions on how to set up and run, see the full example on [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_lex_chatbot).
**Services used in this example**
+ Amazon Comprehend
+ Amazon Lex
+ Amazon Translate
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
Shows how to use the Amazon Lex API to create a Chatbot within a web application to engage your web site visitors.
For complete source code and instructions on how to set up and run, see the full example [Building an Amazon Lex chatbot](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/lex-bot-example.html) in the AWS SDK for JavaScript developer guide.
**Services used in this example**
+ Amazon Comprehend
+ Amazon Lex
+ Amazon Translate
------
# Create a web application that sends and retrieves messages by using Amazon SQS
The following code examples show how to create a messaging application by using Amazon SQS.
------
#### [ Java ]
**SDK for Java 2.x**
Shows how to use the Amazon SQS API to develop a Spring REST API that sends and retrieves messages.
For complete source code and instructions on how to set up and run, see the full example on [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_message_application).
**Services used in this example**
+ Amazon Comprehend
+ Amazon SQS
------
#### [ Kotlin ]
**SDK for Kotlin**
Shows how to use the Amazon SQS API to develop a Spring REST API that sends and retrieves messages.
For complete source code and instructions on how to set up and run, see the full example on [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/creating_message_application).
**Services used in this example**
+ Amazon Comprehend
+ Amazon SQS
------
# Create an application that analyzes customer feedback and synthesizes audio
The following code examples show how to create an application that analyzes customer comment cards, translates them from their original language, determines their sentiment, and generates an audio file from the translated text.
------
#### [ .NET ]
**SDK for .NET**
This example application analyzes and stores customer feedback cards. Specifically, it fulfills the need of a fictitious hotel in New York City. The hotel receives feedback from guests in various languages in the form of physical comment cards. That feedback is uploaded into the app through a web client. After an image of a comment card is uploaded, the following steps occur:
+ Text is extracted from the image using Amazon Textract.
+ Amazon Comprehend determines the sentiment of the extracted text and its language.
+ The extracted text is translated to English using Amazon Translate.
+ Amazon Polly synthesizes an audio file from the extracted text.
The full app can be deployed with the AWS CDK. For source code and deployment instructions, see the project in [ GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/cross-service/FeedbackSentimentAnalyzer).
**Services used in this example**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
#### [ Java ]
**SDK for Java 2.x**
This example application analyzes and stores customer feedback cards. Specifically, it fulfills the need of a fictitious hotel in New York City. The hotel receives feedback from guests in various languages in the form of physical comment cards. That feedback is uploaded into the app through a web client. After an image of a comment card is uploaded, the following steps occur:
+ Text is extracted from the image using Amazon Textract.
+ Amazon Comprehend determines the sentiment of the extracted text and its language.
+ The extracted text is translated to English using Amazon Translate.
+ Amazon Polly synthesizes an audio file from the extracted text.
The full app can be deployed with the AWS CDK. For source code and deployment instructions, see the project in [ GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_fsa_app).
**Services used in this example**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
This example application analyzes and stores customer feedback cards. Specifically, it fulfills the need of a fictitious hotel in New York City. The hotel receives feedback from guests in various languages in the form of physical comment cards. That feedback is uploaded into the app through a web client. After an image of a comment card is uploaded, the following steps occur:
+ Text is extracted from the image using Amazon Textract.
+ Amazon Comprehend determines the sentiment of the extracted text and its language.
+ The extracted text is translated to English using Amazon Translate.
+ Amazon Polly synthesizes an audio file from the extracted text.
The full app can be deployed with the AWS CDK. For source code and deployment instructions, see the project in [ GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cross-services/feedback-sentiment-analyzer). The following excerpts show how the AWS SDK for JavaScript is used inside of Lambda functions.
```
import {
ComprehendClient,
DetectDominantLanguageCommand,
DetectSentimentCommand,
} from "@aws-sdk/client-comprehend";
/**
* Determine the language and sentiment of the extracted text.
*
* @param {{ source_text: string}} extractTextOutput
*/
export const handler = async (extractTextOutput) => {
const comprehendClient = new ComprehendClient({});
const detectDominantLanguageCommand = new DetectDominantLanguageCommand({
Text: extractTextOutput.source_text,
});
// The source language is required for sentiment analysis and
// translation in the next step.
const { Languages } = await comprehendClient.send(
detectDominantLanguageCommand,
);
const languageCode = Languages[0].LanguageCode;
const detectSentimentCommand = new DetectSentimentCommand({
Text: extractTextOutput.source_text,
LanguageCode: languageCode,
});
const { Sentiment } = await comprehendClient.send(detectSentimentCommand);
return {
sentiment: Sentiment,
language_code: languageCode,
};
};
```
```
import {
DetectDocumentTextCommand,
TextractClient,
} from "@aws-sdk/client-textract";
/**
* Fetch the S3 object from the event and analyze it using Amazon Textract.
*
* @param {import("@types/aws-lambda").EventBridgeEvent<"Object Created">} eventBridgeS3Event
*/
export const handler = async (eventBridgeS3Event) => {
const textractClient = new TextractClient();
const detectDocumentTextCommand = new DetectDocumentTextCommand({
Document: {
S3Object: {
Bucket: eventBridgeS3Event.bucket,
Name: eventBridgeS3Event.object,
},
},
});
// Textract returns a list of blocks. A block can be a line, a page, word, etc.
// Each block also contains geometry of the detected text.
// For more information on the Block type, see https://docs.aws.amazon.com/textract/latest/dg/API_Block.html.
const { Blocks } = await textractClient.send(detectDocumentTextCommand);
// For the purpose of this example, we are only interested in words.
const extractedWords = Blocks.filter((b) => b.BlockType === "WORD").map(
(b) => b.Text,
);
return extractedWords.join(" ");
};
```
```
import { PollyClient, SynthesizeSpeechCommand } from "@aws-sdk/client-polly";
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
/**
* Synthesize an audio file from text.
*
* @param {{ bucket: string, translated_text: string, object: string}} sourceDestinationConfig
*/
export const handler = async (sourceDestinationConfig) => {
const pollyClient = new PollyClient({});
const synthesizeSpeechCommand = new SynthesizeSpeechCommand({
Engine: "neural",
Text: sourceDestinationConfig.translated_text,
VoiceId: "Ruth",
OutputFormat: "mp3",
});
const { AudioStream } = await pollyClient.send(synthesizeSpeechCommand);
const audioKey = `${sourceDestinationConfig.object}.mp3`;
// Store the audio file in S3.
const s3Client = new S3Client();
const upload = new Upload({
client: s3Client,
params: {
Bucket: sourceDestinationConfig.bucket,
Key: audioKey,
Body: AudioStream,
ContentType: "audio/mp3",
},
});
await upload.done();
return audioKey;
};
```
```
import {
TranslateClient,
TranslateTextCommand,
} from "@aws-sdk/client-translate";
/**
* Translate the extracted text to English.
*
* @param {{ extracted_text: string, source_language_code: string}} textAndSourceLanguage
*/
export const handler = async (textAndSourceLanguage) => {
const translateClient = new TranslateClient({});
const translateCommand = new TranslateTextCommand({
SourceLanguageCode: textAndSourceLanguage.source_language_code,
TargetLanguageCode: "en",
Text: textAndSourceLanguage.extracted_text,
});
const { TranslatedText } = await translateClient.send(translateCommand);
return { translated_text: TranslatedText };
};
```
**Services used in this example**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
#### [ Ruby ]
**SDK for Ruby**
This example application analyzes and stores customer feedback cards. Specifically, it fulfills the need of a fictitious hotel in New York City. The hotel receives feedback from guests in various languages in the form of physical comment cards. That feedback is uploaded into the app through a web client. After an image of a comment card is uploaded, the following steps occur:
+ Text is extracted from the image using Amazon Textract.
+ Amazon Comprehend determines the sentiment of the extracted text and its language.
+ The extracted text is translated to English using Amazon Translate.
+ Amazon Polly synthesizes an audio file from the extracted text.
The full app can be deployed with the AWS CDK. For source code and deployment instructions, see the project in [ GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/ruby/cross_service_examples/feedback_sentiment_analyzer).
**Services used in this example**
+ Amazon Comprehend
+ Lambda
+ Amazon Polly
+ Amazon Textract
+ Amazon Translate
------
# Detect document elements with Amazon Comprehend and an AWS SDK
The following code example shows how to:
+ Detect languages, entities, and key phrases in a document.
+ Detect personally identifiable information (PII) in a document.
+ Detect the sentiment of a document.
+ Detect syntax elements in a document.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
Create a class that wraps Amazon Comprehend actions.
```
import logging
from pprint import pprint
import boto3
from botocore.exceptions import ClientError
logger = logging.getLogger(__name__)
class ComprehendDetect:
"""Encapsulates Comprehend detection functions."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def detect_languages(self, text):
"""
Detects languages used in a document.
:param text: The document to inspect.
:return: The list of languages along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_dominant_language(Text=text)
languages = response["Languages"]
logger.info("Detected %s languages.", len(languages))
except ClientError:
logger.exception("Couldn't detect languages.")
raise
else:
return languages
def detect_entities(self, text, language_code):
"""
Detects entities in a document. Entities can be things like people and places
or other common terms.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of entities along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_entities(
Text=text, LanguageCode=language_code
)
entities = response["Entities"]
logger.info("Detected %s entities.", len(entities))
except ClientError:
logger.exception("Couldn't detect entities.")
raise
else:
return entities
def detect_key_phrases(self, text, language_code):
"""
Detects key phrases in a document. A key phrase is typically a noun and its
modifiers.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of key phrases along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_key_phrases(
Text=text, LanguageCode=language_code
)
phrases = response["KeyPhrases"]
logger.info("Detected %s phrases.", len(phrases))
except ClientError:
logger.exception("Couldn't detect phrases.")
raise
else:
return phrases
def detect_pii(self, text, language_code):
"""
Detects personally identifiable information (PII) in a document. PII can be
things like names, account numbers, or addresses.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of PII entities along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_pii_entities(
Text=text, LanguageCode=language_code
)
entities = response["Entities"]
logger.info("Detected %s PII entities.", len(entities))
except ClientError:
logger.exception("Couldn't detect PII entities.")
raise
else:
return entities
def detect_sentiment(self, text, language_code):
"""
Detects the overall sentiment expressed in a document. Sentiment can
be positive, negative, neutral, or a mixture.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The sentiments along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_sentiment(
Text=text, LanguageCode=language_code
)
logger.info("Detected primary sentiment %s.", response["Sentiment"])
except ClientError:
logger.exception("Couldn't detect sentiment.")
raise
else:
return response
def detect_syntax(self, text, language_code):
"""
Detects syntactical elements of a document. Syntax tokens are portions of
text along with their use as parts of speech, such as nouns, verbs, and
interjections.
:param text: The document to inspect.
:param language_code: The language of the document.
:return: The list of syntax tokens along with their confidence scores.
"""
try:
response = self.comprehend_client.detect_syntax(
Text=text, LanguageCode=language_code
)
tokens = response["SyntaxTokens"]
logger.info("Detected %s syntax tokens.", len(tokens))
except ClientError:
logger.exception("Couldn't detect syntax.")
raise
else:
return tokens
```
Call functions on the wrapper class to detect entities, phrases, and more in a document.
```
def usage_demo():
print("-" * 88)
print("Welcome to the Amazon Comprehend detection demo!")
print("-" * 88)
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
comp_detect = ComprehendDetect(boto3.client("comprehend"))
with open("detect_sample.txt") as sample_file:
sample_text = sample_file.read()
demo_size = 3
print("Sample text used for this demo:")
print("-" * 88)
print(sample_text)
print("-" * 88)
print("Detecting languages.")
languages = comp_detect.detect_languages(sample_text)
pprint(languages)
lang_code = languages[0]["LanguageCode"]
print("Detecting entities.")
entities = comp_detect.detect_entities(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(entities[:demo_size])
print("Detecting key phrases.")
phrases = comp_detect.detect_key_phrases(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(phrases[:demo_size])
print("Detecting personally identifiable information (PII).")
pii_entities = comp_detect.detect_pii(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(pii_entities[:demo_size])
print("Detecting sentiment.")
sentiment = comp_detect.detect_sentiment(sample_text, lang_code)
print(f"Sentiment: {sentiment['Sentiment']}")
print("SentimentScore:")
pprint(sentiment["SentimentScore"])
print("Detecting syntax elements.")
syntax_tokens = comp_detect.detect_syntax(sample_text, lang_code)
print(f"The first {demo_size} are:")
pprint(syntax_tokens[:demo_size])
print("Thanks for watching!")
print("-" * 88)
```
+ For API details, see the following topics in *AWS SDK for Python (Boto3) API Reference*.
+ [DetectDominantLanguage](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectDominantLanguage)
+ [DetectEntities](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectEntities)
+ [DetectKeyPhrases](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectKeyPhrases)
+ [DetectPiiEntities](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectPiiEntities)
+ [DetectSentiment](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectSentiment)
+ [DetectSyntax](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DetectSyntax)
------
# Detect entities in text extracted from an image using an AWS SDK
The following code example shows how to use Amazon Comprehend to detect entities in text extracted by Amazon Textract from an image that is stored in Amazon S3.
------
#### [ Python ]
**SDK for Python (Boto3)**
Shows how to use the AWS SDK for Python (Boto3) in a Jupyter notebook to detect entities in text that is extracted from an image. This example uses Amazon Textract to extract text from an image stored in Amazon Simple Storage Service (Amazon S3) and Amazon Comprehend to detect entities in the extracted text.
This example is a Jupyter notebook and must be run in an environment that can host notebooks. For instructions on how to run the example using Amazon SageMaker AI, see the directions in [TextractAndComprehendNotebook.ipynb](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/cross_service/textract_comprehend_notebook/TextractAndComprehendNotebook.ipynb).
For complete source code and instructions on how to set up and run, see the full example on [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/cross_service/textract_comprehend_notebook#readme).
**Services used in this example**
+ Amazon Comprehend
+ Amazon S3
+ Amazon Textract
------
# Run an Amazon Comprehend topic modeling job on sample data using an AWS SDK
The following code example shows how to:
+ Run an Amazon Comprehend topic modeling job on sample data.
+ Get information about the job.
+ Extract job output data from Amazon S3.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
Create a wrapper class to call Amazon Comprehend topic modeling actions.
```
class ComprehendTopicModeler:
"""Encapsulates a Comprehend topic modeler."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
def start_job(
self,
job_name,
input_bucket,
input_key,
input_format,
output_bucket,
output_key,
data_access_role_arn,
):
"""
Starts a topic modeling job. Input is read from the specified Amazon S3
input bucket and written to the specified output bucket. Output data is stored
in a tar archive compressed in gzip format. The job runs asynchronously, so you
can call `describe_topics_detection_job` to get job status until it
returns a status of SUCCEEDED.
:param job_name: The name of the job.
:param input_bucket: An Amazon S3 bucket that contains job input.
:param input_key: The prefix used to find input data in the input
bucket. If multiple objects have the same prefix, all
of them are used.
:param input_format: The format of the input data, either one document per
file or one document per line.
:param output_bucket: The Amazon S3 bucket where output data is written.
:param output_key: The prefix prepended to the output data.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
input bucket and write to the output bucket.
:return: Information about the job, including the job ID.
"""
try:
response = self.comprehend_client.start_topics_detection_job(
JobName=job_name,
DataAccessRoleArn=data_access_role_arn,
InputDataConfig={
"S3Uri": f"s3://{input_bucket}/{input_key}",
"InputFormat": input_format.value,
},
OutputDataConfig={"S3Uri": f"s3://{output_bucket}/{output_key}"},
)
logger.info("Started topic modeling job %s.", response["JobId"])
except ClientError:
logger.exception("Couldn't start topic modeling job.")
raise
else:
return response
def describe_job(self, job_id):
"""
Gets metadata about a topic modeling job.
:param job_id: The ID of the job to look up.
:return: Metadata about the job.
"""
try:
response = self.comprehend_client.describe_topics_detection_job(
JobId=job_id
)
job = response["TopicsDetectionJobProperties"]
logger.info("Got topic detection job %s.", job_id)
except ClientError:
logger.exception("Couldn't get topic detection job %s.", job_id)
raise
else:
return job
def list_jobs(self):
"""
Lists topic modeling jobs for the current account.
:return: The list of jobs.
"""
try:
response = self.comprehend_client.list_topics_detection_jobs()
jobs = response["TopicsDetectionJobPropertiesList"]
logger.info("Got %s topic detection jobs.", len(jobs))
except ClientError:
logger.exception("Couldn't get topic detection jobs.")
raise
else:
return jobs
```
Use the wrapper class to run a topic modeling job and get job data.
```
def usage_demo():
print("-" * 88)
print("Welcome to the Amazon Comprehend topic modeling demo!")
print("-" * 88)
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
input_prefix = "input/"
output_prefix = "output/"
demo_resources = ComprehendDemoResources(
boto3.resource("s3"), boto3.resource("iam")
)
topic_modeler = ComprehendTopicModeler(boto3.client("comprehend"))
print("Setting up storage and security resources needed for the demo.")
demo_resources.setup("comprehend-topic-modeler-demo")
print("Copying sample data from public bucket into input bucket.")
demo_resources.bucket.copy(
{"Bucket": "public-sample-us-west-2", "Key": "TopicModeling/Sample.txt"},
f"{input_prefix}sample.txt",
)
print("Starting topic modeling job on sample data.")
job_info = topic_modeler.start_job(
"demo-topic-modeling-job",
demo_resources.bucket.name,
input_prefix,
JobInputFormat.per_line,
demo_resources.bucket.name,
output_prefix,
demo_resources.data_access_role.arn,
)
print(
f"Waiting for job {job_info['JobId']} to complete. This typically takes "
f"20 - 30 minutes."
)
job_waiter = JobCompleteWaiter(topic_modeler.comprehend_client)
job_waiter.wait(job_info["JobId"])
job = topic_modeler.describe_job(job_info["JobId"])
print(f"Job {job['JobId']} complete:")
pprint(job)
print(
f"Getting job output data from the output Amazon S3 bucket: "
f"{job['OutputDataConfig']['S3Uri']}."
)
job_output = demo_resources.extract_job_output(job)
lines = 10
print(f"First {lines} lines of document topics output:")
pprint(job_output["doc-topics.csv"]["data"][:lines])
print(f"First {lines} lines of terms output:")
pprint(job_output["topic-terms.csv"]["data"][:lines])
print("Cleaning up resources created for the demo.")
demo_resources.cleanup()
print("Thanks for watching!")
print("-" * 88)
```
+ For API details, see the following topics in *AWS SDK for Python (Boto3) API Reference*.
+ [DescribeTopicsDetectionJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeTopicsDetectionJob)
+ [ListTopicsDetectionJobs](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListTopicsDetectionJobs)
+ [StartTopicsDetectionJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/StartTopicsDetectionJob)
------
# Train a custom Amazon Comprehend classifier and classify documents using an AWS SDK
The following code example shows how to:
+ Create an Amazon Comprehend multi-label classifier.
+ Train the classifier on sample data.
+ Run a classification job on a second set of data.
+ Extract the job output data from Amazon S3.
------
#### [ Python ]
**SDK for Python (Boto3)**
There's more on GitHub. Find the complete example and learn how to set up and run in the [AWS Code Examples Repository](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/comprehend#code-examples).
Create a wrapper class to call Amazon Comprehend document classifier actions.
```
class ComprehendClassifier:
"""Encapsulates an Amazon Comprehend custom classifier."""
def __init__(self, comprehend_client):
"""
:param comprehend_client: A Boto3 Comprehend client.
"""
self.comprehend_client = comprehend_client
self.classifier_arn = None
def create(
self,
name,
language_code,
training_bucket,
training_key,
data_access_role_arn,
mode,
):
"""
Creates a custom classifier. After the classifier is created, it immediately
starts training on the data found in the specified Amazon S3 bucket. Training
can take 30 minutes or longer. The `describe_document_classifier` function
can be used to get training status and returns a status of TRAINED when the
classifier is ready to use.
:param name: The name of the classifier.
:param language_code: The language the classifier can operate on.
:param training_bucket: The Amazon S3 bucket that contains the training data.
:param training_key: The prefix used to find training data in the training
bucket. If multiple objects have the same prefix, all
of them are used.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
training bucket.
:return: The ARN of the newly created classifier.
"""
try:
response = self.comprehend_client.create_document_classifier(
DocumentClassifierName=name,
LanguageCode=language_code,
InputDataConfig={"S3Uri": f"s3://{training_bucket}/{training_key}"},
DataAccessRoleArn=data_access_role_arn,
Mode=mode.value,
)
self.classifier_arn = response["DocumentClassifierArn"]
logger.info("Started classifier creation. Arn is: %s.", self.classifier_arn)
except ClientError:
logger.exception("Couldn't create classifier %s.", name)
raise
else:
return self.classifier_arn
def describe(self, classifier_arn=None):
"""
Gets metadata about a custom classifier, including its current status.
:param classifier_arn: The ARN of the classifier to look up.
:return: Metadata about the classifier.
"""
if classifier_arn is not None:
self.classifier_arn = classifier_arn
try:
response = self.comprehend_client.describe_document_classifier(
DocumentClassifierArn=self.classifier_arn
)
classifier = response["DocumentClassifierProperties"]
logger.info("Got classifier %s.", self.classifier_arn)
except ClientError:
logger.exception("Couldn't get classifier %s.", self.classifier_arn)
raise
else:
return classifier
def list(self):
"""
Lists custom classifiers for the current account.
:return: The list of classifiers.
"""
try:
response = self.comprehend_client.list_document_classifiers()
classifiers = response["DocumentClassifierPropertiesList"]
logger.info("Got %s classifiers.", len(classifiers))
except ClientError:
logger.exception(
"Couldn't get classifiers.",
)
raise
else:
return classifiers
def delete(self):
"""
Deletes the classifier.
"""
try:
self.comprehend_client.delete_document_classifier(
DocumentClassifierArn=self.classifier_arn
)
logger.info("Deleted classifier %s.", self.classifier_arn)
self.classifier_arn = None
except ClientError:
logger.exception("Couldn't deleted classifier %s.", self.classifier_arn)
raise
def start_job(
self,
job_name,
input_bucket,
input_key,
input_format,
output_bucket,
output_key,
data_access_role_arn,
):
"""
Starts a classification job. The classifier must be trained or the job
will fail. Input is read from the specified Amazon S3 input bucket and
written to the specified output bucket. Output data is stored in a tar
archive compressed in gzip format. The job runs asynchronously, so you can
call `describe_document_classification_job` to get job status until it
returns a status of SUCCEEDED.
:param job_name: The name of the job.
:param input_bucket: The Amazon S3 bucket that contains input data.
:param input_key: The prefix used to find input data in the input
bucket. If multiple objects have the same prefix, all
of them are used.
:param input_format: The format of the input data, either one document per
file or one document per line.
:param output_bucket: The Amazon S3 bucket where output data is written.
:param output_key: The prefix prepended to the output data.
:param data_access_role_arn: The Amazon Resource Name (ARN) of a role that
grants Comprehend permission to read from the
input bucket and write to the output bucket.
:return: Information about the job, including the job ID.
"""
try:
response = self.comprehend_client.start_document_classification_job(
DocumentClassifierArn=self.classifier_arn,
JobName=job_name,
InputDataConfig={
"S3Uri": f"s3://{input_bucket}/{input_key}",
"InputFormat": input_format.value,
},
OutputDataConfig={"S3Uri": f"s3://{output_bucket}/{output_key}"},
DataAccessRoleArn=data_access_role_arn,
)
logger.info(
"Document classification job %s is %s.", job_name, response["JobStatus"]
)
except ClientError:
logger.exception("Couldn't start classification job %s.", job_name)
raise
else:
return response
def describe_job(self, job_id):
"""
Gets metadata about a classification job.
:param job_id: The ID of the job to look up.
:return: Metadata about the job.
"""
try:
response = self.comprehend_client.describe_document_classification_job(
JobId=job_id
)
job = response["DocumentClassificationJobProperties"]
logger.info("Got classification job %s.", job["JobName"])
except ClientError:
logger.exception("Couldn't get classification job %s.", job_id)
raise
else:
return job
def list_jobs(self):
"""
Lists the classification jobs for the current account.
:return: The list of jobs.
"""
try:
response = self.comprehend_client.list_document_classification_jobs()
jobs = response["DocumentClassificationJobPropertiesList"]
logger.info("Got %s document classification jobs.", len(jobs))
except ClientError:
logger.exception(
"Couldn't get document classification jobs.",
)
raise
else:
return jobs
```
Create a class to help run the scenario.
```
class ClassifierDemo:
"""
Encapsulates functions used to run the demonstration.
"""
def __init__(self, demo_resources):
"""
:param demo_resources: A ComprehendDemoResources class that manages resources
for the demonstration.
"""
self.demo_resources = demo_resources
self.training_prefix = "training/"
self.input_prefix = "input/"
self.input_format = JobInputFormat.per_line
self.output_prefix = "output/"
def setup(self):
"""Creates AWS resources used by the demo."""
self.demo_resources.setup("comprehend-classifier-demo")
def cleanup(self):
"""Deletes AWS resources used by the demo."""
self.demo_resources.cleanup()
@staticmethod
def _sanitize_text(text):
"""Removes characters that cause errors for the document parser."""
return text.replace("\r", " ").replace("\n", " ").replace(",", ";")
@staticmethod
def _get_issues(query, issue_count):
"""
Gets issues from GitHub using the specified query parameters.
:param query: The query string used to request issues from the GitHub API.
:param issue_count: The number of issues to retrieve.
:return: The list of issues retrieved from GitHub.
"""
issues = []
logger.info("Requesting issues from %s?%s.", GITHUB_SEARCH_URL, query)
response = requests.get(f"{GITHUB_SEARCH_URL}?{query}&per_page={issue_count}")
if response.status_code == 200:
issue_page = response.json()["items"]
logger.info("Got %s issues.", len(issue_page))
issues = [
{
"title": ClassifierDemo._sanitize_text(issue["title"]),
"body": ClassifierDemo._sanitize_text(issue["body"]),
"labels": {label["name"] for label in issue["labels"]},
}
for issue in issue_page
]
else:
logger.error(
"GitHub returned error code %s with message %s.",
response.status_code,
response.json(),
)
logger.info("Found %s issues.", len(issues))
return issues
def get_training_issues(self, training_labels):
"""
Gets issues used for training the custom classifier. Training issues are
closed issues from the Boto3 repo that have known labels. Comprehend
requires a minimum of ten training issues per label.
:param training_labels: The issue labels to use for training.
:return: The set of issues used for training.
"""
issues = []
per_label_count = 15
for label in training_labels:
issues += self._get_issues(
f"q=type:issue+repo:boto/boto3+state:closed+label:{label}",
per_label_count,
)
for issue in issues:
issue["labels"] = issue["labels"].intersection(training_labels)
return issues
def get_input_issues(self, training_labels):
"""
Gets input issues from GitHub. For demonstration purposes, input issues
are open issues from the Boto3 repo with known labels, though in practice
any issue could be submitted to the classifier for labeling.
:param training_labels: The set of labels to query for.
:return: The set of issues used for input.
"""
issues = []
per_label_count = 5
for label in training_labels:
issues += self._get_issues(
f"q=type:issue+repo:boto/boto3+state:open+label:{label}",
per_label_count,
)
return issues
def upload_issue_data(self, issues, training=False):
"""
Uploads issue data to an Amazon S3 bucket, either for training or for input.
The data is first put into the format expected by Comprehend. For training,
the set of pipe-delimited labels is prepended to each document. For
input, labels are not sent.
:param issues: The set of issues to upload to Amazon S3.
:param training: Indicates whether the issue data is used for training or
input.
"""
try:
obj_key = (
self.training_prefix if training else self.input_prefix
) + "issues.txt"
if training:
issue_strings = [
f"{'|'.join(issue['labels'])},{issue['title']} {issue['body']}"
for issue in issues
]
else:
issue_strings = [
f"{issue['title']} {issue['body']}" for issue in issues
]
issue_bytes = BytesIO("\n".join(issue_strings).encode("utf-8"))
self.demo_resources.bucket.upload_fileobj(issue_bytes, obj_key)
logger.info(
"Uploaded data as %s to bucket %s.",
obj_key,
self.demo_resources.bucket.name,
)
except ClientError:
logger.exception(
"Couldn't upload data to bucket %s.", self.demo_resources.bucket.name
)
raise
def extract_job_output(self, job):
"""Extracts job output from Amazon S3."""
return self.demo_resources.extract_job_output(job)
@staticmethod
def reconcile_job_output(input_issues, output_dict):
"""
Reconciles job output with the list of input issues. Because the input issues
have known labels, these can be compared with the labels added by the
classifier to judge the accuracy of the output.
:param input_issues: The list of issues used as input.
:param output_dict: The dictionary of data that is output by the classifier.
:return: The list of reconciled input and output data.
"""
reconciled = []
for archive in output_dict.values():
for line in archive["data"]:
in_line = int(line["Line"])
in_labels = input_issues[in_line]["labels"]
out_labels = {
label["Name"]
for label in line["Labels"]
if float(label["Score"]) > 0.3
}
reconciled.append(
f"{line['File']}, line {in_line} has labels {in_labels}.\n"
f"\tClassifier assigned {out_labels}."
)
logger.info("Reconciled input and output labels.")
return reconciled
```
Train a classifier on a set of GitHub issues with known labels, then send a second set of GitHub issues to the classifier so that they can be labeled.
```
def usage_demo():
print("-" * 88)
print("Welcome to the Amazon Comprehend custom document classifier demo!")
print("-" * 88)
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
comp_demo = ClassifierDemo(
ComprehendDemoResources(boto3.resource("s3"), boto3.resource("iam"))
)
comp_classifier = ComprehendClassifier(boto3.client("comprehend"))
classifier_trained_waiter = ClassifierTrainedWaiter(
comp_classifier.comprehend_client
)
training_labels = {"bug", "feature-request", "dynamodb", "s3"}
print("Setting up storage and security resources needed for the demo.")
comp_demo.setup()
print("Getting training data from GitHub and uploading it to Amazon S3.")
training_issues = comp_demo.get_training_issues(training_labels)
comp_demo.upload_issue_data(training_issues, True)
classifier_name = "doc-example-classifier"
print(f"Creating document classifier {classifier_name}.")
comp_classifier.create(
classifier_name,
"en",
comp_demo.demo_resources.bucket.name,
comp_demo.training_prefix,
comp_demo.demo_resources.data_access_role.arn,
ClassifierMode.multi_label,
)
print(
f"Waiting until {classifier_name} is trained. This typically takes "
f"30–40 minutes."
)
classifier_trained_waiter.wait(comp_classifier.classifier_arn)
print(f"Classifier {classifier_name} is trained:")
pprint(comp_classifier.describe())
print("Getting input data from GitHub and uploading it to Amazon S3.")
input_issues = comp_demo.get_input_issues(training_labels)
comp_demo.upload_issue_data(input_issues)
print("Starting classification job on input data.")
job_info = comp_classifier.start_job(
"issue_classification_job",
comp_demo.demo_resources.bucket.name,
comp_demo.input_prefix,
comp_demo.input_format,
comp_demo.demo_resources.bucket.name,
comp_demo.output_prefix,
comp_demo.demo_resources.data_access_role.arn,
)
print(f"Waiting for job {job_info['JobId']} to complete.")
job_waiter = JobCompleteWaiter(comp_classifier.comprehend_client)
job_waiter.wait(job_info["JobId"])
job = comp_classifier.describe_job(job_info["JobId"])
print(f"Job {job['JobId']} complete:")
pprint(job)
print(
f"Getting job output data from Amazon S3: "
f"{job['OutputDataConfig']['S3Uri']}."
)
job_output = comp_demo.extract_job_output(job)
print("Job output:")
pprint(job_output)
print("Reconciling job output with labels from GitHub:")
reconciled_output = comp_demo.reconcile_job_output(input_issues, job_output)
print(*reconciled_output, sep="\n")
answer = input(f"Do you want to delete the classifier {classifier_name} (y/n)? ")
if answer.lower() == "y":
print(f"Deleting {classifier_name}.")
comp_classifier.delete()
print("Cleaning up resources created for the demo.")
comp_demo.cleanup()
print("Thanks for watching!")
print("-" * 88)
```
+ For API details, see the following topics in *AWS SDK for Python (Boto3) API Reference*.
+ [CreateDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/CreateDocumentClassifier)
+ [DeleteDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DeleteDocumentClassifier)
+ [DescribeDocumentClassificationJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeDocumentClassificationJob)
+ [DescribeDocumentClassifier](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/DescribeDocumentClassifier)
+ [ListDocumentClassificationJobs](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListDocumentClassificationJobs)
+ [ListDocumentClassifiers](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/ListDocumentClassifiers)
+ [StartDocumentClassificationJob](https://docs.aws.amazon.com/goto/boto3/comprehend-2017-11-27/StartDocumentClassificationJob)
------