# Transform unstructured data into meaningful insights using Amazon Bedrock Data Automation ## What is Bedrock Data Automation? Bedrock Data Automation (BDA) is a cloud-based service that simplifies the process of extracting valuable insights from unstructured content—such as documents, images, video, and audio. BDA leverages generative AI to automate the transformation of multi-modal data into structured formats, enabling developers to build applications and automate complex workflows with greater speed and accuracy. Here are some example use cases: + **Document processing:** BDA enables you to automate intelligent document processing (IDP) workflows at scale without the need to orchestrate complex document processing tasks like classification, extraction, normalization, or validation. This helps you to transform unstructured documents to business-specific, structured data outputs. You can customize BDA output to integrate with your existing systems and workflows. + **Media analysis:** Add meaningful insights to unstructured video. Create summaries of each scene, help identify unsafe or explicit content, extract text that appears in the video, and classify content based on advertisements or brands. You can then leverage these insights to enable intelligent video search, improve contextual advertising placement, and help with brand safety and compliance. + **Generative AI assistants:** Enhance the performance of your retrieval-augmented generation (RAG) powered question answering applications by providing them with rich, modality-specific data representations extracted from your documents, images, video, and audio. BDA provides a unified, API-driven experience that allows you to process multi-modal content through a single interface, eliminating the need to manage and orchestrate multiple AI models and services. With built-in safeguards, such as visual grounding and confidence scores, BDA helps you improve the accuracy and trustworthiness of the extracted insights, making it easier to integrate into your enterprise workflows. # How Bedrock Data Automation works Bedrock Data Automation (BDA) lets you configure output based on your processing needs for a specific data type: documents, images, video or audio. BDA can generate standard output or custom output. Below are some key concepts for understanding how BDA works. If you're a new user, start with the information about standard output. + Standard output – Sending a file to BDA with no other information returns the default standard output, which consists of commonly required information that's based on the data type. Examples include audio transcriptions, scene summaries for video, and document summaries. These outputs can be tuned to your use case using projects to modify them. For more information, see [Standard output in Bedrock Data Automation](bda-standard-output.md). + Custom output – For documents, audio,and images only. Choose custom output to define exactly what information you want to extract using a blueprint. A blueprint consists of a list of expected fields that you want retrieved from a file. Each field represents a piece of information that needs to be extracted to meet your specific use case. You can create your own blueprints, or select predefined blueprints from the BDA blueprint catalog. For more information, see [Custom output and blueprints](bda-custom-output-idp.md). + Projects – A project is a BDA resource that allows you to modify and organize output configurations. Each project can contain standard output configurations for documents, images, video, and audio, as well as custom output blueprints for documents, audio, and images. Projects are referenced in the `InvokeDataAutomationAsync` API call to instruct BDA on how to process the files. For more information about projects and their use cases, see [Bedrock Data Automation projects](bda-projects.md). # Bedrock Data Automation projects One way to process files using Amazon Bedrock data automation (BDA) is to create a project. A project is a grouping of both standard and custom output configurations. Standard outputs are required in projects, but custom outputs are optional. When you call the `InvokeDataAutomationAsync` API with a project ARN, the file is automatically processed using the configuration settings defined in that project. Output is then generated based on the project's configuration. A project can be given a stage, either `LIVE` or `DEVELOPMENT`. Each stage is a unique and mutable version of the project. This means you can edit or test with the `DEVELOPMENT` stage, and process customer requests using the `LIVE` stage. `DEVELOPMENT` projects can't be accessed in the console, and must be changed and invoked through the API. A project allows you to use a single resource for multiple file types. For example, an audio file sent to BDA using project name ABC will be processed using project ABC’s audio standard output configuration. A document sent to BDA using project name ABC will be processed using project ABC’s document standard output configuration. Projects grant you greater flexibility when setting up standard outputs. Each standard output has its own set of configurable options, such as transcripts or summaries, and projects allow you to change those options to better suit your use case. You can also configure a project with Blueprints for documents, audio, and images to define custom output. A project configured to generate custom output will also generate standard output automatically. The following sections will go through a few examples of using projects. ## Using Projects with Standard Output Let's consider a use case in which you're only interested in extracting transcript summaries of your full audio and video files. By default, when you send audio and video files to BDA, you receive the transcript summaries along with full transcripts, scene level summaries, detected text, and other information. For this use case, you don't want to spend the extra time and resources to collect information you don't need. For this use case, you can configure a standard output project to enable only the summary feature for audio and video files. To do this using the API or the console, create a project and modify the standard output settings for audio and video. For video, enable **Full Video Summary** but ensure that other extractions (e.g., Full Audio Transcript, Scene Summaries, Content Moderation, etc.) are disabled. Repeat this configuration for audio. After you configure the project to generate only summaries, save the project and note the project's Amazon Resource Names (ARN). This ARN can be used for the `InvokeDataAutomationAsync` operation to process your files at scale. By passing an audio or video file to the BDA and specifying this project ARN, you will receive an output of only the summaries for each of the files. Note, in this example there was no configuration performed for documents or images. This means that if you pass an image or document to BDA using that project ARN, you will receive the default standard output for those files. ## Using projects with custom output and standard output For this use case, let's assume that you want to generate standard output summaries for documents and audio files, and to also extract custom fields from your documents. After you create a project, configure the standard output for audio to enable **Full audio summary** and ensure that other extractions are not enabled. Repeat this standard output configuration for documents. You can then configure custom output for documents by adding a new blueprint or a preexisting blueprint from the BDA global catalog. Documents passed to BDA using this project ARN will generate the standard output full document summaries and the blueprint output for defined custom fields. Audio files passed to BDA using this project ARN will generate full summaries. When processing documents, you might want to use multiple blueprints for different kinds of documents that are passed to your project. A project can have up to 40 document blueprints attached. BDA automatically matches your documents to the appropriate blueprint that's configured in your project, and generates custom output using that blueprint. Additionally, you might want to pass documents in bulk. If you pass a file that contains multiple documents, you can choose to split the document when creating your project. If you choose to do this, BDA scans the file and splits it into individual documents based on context. Those individual documents are then matched to the correct blueprint for processing. Currently, images only support a single blueprint definition per project. Image file types JPG and PNG might be treated as images or as scanned documents based on their contents. We recommend that you create a custom blueprint for images when you process custom output for documents so BDA provides the desired output for image files that contain text. Audio files also only support a single blueprint definition per project. # Splitting documents while using projects Amazon Bedrock Data Automation (BDA) supports splitting documents when using the Amazon Bedrock API. When enabled, splitting allows BDA to take a PDF containing multiple logical documents and split it into separate documents for processing. Once splitting is complete, each segment of the split document is processed independently. This means an input document can contain different document types. For example, if you have a PDF containing 3 bank statements and one W2, splitting would attempt to divide it into 4 separate documents that would be processed individually. BDA automatic splitting supports files with up to 3000 pages, and supports individual documents of up to 20 pages each. The option to split documents is off by default, but can be toggled on when using the API. Below is an example of creating a project with the splitter enabled. The ellipsies represent additional blueprints provided to the project. ``` response = client.create_data_automation_project( projectName=project_name, projectDescription="Provide a project description", projectStage='LIVE', standardOutputConfiguration=output_config, customOutputConfiguration={ 'blueprints': [ { 'blueprintArn': Blueprint ARN, 'blueprintStage': 'LIVE' }, ... ] }, overrideConfiguration={'document': {'splitter': {'state': 'ENABLED'}}} ) ``` The part that enables the splitting process is the overrideConfiguration line. This line sets up the splitter and allows you to pass multiple documents within the same file. Documents are split by the semantic boundaries in the document. Document splitting happens independently of applying blueprints, and documents that are split will be matched to the closest blueprint. For more information on how BDA matches blueprints see [Understanding blueprint matching](#bda-blueprint-matching). ## Understanding blueprint matching Blueprint matching is based on the following elements: + Blueprint name + Blueprint description + Blueprint fields When processing documents, you can provide multiple blueprints to match against. This allows processing different document types with appropriate blueprints. You can provide multiple blueprint IDs when invoking the data automation API, and BDA will attempt to match each document to the best fitting blueprint. This allows processing mixed document types in a single batch. This is useful when documents are expected to be of different types (e.g. bank statements, invoices, passports). If you need separate blueprints because document formats are very different or require specialized prompts, creating one blueprint per document type can help with matching. For more information on creating useful blueprints, see [Best practices for creating blueprints](#bda-blueprint-best-practices). ## Best practices for creating blueprints Follow the following best practices to get the most out of your blueprints: + Be explicit and detailed in blueprint names and descriptions to aid matching + Providing multiple relevant blueprints allows BDA to select the best match. Create separate blueprints for significantly different document formats + Consider creating specialized blueprints for every vendor/document source, if you need maximum accuracy + Do not include two blueprints of the same type in a project (e.g. two W2 blueprints). Information from the document itself and the blueprint is used to process documents, and including multiple blueprints of the same type in a project will lead to worse performance. By leveraging document splitting and multiple blueprint matching, BDA can more flexibly handle varied document sets while applying the most appropriate extraction logic to each document. # Disabling modalities and routing file types By default, projects in BDA process supported file types, by sorting them to different semantic modalities. When creating or editing your project, you can modify what modalities will be processed, and which file types will be sent to which modalities. In this section, we will go through enabling and disabling different modalities, routing files to specific modalities, and the default routing procedure for BDA. ## Disabling modality processing When you create a project, you might have a use case in mind that doesn't include processing all kinds of files. For example, you may want to only process documents and audio files. If that is the case, you don't want BDA to send a JPEG to be processed as an image or an MP4 to be processed as a video. Modality enablement allows you to turn off certain modalities in a project, curating responses from BDA's processing. **Disabling modalities with the BDA Console** When using the BDA console, modality enablement is handled by a checklist, where you can simply select or deselect each modality while editing or creating your Project. These options are located under the Advanced settings tab. At least one Modality must be selected for a project. ![\[Modality enablement options with checkboxes for document, image, video, and audio.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/modalityenableconsole.png) **Disabling modalities with the BDA API** When using the BDA API, modality enablement is handled by the `overrideConfiguration` request element, located in the `CreateDataAutomation` operation. Each modality has an associate section where you can declare the modality `ENABLED` or `DISABLED`. Below is an example of the `overrideConfiguration` element with only document and audio modalities enabled. The `modalityProcessing` flag defaults to `ENABLED`. ``` "overrideConfiguration" : { "document": { "splitter": { "state": ENABLED }, "modalityProcessing": { "state": ENABLED }, }, "image": { "modalityProcessing": { "state": DISABLED } }, "video": { "modalityProcessing": { "state": DISABLED } }, "audio": { "modalityProcessing": { "state": ENABLED } }, ... } ``` The ellipsies at the end of this section indicates the removal of the `modalityRouting` element, which we will discuss more in the next section. ## Routing files to certain processing types Certain file types are capable of being routed to different modalities, based on a variety of factors. With modality routing you can set certain file types to route to certain modality processing manually. JPEGs and PNGs can be routed to either document or image processing. MP4s and MOVs can be routed to either video or audio processing. **Routing with the BDA Console** While in the Advanced settings tab when creating or editing a blueprint, you can choose to add a new manual modality routing. This lets you select one of the 4 available file types and then which processing modality they will be routed to. Below is a screenshot of the console, with a manual modality routing that sends PNG files to the document processing modality. **Note** Settings for JPEG files apply to both ".jpeg" and ".jpg" files. Settings for MP4 settings apply to both ".mp4" ".m4v" files. ![\[An image from the AWS console, showing a pair of drop down menus labeled file type and modality destination, selected as PNG and Document.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/manualrouting.png) **Routing with the BDA API** Similar to modality enablement, modality routing is handled via the `overrideConfiguration` request element. Below is an example of the `modalityRouting` portion of `overrideConfiguration`. This example assumes all modalities are enabled, and routes JPEG and PNG files to the document modality, and MP4 and MOV files to the audio modality. ``` ... "modalityRouting": { "jpeg": DOCUMENT, "png": DOCUMENT, "mp4": AUDIO, "mov": AUDIO } } ``` The ellipsies at the beginning of the example indicate the removal of the rest of the `overrideConfiguration`, which is discussed more in the section on modality enablement and the document splitter functionality. ## Standard routing for the InvokeDataAutomationAsync API Without setting up your own routing procedures, BDA uses a standard set of procedures based on file type to determine what modality BDA will route to. The default procedures are listed in the table below. PNGs and JPEGs list Semantic Classifer as a default behavior. This means that BDA will look at indicators of whether or not a submitted file is an image or a document using internal models and perform routing automatically. | File Types | Default Routing Behavior | | --- | --- | | PNG | Semantic Classifer; Either Image or Document | | JPEG | Semantic Classifer; Either Image or Document | | PDF, TIFF | Document | | MP4, MOV | Video | | AMR, FLAC, M4A, MP3, OGG, WEBM, WAV | Audio | ## Standard routing for the InvokeDataAutomation API The [InvokeDataAutomation](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomation.html) API will also look at indicators of whether or not a submitted PNG or JPEG file is an image or a document using internal models and perform routing automatically. The default procedures are listed in the table below. PNGs and JPEGs list Semantic Classifier as a default behavior. This means that BDA will look at indicators of whether or not a submitted file is an image or a document using internal models and perform routing automatically. PDF and TIFF files will be routed to Documents modality for processing. InvokeDataAutomation API does not currently support Audio and Video files. | File Types | Default Routing Behavior | | --- | --- | | PNG | Semantic Classifer; Either Image or Document | | JPEG | Semantic Classifer; Either Image or Document | | PDF, TIFF | Document | # Cross Region support required for Bedrock Data Automation BDA requires users to use cross Region inference support when processing files. With cross-Region inference, Amazon Bedrock Data Automation will automatically select the optimal Region within your geography (as shown in the table below) to process your inference request, maximizing available compute resources and model availability, and providing the best customer experience. There's no additional cost for using cross-Region inference. Cross-Region inference requests are kept within the AWS Regions that are part of the geography where the data originally resides. For example, a request made within the US is kept within the AWS Regions in the US. Although the data remains stored only in the source Region, when using cross-Region inference, your requests and output results may move outside of your primary Region. All data will be encrypted while transmitted across Amazon's secure network. The following table includes the ARNs for different inference profiles. Replace account id with the account id you're using. | Source Region | Amazon Resource Name (ARN) | Supported Regions | | --- | --- | --- | | US East (N. Virginia) | arn:aws:bedrock:us-east-1:*account id*:data-automation-profile/us.data-automation-v1 | us-east-1 us-east-2 us-west-1 us-west-2 | | US West (Oregon) | arn:aws:bedrock:us-west-2:*account id*:data-automation-profile/us.data-automation-v1 | us-east-1 us-east-2 us-west-1 us-west-2 | | Europe (Frankfurt) | arn:aws:bedrock:eu-central-1:*account id*:data-automation-profile/eu.data-automation-v1 | eu-central-1 eu-north-1 eu-south-1 eu-south-2 eu-west-1 eu-west-3 | | Europe (Ireland) | arn:aws:bedrock:eu-west-1:*account id*:data-automation-profile/eu.data-automation-v1 | eu-central-1 eu-north-1 eu-south-1 eu-south-2 eu-west-1 eu-west-3 | | Europe (London) | arn:aws:bedrock:eu-west-2:*account id*:data-automation-profile/eu.data-automation-v1 | eu-west-2 | | Asia Pacific (Mumbai) | arn:aws:bedrock:ap-south-1:*account id*:data-automation-profile/apac.data-automation-v1 | ap-northeast-1 ap-northeast-2 ap-northeast-3 ap-south-1 ap-south-2 ap-southeast-1 ap-southeast-2 ap-southeast-4 | | Asia Pacific (Sydney) | arn:aws:bedrock:ap-southeast-2:*account id*:data-automation-profile/apac.data-automation-v1 | ap-northeast-1 ap-northeast-2 ap-northeast-3 ap-south-1 ap-south-2 ap-southeast-1 ap-southeast-2 ap-southeast-4 | | AWS GovCloud (US-West) | arn:aws:bedrock:gov-cloud:*account id*:data-automation-profile/us-gov.data-automation-v1 | us-gov-west-1 | Below is an example IAM policy for processing documents with CRIS enabled for `us-east-1` or `us-west-2`. ``` {"Effect": "Allow", "Action": ["bedrock:InvokeDataAutomationAsync"], "Resource": [ "arn:aws:bedrock:us-east-1:account_id:data-automation-profile/us.data-automation-v1", "arn:aws:bedrock:us-east-2:account_id:data-automation-profile/us.data-automation-v1", "arn:aws:bedrock:us-west-1:account_id:data-automation-profile/us.data-automation-v1", "arn:aws:bedrock:us-west-2:account_id:data-automation-profile/us.data-automation-v1"]} ``` # Standard output in Bedrock Data Automation Standard output is the default way of interacting with Amazon Bedrock Data Automation (BDA). If you pass a document to the BDA API with no established blueprint or project it returns the default standard output for that file type. Standard output can be modified using projects, which store configuration information for each data type. You can have one standard output configuration per data type for each project. BDA always provides a standard output response even if it's alongside a custom output response. Each data type has different standard output options. Some of these options are part of the default Bedrock Data Automation response, while some exist only as toggles for working with the data type in a project. The following sections go over each data type's unique response options, noting which are defaults and which are optional. # Documents Standard output for documents lets you set the granularity of response you're interested in as well as establishing output format and text format in the output. Below are some of the outputs you can enable. **Note** BDA can process DOCX files. To process DOCX files, they are converted into PDFs. This means page number mapping will not work for DOCX files. Images of the converted PDFs will be uploaded to your output bucket if the if the JSON\$1 option and page granularity are selected. ## Response Granularity Response granularity determines what kind of response you want to receive from document text extraction. Each level of granularity gives you more and more separated responses, with page providing all of the text extracted together, and word providing each word as a separate response. The available granularity levels are: + Page level granularity – This is enabled by default. Page level granularity provides each page of the document in the text output format of your choice. If you're processing a PDF, enabling this level of granularity will detect and return embedded hyperlinks. + Element level granularity (Layout) – This is enabled by default. Provides the text of the document in the output format of your choice, separated into different elements. These elements, such as figures, tables, or paragraphs. These are returned in logical reading order based off the structure of the document. If you're processing a PDF, enabling this level of granularity will detect and return embedded hyperlinks. + Word level granularity – Provides information about individual words without using broader context analysis. Provides you with each word and its location on the page. ## Output Settings Output settings determine the way your downloaded results will be structured. This setting is exclusive to the console. The options for output settings are: + JSON – The default output structure for document analysis. Provides a JSON output file with the information from your configuration settings. + Async [InvokeDataAutomationAsync](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomationAsync.html) API: JSON output for Async API is S3 only. + Sync [InvokeDataAutomation](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomation.html) API: JSON output can be set to S3 or inline by leveraging `outputconfiguration`. If S3 is selected, then output JSON goes to S3 only (not inline). If S3 not provided, Sync API output supports JSON inline only. + JSON\$1files – Only available for Async [InvokeDataAutomationAsync](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomationAsync.html) API. Using this setting generates both a JSON output and files that correspond with different outputs. For example, this setting gives you a text file for the overall text extraction, a markdown file for the text with structural markdown, and CSV files for each table that's found in the text. Figures located inside a document will be saved as well as figure crops and rectified images. Also, if you are processing a DOCX file and have this option selected the converted PDF of your DOCX file will be in the output folder. These outputs are located in `standard_output/logical_doc_id/assets/` in your output folder. **Note** The sync API does not output any additional files beyond the JSON. The output JSON contains only the text format that was selected as part of the Standard Output Text format. Sync API will not output Figure crops or rectified images. DocX not supported by Sync API. ## Text Format Text format determines the different kinds of texts that will be provided via various extraction operations. You can select any number of the following options for your text format. + Plaintext – This setting provides a text-only output with no formatting or other markdown elements noted. + Text with markdown – The default output setting for standard output. Provides text with markdown elements integrated. + Text with HTML – Provides text with HTML elements integrated in the response. + CSV – Provides a CSV structured output for tables within the document. This will only give a response for tables, and not other elements of the document. ## Bounding Boxes and Generative Fields For Documents, there are two response options that change their output based on the selected granularity. These are Bounding Boxes, and Generative Fields. Selecting Bounding Boxes will provide a visual outline of the element or word you click on in the console response dropdown. This lets you track down particular elements of your response more easily. Bounding Boxes are returned in your JSON as the coordinates of the four corners of the box. When you select Generative Fields, you are generated a summary of the document, both a 10 word and 250 word version. Then, if you select elements as a response granularity, you generate a descriptive caption of each figure detected in the document. Figures include things like charts, graphs, and images. ------ #### [ Async ] This section focuses on the different response objects you receive from running the API operation InvokeDataAutomationAsync on a document file. Below we'll break down each section of the response object and then see a full, populated response for an example document. The first section we'll receive is `metadata`. ``` "metadata":{ "logical_subdocument_id":"XXXX-XXXX-XXXX-XXXX", "semantic_modality":"DOCUMENT", "s3_bucket":"bucket", "s3_prefix":"prefix" }, ``` The first section above provides an overview of the metadata associated with the document. Along with the S3 information, this section also informs you which modality was selected for your response. ``` "document":{ "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "description":"document text", "summary":"summary text", "statistics":{ "element_count":5, "table_count":1, "figure_count":1, "word_count":1000, "line_count":32 } }, ``` The above section provides document level granularity information. The description and summary sections are the generated fields based on the document The representation section provides the actual content of the document with various formatting styles. Finally statistics contains information on the actual content of the document, such as how many semantic elements there are, how many figures, words, lines, etc. This is the information for a table entity. For InvokeDataAutomationAsync (async) request, in addition to location information, the different formats of the text, tables, and reading order, they specifically return csv information and cropped images of the table in S3 buckets. The CSV information shows the different headers, footers, and titles. The images will be routed to the s3 bucket of the prefix set in the InvokeDataAutomationAsync request. For InvokeDataAutomation (sync) request, csv and cropped image of the table in S3 buckets are not supported. When you process a PDF, the statistics section of the response will also contain `hyperlinks_count` which tells you how many hyperlinks exist in your document. ``` { "id":"entity_id", "type":"TEXT", "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "reading_order":2, "page_indices":[ 0 ], "locations":[ { "page_index":0, "bounding_box":{ "left":0.0, "top":0.0, "width":0.05, "height":0.5 } } ], "sub_type":"TITLE/SECTION_TITLE/HEADER/FOOTER/PARAGRAPH/LIST/PAGE_NUMBER" }, ``` This is the entity used for text within a document, indicated by the `TYPE` line in the response. Again representation shows the text in different formats. `reading_order` shows when a reader would logically see the text. This is a semantic ordering based on associated keys and values. For example, it associates titles of paragraphs with their respective paragraph in reading order. `page_indices` tells you which pages the text is on. Next is location information, with a provided text bounding box if it was enabled in response. Finally, we have the entity subtype. This subtype provides more detailed information on what kind of text is being detected. For a complete list of subtypes see the API Reference. ``` { "id":"entity_id", "type":"TABLE", "representation":{ "html":"table.../table", "markdown":"| header | ...", "text":"header \t header", "csv":"header, header, header\n..." }, "csv_s3_uri":"s3://", "headers":[ "date", "amount", "description", "total" ], "reading_order":3, "title":"Title of the table", "footers":[ "the footers of the table" ], "crop_images":[ "s3://bucket/prefix.png", "s3://bucket/prefix.png" ], "page_indices":[ 0, 1 ], "locations":[ { "page_index":0, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } }, { "page_index":1, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } } ], "sub_type":"TITLE/SECTION_TITLE/HEADER/FOOTER/PARAGRAPH/LIST/PAGE_NUMBER" }, ``` This is the information for a table entity. In addition to location information, the different formats of the text, tables, and reading order, they specifically return csv information and cropped images of the table in S3 buckets. The CSV information shows the different headers, footers, and titles. The images will be routed to the s3 bucket of the prefix set in the InvokeDataAutomation request. ``` { "id":"entity_id", "type":"FIGURE", "summary":"", "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "crop_images":[ "s3://bucket/prefix.png", "s3://bucket/prefix.png" ], "locations":[ { "page_index":0, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } } ], "sub_type":"CHART", "title":"figure title", "rai_flag":"APPROVED/REDACTED/REJECTED", "reading_order":1, "page_indices":[ 0 ] } , ``` This is the entity used for figures such as in document graphs and charts. Similar to tables, these figures will be cropped and images sent to the s3 bucket set in your prefix. Additionally, you'll receive a `sub_type` and a figure title response for the title text and an indication on what kind of figure it is. ``` "pages":[ { "id":"page_id", "page_index":0, "detected_page_number":1, "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "statistics":{ "element_count":5, "table_count":1, "figure_count":1, "word_count":1000, "line_count":32 }, "asset_metadata":{ "rectified_image":"s3://bucket/prefix.png", "rectified_image_width_pixels":1700, "rectified_image_height_pixels":2200 } } ], ``` The last of the entities we extract through standard output is Pages. Pages are the same as Text entities, but additionally contain page numbers, for which detected page number is on the page. ``` "text_lines":[ { "id":"line_id", "text":"line text", "reading_order":1, "page_index":0, "locations":{ "page_index":0, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } } } ], ``` ``` "text_words":[ { "id":"word_id", "text":"word text", "line_id":"line_id", "reading_order":1, "page_index":0, "locations":{ "page_index":0, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } } } ] ``` These final two elements are for individual text portions. Word level granularity returns a response for each word, while default output reports only lines of text. ------ #### [ Sync ] This section focuses on the different response objects you receive from running the API operation InvokeDataAutomation on a document file. Below we'll break down each section of the response object and then see a full, populated response for an example document. The first section we'll receive is `metadata`. ``` "metadata": { "logical_subdocument_id": "1", "semantic_modality": "DOCUMENT", "number_of_pages": X, "start_page_index": "1", "end_page_index": X, "file_type": "PDF" }, ``` The first section above provides an overview of the metadata associated with the document. Since the Synchronous InvokeDataAutomation API does not currently support document splitting, logical\$1subdocument\$1id is always equal to 1. ``` "document":{ "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "description":"document text", "summary":"summary text", "statistics":{ "element_count":5, "table_count":1, "figure_count":1, "word_count":1000, "line_count":32 } }, ``` The above section provides document level granularity information. The description and summary sections are the generated fields based on the document The representation section provides the actual content of the document with various formatting styles. Finally statistics contains information on the actual content of the document, such as how many semantic elements there are, how many figures, words, lines, etc. Note: Unlike the asynchronous InvokeDataAutomationAsync request, the synchronous InvokeDataAutomation request does not support returning csv information and cropped image of the table in S3 buckets. ``` { "id":"entity_id", "type":"TEXT", "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "reading_order":2, "page_indices":[ 0 ], "locations":[ { "page_index":0, "bounding_box":{ "left":0.0, "top":0.0, "width":0.05, "height":0.5 } } ], "sub_type":"TITLE/SECTION_TITLE/HEADER/FOOTER/PARAGRAPH/LIST/PAGE_NUMBER" }, ``` This is the entity used for text within a document, indicated by the TYPE line in the response. Again representation shows the text in different formats. reading\$1order shows when a reader would logically see the text. This is a semantic ordering based on associated keys and values. For example, it associates titles of paragraphs with their respective paragraph in reading order. page\$1indices tells you which pages the text is on. Next is location information, with a provided text bounding box if it was enabled in response. Finally, we have the entity subtype. This subtype provides more detailed information on what kind of text is being detected. For a complete list of subtypes see the API Reference. ``` { "id": "entity_id", "type": "TABLE", "representation": { "html": "table.../table", "markdown": "| header | ...", "text": "header \t header", "csv": "header, header, header\n..." }, "headers": ["date", "amount", "description", "total"], "reading_order": 3, "title": "Title of the table", "footers": ["the footers of the table"], "page_indices": [0, 1], "locations": [{ "page_index": 0, "bounding_box": { "left": 0, "top": 0, "width": 1, "height": 1 } }, { "page_index": 1, "bounding_box": { "left": 0, "top": 0, "width": 1, "height": 1 } }] }, ``` This is the information for a table entity. The CSV information shows the different headers, footers, and titles. ``` { "id": "entity_id", "type": "FIGURE", "summary": "", "representation": { "text": "document text", "html": "document title document content", "markdown": "# text" }, "locations": [ { "page_index": 0, "bounding_box": { "left": 0, "top": 0, "width": 1, "height": 1 } } ], "sub_type": "CHART", "title": "figure title", "reading_order": 1, "page_indices": [ 0 ] }, ​ ``` This is the entity used for figures such as in document graphs and charts. You'll receive a `sub_type` and a figure title response for the title text and an indication on what kind of figure it is. ``` "pages":[ "pages":[ { "id":"page_id", "page_index":0, "detected_page_number":1, "representation":{ "text":"document text", "html":"document title document content", "markdown":"# text" }, "statistics":{ "element_count":5, "table_count":1, "figure_count":1, "word_count":1000, "line_count":32 }, "asset_metadata":{ "rectified_image":"s3://bucket/prefix.png", "rectified_image_width_pixels":1700, "rectified_image_height_pixels":2200 } } ], ``` The last of the entities we extract through standard output is Pages. Pages are the same as Text entities, but additionally contain page numbers, for which detected page number is on the page. ``` "text_lines":[ { "id":"line_id", "text":"line text", "reading_order":1, "page_index":0, "locations":{ "page_index":0, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } } } ], ``` ``` "text_words":[ { "id":"word_id", "text":"word text", "line_id":"line_id", "reading_order":1, "page_index":0, "locations":{ "page_index":0, "bounding_box":{ "left":0, "top":0, "width":1, "height":1 } } } ] ``` These final two elements are for individual text portions. Word level granularity returns a response for each word, while default output reports only lines of text. ------ ## Additional file format metadata JSON When you receive your additional files from the additional file formats flag, you will get a JSON file for any rectified images that are extracted. BDA rectifies rotated images by using a homography to rotate the image to be at a 90 degree angle. An example of the JSON is below: ``` "asset_metadata": { "rectified_image": "s3://bucket/prefix.png", "rectified_image_width_pixels": 1700, "rectified_image_height_pixels": 2200, "corners": [ [ 0.006980135689736235, -0.061692718505859376 ], [ 1.10847711439684, 0.00673927116394043 ], [ 0.994479346419327, 1.050548828125 ], [ -0.11249661383904497, 0.9942819010416667 ] ] } ``` Corners represent the detected corners of an image, used to form a homography of the document. This homography is used to rotate the image while maintaining its other properties. # Images The Amazon Bedrock Data Automation (BDA) feature offers a comprehensive set of standard outputs for image processing to generate insights from your images. You can use these insights to enable a wide range of applications and use cases, such as content discovery, contextual ad placement, and brand safety. Here's an overview of each operation type available as part of standard outputs for images: ## Image Summary Image summary generates a descriptive caption for an image. This feature is enabled within the standard output configuration by default. ## IAB Taxonomy The Interactive Advertising Bureau (IAB) classification applies a standard advertising taxonomy to classify image content. For Preview, BDA will support 24 top-level (L1) categories and 85 second-level (L2) categories. To download the list of IAB categories supported by BDA, click [here](samples/iab-taxonomy.zip). ## Logo Detection This feature identifies logos in an image and provides bounding box information, indicating the coordinates of each detected logos within the image, and confidence scores. This feature is not enabled by default. ## Image Text Detection This feature detects and extracts text that appears visually in an image and provides bounding box information, indicating the coordinates of each detected text element within the image, and confidence scores. This feature is enabled within the standard output configuration by default. ## Content Moderation Content moderation detects inappropriate, unwanted, or offensive content in an image. For Preview, BDA will support 7 moderation categories: Explicit, Non-Explicit Nudity of Intimate parts and Kissing, Swimwear or Underwear, Violence, Drugs & Tobacco, Alcohol, Hate symbols. Explicit text in images is not flagged. Bounding boxes and the associated confidence scores can be enabled or disabled for relevant features like text detection to provide location coordinates in the image. By default, image summary and image text detection are enabled. ## Image Standard Output The following is an example of a standard output for an image processed through BDA. Each section has been shortened and separated with an explanation. ``` { "metadata": { "id": "image_123", "semantic_modality": "IMAGE", "s3_bucket": "my-s3-bucket", "s3_prefix": "images/", "image_width_pixels": 1920, "image_height_pixels": 1080, "color_depth": 24, "image_encoding": "JPEG" }, ``` The first part of a response is the metadate of an image. It gives you the file name, encoding type, s3 bucket location and further information about the content. ``` "image": { "summary": "Lively party scene with decorations and supplies", ``` At the beginning of the response is the generative summary of the image. ``` "iab_categories": [ { "id": "iab_12345", "type": "IAB", "category": "Party Supplies", "confidence": 0.9, "parent_name": "Events & Attractions", "taxonomy_level": 2 }, { "id": "iab_67890", "type": "IAB", "category": "Decorations", "confidence": 0.8, "parent_name": "Events & Attractions", "taxonomy_level": 1 } ], ``` Next, we see the IAB catagories attached to a response. These represent different types of advertising classifications, using the standard IAB taxonomy. Each one has a confidence score, taxonomy\$1level, and parent\$1name for the general high level catagory. ``` "content_moderation": [ { "id": "mod_12345", "type": "MODERATION", "category": "Drugs & Tobacco Paraphernalia & Use", "confidence": 0.7, "parent_name": "Drugs & Tobacco", "taxonomy_level": 2 } ], ... ``` Content moderation contains information about possible explicit content in an image. These each have a confidence score and category, aligning with the content moderation categories discussed earlier in the section. ``` "text_words": [ { "id": "word_1", "text": "lively", "confidence": 0.9, "line_id": "line_1", "locations": [ { "bounding_box": { "left": 100, "top": 200, "width": 50, "height": 20 }, "polygon": [ {"x": 100, "y": 200}, {"x": 150, "y": 200}, {"x": 150, "y": 220}, {"x": 100, "y": 220} ] } ] }, ... ``` This section breaks down each detected word within an image, including confidence and an on screen location within the image. It also flags which line the word is in, using `line_id`. ``` "text_lines": [ { "id": "line_1", "text": "lively party", "confidence": 0.9, "locations": [ { "bounding_box": { "left": 100, "top": 200, "width": 200, "height": 20 }, "polygon": [ {"x": 100, "y": 200}, {"x": 300, "y": 200}, {"x": 300, "y": 220}, {"x": 100, "y": 220} ] } ] } ] }, ``` Here, words are detected in their collective lines, with confidence score and bounding box. ``` "statistics": { "entity_count": 7, "object_count": 3, "line_count": 2, "word_count": 9 } } ``` Finally, we have statistics. These break down all content within an image, including object # Videos BDA offers a set of standard outputs to process and generate insights for videos. Here's a detailed look at each operation type: ## Full Video Summary Full video summary generates an overall summary of the entire video. It distills the key themes, events, and information presented throughout the video into a concise summary. Full video summary is optimized for content with descriptive dialogue such as product overviews, trainings, news casts, talk shows, and documentaries. BDA will attempt to provide a name for each unique speaker based on audio signals (e.g., the speaker introduces themself) or visual signals (e.g., a presentation slide shows a speaker’s name) in the full video summaries and the scene summaries. When a unique speaker’s name is not resolved they will be represented by a unique number (e.g., speaker\$10). ## Chapter Summaries Video chapter summarization provides descriptive summaries for individual scenes within a video. A video chapter is a sequence of shots that form a coherent unit of action or narrative within the video. This feature breaks down the video into meaningful segments based on visual and audible cues, provides timestamps for those segments, and summarizes each. ## IAB Taxonomy The Interactive Advertising Bureau (IAB) classification applies a standard advertising taxonomy to classify video scenes based on visual and audio elements. For Preview, BDA will support 24 top-level (L1) categories and 85 second-level (L2) categories. To download the list of IAB categories supported by BDA, click [here](samples/iab-taxonomy.zip). ## Full Audio Transcript The full audio transcript feature provides a complete text representation of all speech in the audio file. It uses advanced speech recognition technology to accurately transcribe dialogue, narration, and other audio elements. The transcription includes speaker identification, making it easy to navigate and search through the audio content based on the speaker. ## Text in Video This feature detects and extracts text that appears visually in the video. It can identify both static text (like titles or captions) and dynamic text (such as moving text in graphics). Similar to image text detection, it provides bounding box information for each detected text element, allowing for precise localization within video frames. ## Logo Detection This feature identifies logos in a video and provides bounding box information, indicating the coordinates of each detected logos within the video frame, and confidence scores. This feature is not enabled by default. ## Content Moderation Content moderation detects inappropriate, unwanted, or offensive content in a video. BDA supports 7 moderation categories: Explicit, Non-Explicit Nudity of Intimate parts and Kissing, Swimwear or Underwear, Violence, Drugs & Tobacco, Alcohol, Hate symbols. Explicit text in videos is not flagged. Bounding boxes and the associated confidence scores can be enabled or disabled for relevant features like text detection, to provide location coordinates and timestamps in the video file. By default, full video summarization, scene summarization, and video text detection are enabled. **Note** Only one audio track per video is supported. Subtitle file formats (e.g., SRT, VTT, etc.) are not supported. ## Video Standard Output The following is an example of a standard output for a video processed through BDA: ``` { "metadata": { "asset_id": "0", "semantic_modality": "VIDEO", "s3_bucket": "bedrock-data-automation-gamma-assets-us-east-1", "s3_key": "demo-assets/Video/MakingTheCut.mp4", "format": "QuickTime / MOV", "frame_rate": 30, "codec": "h264", "duration_millis": 378233, "frame_width": 852, "frame_height": 480 }, ``` This initial section dicusses metadata information regarding the video. This includes the bucket location, format, frame rate and other key pieces of info. ``` "shots": [ ... { "shot_index": 3, "start_timecode_smpte": "00:00:08:19", "end_timecode_smpte": "00:00:09:25", "start_timestamp_millis": 8633, "end_timestamp_millis": 9833, "start_frame_index": 259, "end_frame_index": 295, "duration_smpte": "00:00:01:06", "duration_millis": 1200, "duration_frames": 36, "confidence": 0.9956437242589935, "chapter_indices": [ 1 ] }, ``` This is an example of a shot element in a response. Shots represent small portions of a video, typically associate with an edit or cut in the video. Shots contain start and end elements, and also a chapter\$1indicies element. This element indicates which larger section of the video, called a chapter, the shot is a part of. ``` "chapters": [ { "start_timecode_smpte": "00:00:00:00", "end_timecode_smpte": "00:00:08:18", "start_timestamp_millis": 0, "end_timestamp_millis": 8600, "start_frame_index": 0, "end_frame_index": 258, "duration_millis": 8600, "shot_indices": [ 0, 1, 2 ], "summary": "At an elegant outdoor venue, a man in a suit and a woman in a patterned dress stand on a raised platform overlooking a reflective pool. The setting is adorned with palm trees and lush greenery, creating a tropical atmosphere. The man initiates the event by asking if they should begin, to which the woman responds affirmatively. As the scene progresses, the focus shifts to a woman wearing a distinctive black and white patterned coat, her hair styled in a bun. She stands alone in a dimly lit room, facing away from the camera. The narrative then moves to a formal setting where a man in a dark suit stands before a curtain backdrop, suggesting he may be about to address an audience or perform. The scene concludes with a view of the entire venue, showcasing its tropical charm with a swimming pool surrounded by palm trees and decorative lighting, indicating it's prepared for a special occasion.", ``` Chapters are larger pieces of a video. Then contain start and end information like shots, and a shot\$1indicies element. shot\$1indicies tell you which shots are within a chapter. Finally, the summary element, provides a generated summary of the content of the chapter. ``` "frames": [... { "timecode_smpte": "00:00:03:15", "timestamp_millis": 3500, "frame_index": 105, "content_moderation": [], "text_words": [ { "id": "266db64a-a7dc-463c-b710-7a178a2cc4cc", "type": "TEXT_WORD", "confidence": 0.99844897, "text": "ANDREA", "locations": [ { "bounding_box": { "left": 0.1056338, "top": 0.7363281, "width": 0.19806337, "height": 0.068359375 }, "polygon": [ { "x": 0.1056338, "y": 0.7363281 }, { "x": 0.30369717, "y": 0.7363281 }, { "x": 0.30369717, "y": 0.8046875 }, { "x": 0.1056338, "y": 0.8046875 } ] } ], "line_id": "57b760fc-c410-418e-aee3-7c7ba58a71c2" }, ``` The smallest granularity of a video is a frame, representing a single image within a video. Frames have two notably response elements, content\$1moderation and text\$1words. The first, content\$1moderation provides you with information based on content moderation catagories about the content of the frame if any are detected. The second, text\$1words, provides you with a location and information about any text appearing in a video, such as closed captioning. ``` "statistics": { "shot_count": 148, "chapter_count": 11, "speaker_count": 11 } } ``` Finally, statistics provides a breakdown of information about the detection, such as how many shots, speakers, and chapters are in a given video. # Audio The Amazon Bedrock Data Automation (BDA) feature offers a set of standard output to process and generate insights for audio files. Here's a detailed look at each operation type: ## Full Audio Summary Full audio summary generates an overall summary of the entire audio file. It distills the key themes, events, and information presented throughout the audio into a concise summary. ## Full Audio Transcript The full audio transcript feature provides a complete text representation of all spoken content in the audio. It uses advanced speech recognition technology to accurately transcribe dialogue, narration, and other audio elements. The transcription includes time-stamping, making it easy to navigate and search through audio content based on spoken words. ### Speaker and Channel Labeling For the generated transcript, you can enable channel and/or speaker labeling. This will give each channel or speaker a number, and then indicate within the transcript when a channel is being used and a particular speaker is speaking. This label appears within the response as "spk\$1" followed by a unique number for each speaker, up to 30 speakers. The first speaker would be "spk\$10", "spk\$11" and so on. Audio channels are indicated similarly with the first channel labeled as "ch\$10", but can only be labeled up two channels. ## Topic Summary Audio topic summary separates the audio file into sections called topics, and summarizes them to provide key information. These topics are given timestamps to help place them in the audio file as a whole. This feature is not enabled by default. ## Content Moderation Content moderation uses audio and text-based cues to identify and classify voice-based toxic content into seven different categories: + ****Profanity****: Speech that contains words, phrases, or acronyms that are impolite, vulgar, or offensive. + ****Hate speech:**** Speech that criticizes, insults, denounces, or dehumanizes a person or group on the basis of an identity (such as race, ethnicity, gender, religion, sexual orientation, ability, and national origin). + ****Sexual: ****Speech that indicates sexual interest, activity, or arousal using direct or indirect references to body parts, physical traits, or sex. + ****Insults:**** Speech that includes demeaning, humiliating, mocking, insulting, or belittling language. This type of language is also labeled as bullying + ****Violence or threat: ****Speech that includes threats seeking to inflict pain, injury, or hostility toward a person or group. + ****Graphic:**** Speech that uses visually descriptive and unpleasantly vivid imagery. This type of language is often intentionally verbose to amplify a recipient's discomfort. + ****Harassment or abusive:**** Speech intended to affect the psychological well-being of the recipient, including demeaning and objectifying terms. This type of language is also labeled as harassment. ## Audio Standard Output This section focuses on the different response objects you receive from running the API operation InvokeDataAutomation on an audio file. Below we'll break down each section of the response object and then see a full, populated response for an example document. The first section we'll receive is `metadata`. ``` "metadata": { "asset_id": "0", "semantic_modality": "AUDIO", "s3_bucket": "bedrock-data-automation-gamma-assets-us-east-1", "s3_key": "demo-assets/Audio/AWS_TCA-Call-Recording-2.wav", "sample_rate": 8000, "bitrate": 256000, "number_of_channels": 2, "codec": "pcm_s16le", "duration_millis": 237560, "format": "wav", "dominant_asset_language": "EN", "generative_output_language": "DEFAULT/EN" } ``` This section breaks down information about the file such as its s3 location, bitrate, audio channels, and format. Next we look at `audio_items`. `dominant_asset_language` indicaties what language is the most present in a piece of audio based on length in seconds. `generative_output_language` indicates which language the response output will be in. When set to "DEFAULT" it will use the dominant language. ``` "audio_items": [ { "item_index": 0, "audio_segment_index": 0, "content": "Auto", "start_timestamp_millis": 9, "end_timestamp_millis": 119 }, ... ] ``` The items section includes a breakdown of the audio file on a sound by sound basis. Each item is usually about word length. The item\$1index indicates the item's place in the audio\$1items indicies, and the audio\$1segment\$1index indicates where it sits in the segements indicies which we'll discuss next. ``` "audio_segments": [ { "start_timestamp_millis": 0, "end_timestamp_millis": 1970, "segment_index": 0, "type": "TRANSCRIPT", "text": "Auto sales, Cherry speaking. How can I help you?", "speaker": { "speaker_label": "spk_0" }, "channel": { "channel_label": "ch_0" }, "audio_item_indices": [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 ], "language": "EN" }, ... ] ``` Here we get a breakdown of the file based on longer spans of time, with each segment equal to roughly a sentence. It tells us which audio items are included in the segment and the text of the segment itself. Next let's look at content moderation. With speaker and channel labeling enabled, you can see the `speaker_label` and `channel_label` sections which indicate which speaker and channel are present in this segment. ``` "content_moderation": [ { "id": "93068e72-290d-4aad-8717-a2cd0e02b0d0", "type": "AUDIO_MODERATION", "confidence": 0.0476, "start_timestamp_millis": 0, "end_timestamp_millis": 1970, "moderation_categories": [ { "category": "profanity", "confidence": 0.1582 }, ... ] }, ... ] ``` The content moderation section goes through each individual segment and analyzes it for any of the seven moderation catagories, providing confidence scores for each section. Our next section is topics. ``` "topics": [ { "topic_index": 0, "start_timestamp_millis": 0, "end_timestamp_millis": 36790, "summary": "As follows:\n\nSuzanne, a customer, recently had her Hyundai serviced at the auto sales shop where Carrie works. Suzanne had a 3 p.m. appointment and got her car serviced, which included an oil change and filter changes. However, when Suzanne left the shop, her oil light was still on, which she found concerning. Carrie acknowledged that this sometimes happens, even after a service visit, and assured Suzanne that she would look into the issue further.", "transcript": { "representation": { "text": "Auto sales, Cherry speaking. How can I help you? Yeah, hi Carrie, um, my name is Suzanne. I literally just left your shop. Um, I just went in and got my Hyundai service. Um, it just was, it just needed like filter error changes oil change and all that kind of stuff, um, but. When I left and my oil light is still on and I don't know why. Got it. You just got it serviced here, but when you drove off the light was still on. Is that what happened? Yeah, yeah, yeah, like I literally like I had a 3 p.m. appointment and I just got it, you know, believe it or not, this, this happens." } }, "audio_segment_indices": [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] }, ... ] ``` Topic sections are the next level of granularity from segements. These are groupings of segements are broken down roughly by concepts. Each topic comes with a generated summary of the topic, and the exact test of the topic. Our final part of a response is statistics. ``` "statistics": { "word_count": 749, "topic_count": 4 } ``` This section summarizes the information about the audio file. This includes word count and total topics. # Custom output and blueprints When using Amazon Bedrock Data Automation (BDA) you can further fine tune your extractions using custom output configuration. Custom outputs are configured with artifacts called blueprints. Blueprints are a list of instructions for how to extract information from your file, allowing for transformation and adjustment of output. For more information and a detailed walkthrough of a blueprint, see [Blueprints](bda-blueprint-info.md). Custom output configuration also works alongside projects. When you pass a file to a BDA and reference a project with configured blueprint(s), BDA will process the file using the appropriate blueprint. This works for up to 40 document bluperints, one image blueprint, one audio blueprint, and/or one video blueprint. When working with multiple blueprints, BDA attempts to send documents to the blueprint that best matches the expected layout. For more information about projects and custom output, see [Bedrock Data Automation projects](bda-projects.md). **Note** All files processed by custom output must follow the file restrictions for BDA. For more information on file restrictions see BDA Prerequisites. # Blueprints Blueprints are artifacts that you can use to configure your file processing business logic. Each blueprint consists of a list of field names that you can extract, the data format in which you want the response for the field to be extracted—such as string, number, or boolean—as well as natural language context for each field that you can use to specify data normalization and validation rules. You can create a blueprint for each class of file that you want to process, such as a W2, pay stub or ID card. Blueprints can be created using the console or the API. Each blueprint that you create is an AWS resource with its own blueprint ID and ARN. When using a blueprint for extraction, you can use a catalog blueprint or a custom created blueprint. If you already know the kind of file you're looking to extract from, catalog bluprints provide a premade starting place. You can create custom blueprints for files that aren't in the catalog. When creating a blueprint you can use several methods, such as a generated blueprint via the blueprint prompt, manual creation by adding individual fields, or creating the JSON of a blueprint using the JSON Editor. These can be saved to your account and shared. **Note** Audio blueprints cannot be created via Blueprint Prompts. A blueprint's maximum size is 100,000 characters, JSON formatted. For blueprints that are intended to be used with the [InvokeDataAutomationAsync](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomationAsync.html) API the maximum fields per blueprint is 100. For Blueprints that are intended to be used with the [InvokeDataAutomation](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomation.html) API the maximum fields per blueprint is 15. **Note** When using Blueprints you might find yourself using Prompts, either in fields or for Blueprint creation. Only allow trusted sources to control the prompt input. Amazon Bedrock is not responsible for validating the intent of the blueprint. ## Blueprint walkthrough Lets take an example of an ID document such as a passport and walk through a blueprint for this document. ![\[Sample passport with standard fields, demonstrating layout and data fields that will be extracted.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/passport2.png) Here is an example blueprint for this ID document that we created on the console. ![\[Table layout of passport field definitions, with various categories, showing an example blueprint.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bdapassport.png) At its core, a blueprint is a data structure that contains fields, which in turn contain the information extracted by BDA custom output. There are two types of fields—explicit and implicit—located in the extraction table. Explicit extractions are used for clearly stated information that can be seen in the document. Implicit extractions are used for information that need to be transformed from how they appear in the document. For example, you can remove the dashes from a social security number, converting from 111-22-3333 to 111223333. Fields contain certain basic components: + Field name: This is a name you can provide for each field that you want to extract from the document. You can use the name that you use for the field in your downstream system such as `Place_Birth` or `Place_of_birth`. + Description: This is an input that provides natural language context for each field in the blueprint to describe data normalization or validation rules to be followed. For example, `Date of birth in YYYY-MM-DD format` or `Is the year of birth before 1992?`. You can also use the prompt as a way to iterate on the blueprint and improve the accuracy of BDA’s response. Providing a detailed prompt that describes the field you need helps the underlying models to improve their accuracy. Prompts may be up to 300 characters long. + Results: The information extracted by BDA based on the prompt and field name. + Type: The data format that you want the response for the field to use. We support string, number, boolean, array of string, and array of numbers. + Confidence score: The percentage of certainty that BDA has that your extraction is accurate. Audio and Image blueprints do not return a confidence score. + Extraction Types: The type of extraction, either explict or inferred. + Page Number: Which page of the document that the result was found on. Audio and Video blueprints do not return a page number. In addition to simple fields, BDA custom output offers several options for use cases that you might encounter in document extraction: table fields, groups, and custom types. **Table Fields** When creating a field, you can choose to create a table field instead of a basic field. You can name the field and provide a prompt, as with other fields. You can also provide column fields. These fields have a column name, column description, and column type. When shown in the extraction table, a table field has the column results grouped beneath the table name. Table fields can only have up to 15 subfields. **Groups** A group is a structure that's used to organize several results into a single location within your extraction. When you create a group, you give the group a name and you can create and place fields into that group. This group is marked in your extractions table, and lists below it the fields that are within the group. **Custom types** You can create a custom type while editing a blueprint in the Blueprint Playground. Any field can be a custom type. This type has a unique name, and prompts the creation of the fields that make up the detection. An example would be creating a custom type called Address, and including in it the fields “zip\$1code”, “city\$1name”, “street\$1name”, and “state”. Then, while processing a document, you could use the custom type in a field “company\$1address”. That field then returns all of the information, grouped in rows beneath the custom type. You can have up to 30 custom type fields per blueprint. # Creating blueprints ## How to create blueprints for custom outputs Amazon Bedrock Data Automation (BDA) allows you to create custom blueprints for any file type BDA can extract. You can use blueprints to define the desired output format and extraction logic for your input files. By creating custom blueprints, you can tailor BDA's output to meet your specific requirements. Within one project, you can apply: + Multiple document blueprints, up to 40. This allows you to process different types of documents within the same project, each with its own custom extraction logic. + One image blueprint. This ensures consistency in image processing within a project. + One audio blueprint. + One video blueprint. ### Creating blueprints There are two methods for creating Blueprints in BDA: + Using the Blueprint Prompt + Manual blueprint creation #### Using the Blueprint Prompt The Blueprint Prompt provides a guided, natural language-based interface for creating Blueprints. To create a blueprint using the Prompt: 1. Navigate to the **Blueprints** section in the BDA console. 1. Click on **Create Blueprint** and select **Use Blueprint Prompt**. 1. Choose the data type (document, image, audio, or video) for your Blueprint. 1. Describe the fields and data you want to extract in natural language. For example: "Extract invoice number, total amount, and vendor name from invoices." 1. The Prompt will generate a Blueprint based on your description. 1. Review the generated Blueprint and make any necessary adjustments. Blueprint prompts are single turn based, meaning you will have to re-enter all information for altering your prompt, not just new information. 1. Save and name your Blueprint. ##### Blueprint Prompt Example The following section goes over an example of a blueprint prompt for an audio blueprint. For this use case, we want to create a blueprint to extract information from a conversation between a customer and a customer service representative. The screenshot below shows the prompt window on the console. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/audio-bpa-prompt.png) At the bottom of the screenshot you can see the AI generated prompt based on the input in the box. We can see how the fields we mention get processed. Next, we can look at the blueprint created from the prompt. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/audio-bpa-example.png) Here we can look at the information we'll expect to process from the conversation. If you're satisfied with the fields, you can begin processing an audio file immediately. If you want to edit your blueprint, you'll need to create a duplicate as opposed to editing directly. You can also adjust your prompt for other outcomes. #### Creating blueprints manually For more advanced users or those requiring fine-grained control, you can create Blueprints manually: 1. Navigate to the **Blueprints** section in the BDA console. 1. Click on **Create Blueprint** and select **Create Manually.** 1. Choose the data type (document, image, audio, or video) for your Blueprint. 1. Define the fields you want to extract, specifying data types, formats, and any validation rules. 1. Configure additional settings such as document splitting or layout handling. 1. Save and name your Blueprint. You can also use the Blueprint JSON editor to create or modify a Blueprint. This allows you to adjust the JSON of the Blueprint directly via text editor. ### Adding blueprints to projects Projects serve as containers for your multi-modal content processing workflows, while Blueprints define the extraction logic for those workflows. You add blueprints to projects to apply the blueprint to files you process with that project. To add a Blueprint to a Project: 1. Navigate to the **Projects** section in the BDA console. 1. Select the Project you want to add the Blueprint to. 1. Click on **Add Blueprint** or **Manage Blueprints**. 1. Choose the Blueprint you want to add from the list of available Blueprints. 1. Configure any project-specific settings for the Blueprint. 1. Save the changes to your Project. ### Defining Fields To get started, you can create a field to identify the information you want to extract or generate, such as product\$1type. For each field, you need to provide a description, data type, and inference type. To define a field, you need to specify the following parameters: + *Description:* Provides a natural language explanation of what the field represents. This description helps in understanding the context and purpose of the field, aiding in the accurate extraction of data. + *Type:* Specifies the data type of the field's value. BDA supports the following types: + string: For text-based values + number: For numerical values + boolean: For true or false values + array: For fields that can have multiple values of the same type (e.g., an array of strings or an array of numbers) + *Inference Type:* Instructs BDA on how to handle the response generation of the field's value. For images, BDA only support inferred inference type. This means that BDA infers the field value based on the information present in the image. For video, fields also contain granularity as an option. For more information on this trait, see Creating blueprints for videos. The following image shows "Add fields" module in the Amazon Bedrock console with the following example fields and values: + Field name: product\$1type + Type: String + Instruction: What is the primary product or service being advertised, e.g., Clothing, Electronics, Food & Beverage, etc.? + Extractions type: Inferred. ![\[Amazon Bedrock UI showing drop down menus and text field to specify image fields.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-console-add-fields-new.png) Here is an example of what that same field definition looks like in a JSON schema, for the API: ``` "product_type":{ "type": "string", "inferenceType": "inferred", "description": "What is the primary product or service being advertised, e.g., Clothing, Electronics, Food & Beverage, etc.?" } ``` In this example: + The type is set to string, indicating that the value of the product\$1type field should be text-based. + The inferenceType is set to inferred, instructing BDA to infer the value based on the information present in the image. + The description provides additional context, clarifying that the field should identify the product type in the image. Example values for product\$1type field are: clothing, electronics, and food or beverage. By specifying these parameters for each field, you provide BDA with the necessary information to accurately extract and generate insights from your images. ### Creating project versions When working with projects, you can create a version of a blueprint. A version is an immutable snapshot of a blueprint, preserving its current configurations and extraction logic. This blueprint version can be passed in a request to start processing data, ensuring that BDA processes documents according to the logic specified in the blueprint at the time the version was created. You can create a version using the `CreateBlueprintVersion` operation. The Amazon Bedrock console also lets you create and save blueprints. When you save a blueprint, it an ID is assigned to that blueprint. You can then publish the blueprint, which creates a snapshot version of that blueprint that can’t be edited. For example, if the blueprint associated to your project is “DocBlueprint”, the created project version will be “DocBlueprint\$11”. You will not be able to make any more changes to “DocBlueprint\$11”, but you can still edit the base blueprint. If you make changes to the blueprint and publish again a new version will be created, like “DocBlueprint\$12”. Blueprint versions can be duplicated and used as a base for a new blueprint. # Leverage Blueprints to achieve different IDP tasks Blueprints are an extremely versatile tool for document processing. The following sections discuss the creation of blueprints with various IDP goals in mind. Additionally, this section provides greater insight into the particulars of creating Blueprints for documents in general. # Create Blueprints for Classification With BDA, you can classify documents by assigning a document class and providing a description when you create a blueprint. The document class serves as a high-level categorization of the document type, while the description provides more granular details about the expected content and elements within that class of documents. We recommend that your description specifies the typical type of data found in the documents along with other relevant information such as purpose of the document and entities expected. Examples of document class and their descriptions are: | Document Class | Description | | --- | --- | | Invoice | An invoice is a document that contains the list of service rendered or items purchased from a company by a person or another company. It contains details such as when the payment is due and how much is owed. | | Payslip | This document issued by an employer to an employee contains wages received by an employee for a given period. It usually contains the breakdown of each of the income and tax deductions items. | | Receipts | A document acknowledging that a person has received money or property in payment following a sale or other transfer of goods or provision of a service. All receipts must have the date of purchase on them. | | W2 | This is a tax form to file personal income received from an employer in a fiscal year | After creating your blueprint fields, follow these steps: 1. On the Create Blueprint page, choose **Save and exit blueprint prompt**. 1. For Blueprint name, enter a name for your blueprint. 1. For Document class, enter a class name that represents the type of document you want to classify. 1. In the Description field, provide a detailed description of the document type. Include information about the type of data and elements commonly found in these documents, such as person, company, addresses, product details, or any other relevant information. 1. Choose Publish blueprint. After you create the blueprint, you can use it to classify documents during inference by providing one or more blueprint IDs in the InvokeDataAutomationAsync API request. BDA uses the document class and description provided in each of the blueprints to accurately categorize and process the documents. When you submit a document for processing, BDA analyzes its content and matches it against the list of blueprints provided. The document is then classified and processed based on the blueprint field instructions to produce the output in the desired structure. # Creating Blueprints for Extraction BDA allows you to define the specific data fields you want to extract from your documents when creating a blueprint. This acts as a set of instructions that guide BDA on what information to look for and how to interpret it. **Defining fields** To get started, you can create a property for each field that requires extraction, such as employee\$1id or product\$1name. For each field, you need to provide a description, data type, and inference type. To define a field for extraction, you need to specify the following parameters: + Field Name: Provides a human-readable explanation of what the field represents. This description helps in understanding the context and purpose of the field, aiding in the accurate extraction of data. + Instruction: Provides a natural language explanation of what the field represents. This description helps in understanding the context and purpose of the field, aiding in the accurate extraction of data. + Type: Specifies the data type of the field's value. BDA supports the following data types: + string: For text-based values + number: For numerical values + boolean: For true/false values + array: For fields that can have multiple values of the same type (e.g., an array of strings or an array of numbers) + Inference Type: Instructs BDA on how to handle the extraction of the field's value. The supported inference types are: + Explicit: BDA should extract the value directly from the document. + Inferred: BDA should infer the value based on the information present in the document. Here's an example of a field definition with all the parameters: ------ #### [ Console ] ![\[Console showing how to add 'Field name' and 'Instruction'. The 'Type' is set to 'String' and 'Extraction type' is set to 'Explicit'.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bdaadd.png) ------ #### [ API ] ``` "product_name":{ "type":"string", "inferenceType":"Explicit", "description":"The short name of the product without any extra details" } ``` ------ In this example: + The type is set to string, indicating that the value of the product\$1name field should be text-based. + The inferenceType is set to Explicit, instructing BDA to extract the value directly from the document without any transformation or validation. + The instruction provides additional context, clarifying that the field should contain the short name of the product without any extra details. By specifying these parameters for each field, you provide BDA with the necessary information to accurately extract and interpret the desired data from your documents. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | ApplicantsName | Full Name of the Applicant | Explicit | string | | DateOfBirth | Date of birth of employee | Explicit | string | | Sales | Gross receipts or sales | Explicit | number | | Statement\$1starting\$1balance | Balance at beginning of period | Explicit | number | **Multi-Valued Fields** In cases where a field may contain multiple values, you can define arrays or tables. **List of Fields** For fields that contain a list of values, you can define an array data type. In this example, "OtherExpenses" is defined as an array of strings, allowing BDA to extract multiple expense items for that field. ------ #### [ Console ] ![\[Console showing how to add 'Field name' and 'Instruction'. The 'Type' is set to 'Array of String' and 'Extraction type' is set to 'Explicit'.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bdaarray.png) ------ #### [ API ] ``` "OtherExpenses":{ "type":"array", "inferenceType":"Explicit", "description":"Other business expenses not included in fields 8-26 or field 30", "items":{ "type":"string" } } ``` ------ **Tables** If your document contains tabular data, you can define a table structure within the schema. In this example, "SERVICES\$1TABLE" is defined as a Table type, with column fields such as product name, description, quantity, unit price and amount. ------ #### [ Console ] ![\[Console showing how to add 'Field name' and 'Instruction'. The 'Type' is set to 'Table' and 'Extraction type' is set to 'Explicit' and shows column-specific fields that are added.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bdatable.png) ------ #### [ API ] ``` "definitions":{ "LINEITEM":{ "properties":{ "quantity":{ "type":"number", "inferenceType":"Explicit" }, "unit price":{ "type":"number", "inferenceType":"Explicit" }, "amount":{ "type":"number", "inferenceType":"Explicit", "description":"Unit Price * Quantity" }, "product name":{ "type":"string", "inferenceType":"Explicit", "description":"The short name of the product without any extra details" }, "product description":{ "type":"string", "inferenceType":"Explicit", "description":"The full item list description text" } } } }, "properties":{ "SERVICES_TABLE":{ "type":"array", "description":"Line items table listing all the items / services charged in the invoice including quantity, price, amount, product / service name and description.", "items":{ "$ref":"#/definitions/LINEITEM" } }, "... ..." ] ``` ------ By defining comprehensive schemas with appropriate field descriptions, data types, and inference types, you can ensure that BDA accurately extracts the desired information from your documents, regardless of variations in formatting or representation. # Create Blueprints for Normalization BDA provides normalization capabilities that allow you to convert and standardize the extracted data according to your specific requirements. These normalization tasks can be categorized into Key Normalization and Value Normalization. **Key normalization** In many cases, document fields can have variations in how they are represented or labeled. For example, the "Social Security Number" field could appear as "SSN," "Tax ID," "TIN," or other similar variations. To address this challenge, BDA offers Key Normalization, which enables you to provide instructions on the variations within your field definitions. By leveraging key normalization, you can guide BDA to recognize and map different representations of the same field to a standardized key. This feature ensures that data is consistently extracted and organized, regardless of the variations present in the source documents. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | LastName | Last name or Surname of person | Explicit | String | | BirthNum | Document Number or file number of the birth certificate | Explicit | String | | OtherIncome | Other income, including federal and state gasoline or fuel tax credit or refund | Explicit | Number | | BusinessName | Name of the business, contractor or entity filling the W9 | Explicit | String | | power factor | Power factor or multiplier used for this usage line item | Explicit | String | | BirthPlace | Name of Hospital or institution where the child is born | Explicit | String | | Cause of Injury | Cause of injury or occupational disease, including how it is work related | Explicit | String | For fields with predefined value sets or enumerations, you can provide the expected values or ranges within the field instruction. We recommend that you include the variations in quotation marks as shown in the examples. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | LICENSE\$1CLASS | The single letter class code, one of "A", "B" or "C" | Explicit | String | | sex | The sex. One of "M" or "F" | Explicit | String | | InformantType | The type of the information. One of "Parent" or "Other" | Explicit | String | | INFORMATION COLLECTION CHANNEL | ONE AMONG FOLLOWING: "FACE TO FACE INTERVIEW", "TELEPHONE INTERVIEW", "FAX OR MAIL", "EMAIL OR INTERNET" | Explicit | String | **Value normalization** Value normalization is a key task in data processing pipelines, where extracted data needs to be transformed into a consistent and standardized format. This process ensures that downstream systems can consume and process the data seamlessly, without encountering compatibility issues or ambiguities. Using normalization capabilities in BDA, you can standardize formats, convert units of measurement and cast values to specific data types. For Value Normalization tasks, the Inferred extraction type should be used as the value may not exactly match the raw text or OCR of the document after it is normalized. For example, a date value like "06/25/2022" that requires to be formatted to "YYYY-MM-DD" will be extracted as "2022-06-25" after normalization, thereby not matching the OCR output from the document. Standardize Formats: You can convert values to predefined formats, such as shortened codes, numbering schemes, or specific date formats. This allows you to ensure consistency in data representation by adhering to industry standards or organizational conventions. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | ssn | The SSN, formatted as XXX-XX-XXX | Inferred | String | | STATE | The two letter code of the state | Inferred | String | | EXPIRATION\$1DATE | The date of expiry in YYYY-MM-DD format | Inferred | String | | DATE\$1OF\$1BIRTH | The date of birth of the driver in YYYY-MM-DD format | Inferred | String | | CHECK\$1DATE | The date the check has been signed. Reformat to YYYY-MM-DD | Inferred | String | | PurchaseDate | Purchase date of vehicle in mm/dd/yy format | Inferred | String | You can also convert values to a standard unit of measurement or to a specific data type by handling scenarios like Not applicable. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | WEIGHT | Weight converted to pounds | Inferred | Number | | HEIGHT | Height converted to inches | Inferred | Number | | nonqualified\$1plans\$1income | The value in field 11. 0 if N/A. | Inferred | Number | # Create Blueprints for Transformation BDA allow you to split, and restructure data fields according to your specific requirements. This capability enables you to transform the extracted data into a format that better aligns with your downstream systems or analytical needs. In many cases, documents may contain fields that combine multiple pieces of information into a single field. BDA enables you to split these fields into separate, individual fields for easier data manipulation and analysis. For example, if a document contains a person's name as a single field, you can split it into separate fields for first name, middle name, last name, and suffix. For Transformation tasks, the extraction type can be defined as Explicit or Inferred, depending on if the value requires to be normalized. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | FIRST\$1NAME | The first name | Explicit | String | | MIDDLE\$1NAME | The middle name or initial | Explicit | String | | LAST\$1NAME | The last name of the driver | Explicit | String | | SUFFIX | The suffix, such as PhD, MSc. etc | Explicit | String | Another example is with address blocks that could appear as a single field | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | Street | What is the street address | Explicit | String | | City | What is the city | Explicit | String | | State | What is the state? | Explicit | String | | ZipCode | What is the address zip code? | Explicit | String | You can define these fields as completely individual fields, or create a Custom Type. Custom Types are re that you can reuse for different fields. In the example below, we create a custom type “NameInfo” that we use for “EmployeeName” and “ManagerName”. ![\[Console showing how to add custom type details. It also shows the sub-properties added to the custom type.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bdacustomtype.png) # Create Blueprints for Validation BDA allows you to define validation rules to ensure the accuracy of the extracted data. These validation rules can be incorporated into your blueprints, enabling BDA to perform various checks on the extracted data. BDA allows you to create custom validations tailored to your specific business or industry requirements. Below are some examples of validations to illustrate the range of this capability. **Numeric validations** Numeric validations are used to check whether the extracted numeric data falls within a specified range of values or meets certain criteria. These validations can be applied to fields such as amounts, quantities, or any other numerical data. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | BalanceGreaterCheck | Is previous balance greater than \$11000? | Inferred | Boolean | | Is Gross Profit equal to difference between Sales and COGS? | Validation question | Inferred | Boolean | | is\$1gross\$1pay\$1valid | Is the YTD gross pay the largest dollar amount value on the paystub? | Inferred | Boolean | **Date/Time validations** Date/time validations are used to check whether the extracted date or time data falls within a specific range or meets certain criteria. These validations can be applied to fields such as due dates, expiration dates, or any other date/time-related data. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | was\$1injury\$1reported\$1after\$11\$1month | Was the injury reported to the employer more than 1 month after the injury date? | Inferred | Boolean | | is\$1overdue | Is the statement overdue? Has the balance due date expired? | Inferred | Boolean | | is\$1delivery\$1date\$1valid | Is the delivery date within the next 30 days? | Inferred | Boolean | **String/Format validations** String/format validations are used to check whether the extracted data adheres to a specific format or matches predefined patterns. These validations can be applied to fields such as names, addresses, or any other textual data that requires format validation. | Field | Instruction | Extraction Type | Type | | --- | --- | --- | --- | | routing\$1number\$1valid | True if the bank routing number has 9 digits | Inferred | Boolean | | Is\$1NumMeterIDsListed | Are there more than 5 meter IDs listed on the bill? | Inferred | Boolean | With BDA's custom validation capabilities, you can create complex validation rules that combine multiple conditions, calculations, or logical operations to ensure the extracted data meets your desired criteria. These validations can involve cross-field checks, calculations, or any other custom logic specific to your business processes or regulatory requirements. By incorporating these validation rules into your blueprints, BDA can automatically validate the extracted data, ensuring its accuracy and compliance with your specific requirements. This capability enables you to make trigger human reviews where validations have failed. # Creating blueprints for images Amazon Bedrock Data Automation (BDA) allows you to create custom blueprints for image modalities. You can use blueprints to define the desired output format and extraction logic for your input files. By creating custom blueprints, you can tailor BDA's output to meet your specific requirements. Within one project, you can apply a single image blueprint. ## Defining data fields for images BDA allows you to define the specific fields you want to identify from your images by creating a blueprint. This acts as a set of instructions that guide BDA on what information to extract and generate from your images. ### Blueprint fields examples for advertisement images Here are some examples of blueprint fields to analyze advertisement images. | | | | | | --- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | | product\$1type | What is the primary product or service being advertised? Ex: Clothing, Electronics, Food & Beverage | inferred | string | | product\$1placement | How is the product placed in the advertisement image, e.g., centered, in the background, held by a person, etc.? | inferred | string | | product\$1size | Product size is small if size is less than 30% of the image, medium if it is between 30 to 60%, and large if it is larger than 60% of the image | inferred | string | | image\$1style | Classify the image style of the ad. For example, product image, lifestyle, portrait, retro, infographic, none of the above. | inferred | string | | image\$1background | Background can be" solid color, natural landscape, indoor, outdoor, or abstract. | inferred | string | | promotional\$1offer | Does the advertisement include any discounts, offers, or promotional messages? | inferred | boolean | ### Examples of blueprint fields for media search Here are some examples of blueprint fields to generate metadata from images for media search. | | | | | | --- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | | person\$1counting | How many people are in the image? | inferred | number | | indoor\$1outdoor\$1classification | Is the image indoor or outdoor? | inferred | string | | scene\$1classification | Classify the setting or environment of the image. Ex: Urban, Rural, Natural, Historical, Residential, Commercial, Recreational, Public Spaces | inferred | string | | animal\$1identification | Does the image contain any animals? | inferred | boolean | | animal\$1type | What type of animals are present in the image? | inferred | string | | color\$1identification | Is the image in color or black and white? | inferred | string | | vehicle\$1identification | Is there any vehicle visible in the image? | inferred | string | | vehicle\$1type | What type of vehicle is present in the image? | inferred | string | | watermark\$1identification | Is there any watermark visible in the image? | inferred | boolean | # Creating blueprints for audio Similar to image blueprints, you can only have one audio blueprint per project. Below are some example fields for audio processing. ## Blueprint field examples for audio files | | | | | | --- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | | transcript\$1summary | Generate a concise abstractive summary of the conversation, focusing on the main topics and key themes. Ensure accuracy by summarizing only what is explicitly discussed, without adding specific details not present in the conversation. Keeping the response within 100 words. | inferred | string | | topics | The main topics of the audio transcript, listed as single words. | inferred | [string] (Array of strings) | | category | The category of the audio (not the topic). Choose from General conversation, Media, Hospitality, Speeches, Meetings, Education, Financial, Public sector, Healthcare, Sales, Audiobooks, Podcasts, 911 calls, Other. | inferred | string | | spoken\$1named\$1entities | Any named entities (typically proper nouns) explicitly mentioned in the audio transcript including locations, brand names, company names, product names, services, events, organizations, etc. Do not include names of people, email addresses or common nouns. | extractive | [string] (Array of strings) | ## Blueprint field examples for conversational analytics | | | | | | --- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | | call\$1summary | Summarize the caller-agent conversation in under 100 words. Start with the caller's request, then the agent's response and actions, ending with outcomes or follow-ups. Include key details like emails, links, or callbacks. For multiple issues, summarize each with its outcome and next steps. | inferred | string | | call\$1categories | The category (or categories) of the call. Choose one or more from Billing, Tech support, Customer service, Account support, Sales, Complaints, Product issues, Service issues, General inquiries, Other. | inferred | [string] (Array of strings) | | spoken\$1locations | Locations explicitly mentioned in the conversation, including cities, states, and countries. | extractive | [string] | | call\$1opening | Did the agent greet the caller and introduce themselves at the beginning of the call? | extractive | boolean | # Creating blueprints for video Blueprints for video files have a few unique qualities compared to other blueprints, particularly in field creation. Video blueprints have a parameter called granularity, which lets you set a field to either Video, or Chapter. When the field is set to video, it will be detected across an entire video. For example, if you wanted a summary of the entire clip, you would want to set that field's granularity to video. A field with granularity set to Chapter will instead return a response for each chapter of the video. The field will return a value for each video chapter. Continuing from the previous example, if you wanted a summary of each portion of a video, you'd set the granularity to chapter. When you create a chapter granularity field, you can set a unique data type, an array of entities. For example, if you want to detect the visually prominent objects in your video, you could create a field called `key-visual-objects`, and set the type it an array of entities. This field would then return the names of the entities in an array object. Below are some example fields for video processing. All fields in video blueprints are considered inferred, except for entities and entity arrays. ## Blueprint field examples for media search | | | | | | | --- |--- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | Granularity | | key-visual-objects | Please detect all the visually prominent objects in the video | extractive | Array of entities | [ "chapter" ] | | keywords | Searchable terms that capture key themes, cast, plot elements, and notable aspects of TV shows and movies to enhance content discovery. | inferred | Array of strings | ["video"] | | genre | The genre of the content. | inferred | string | ["video"] | | video-type | Identify the type of video content | inferred | enums: ["Movie", "TV series", "News", "Others"] | [ "video" ] | ## Blueprint field examples for keynote highlights | | | | | | | --- |--- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | Granularity | | broadcast-setting | The physical setting or environment where the broadcast or training session is taking place. | inferred | enums["conference hall", "classroom", "outdoor venue", "Others", "Not applicable to the video"] | [ "video" ] | | broadcast-audience-engagement | The level of engagement or interaction between the speakers and the audience. | inferred | enums["interactive", "passive", "Not applicable to the video"] | ["video"] | | broadcast-visual-aids | A list of notable visual aids or materials used during the presentation, such as slides, diagrams, or demonstrations. | inferred | Array of strings | ["video"] | | broadcast-audience-size | The size of the audience present at the event. | inferred | enums["large crowd", "medium crowd", "small group", "Not applicable to this video"] | [ "chapter" ] | | broadcast-presentation-topics | A list of key topics, subjects, or themes covered in the presentation or training session. | inferred | enums: ["Movie", "TV series", "News", "Others"] | [ "video" ] | ## Blueprint field examples for advertisement analysis | | | | | | | --- |--- |--- |--- |--- | | Field | Instruction | Extraction Type | Type | Granularity | | ads-video-ad-categories | The ad categories for the video | inferred | enums["Health and Beauty", "Weight Loss", "Food and Beverage", "Restaurants", "Political", "Cryptocurrencies and NFT", "Money Lending and Finance", "Tobacco", "Other", "Video is not an advertisement"] | [ "video" ] | | ads-video-language | The primary language of the advertisement | inferred | string | ["video"] | | ads-video-primary-brand | The main brand or company being advertised in the video. | inferred | string | ["video"] | | ads-video-main-message | The primary message or tagline conveyed in the advertisement | inferred | string | [ "video" ] | | ads-video-message-clarity | How clear and understandable the main message of the advertisement is | inferred | enums: ["clear", "ambiguous", "Not applicable to the video"] | [ "video" ] | | ads-video-target-audience-interests | Specific interests or hobbies that the target audience is likely to have | inferred | Array of strings | [ "video" ] | | ads-video-product-type | The category or type of product being advertised | inferred | enums: ["electronics", "apparel", "food\$1and\$1beverage", "automotive", "home\$1appliances", "other", "Not applicable to the video"] | [ "video" ] | | ads-video-product-placement | The way the product is positioned or showcased in the advertisement | inferred | enums: ["front\$1and\$1center", "background", "held\$1by\$1person", "other", "Not applicable to the video"] | [ "video" ] | | ads-video-product-features | The key features or specifications of the advertised product highlighted in the video | inferred | Array of strings | [ "video" ] | | ads-video-number-of-products | The number of distinct products or variations featured in the advertisement | inferred | number | [ "video" ] | Video also supports array of entities type which helps identify and locate specific entities within video content. This feature returns an array of detected entities. Below is an example of an array of entities in a customer blueprint: ``` "field_name": { "items": { "$ref": "bedrock-data-automation#/definitions/Entity" }, "type": "array", "instruction": "Please detect all the visually prominent objects in the video", "granularity": [ "chapter" ] } ``` **Note** `bedrock-data-automation#/definitions/Entity` is a BDA owned service type. To parse the results you can use the following schema. ``` { "$schema": "http://json-schema.org/draft-07/schema#", "$id": "bedrock-data-automation", "type": "object", "definitions": { "BoundingBox": { "type": "object", "additionalProperties": false, "properties": { "left": { "type": "number" }, "top": { "type": "number" }, "width": { "type": "number" }, "height": { "type": "number" } } }, "Entity": { "type": "object", "additionalProperties": false, "properties": { "label": { "type": "string" }, "bounding_box": { "$ref": "bedrock-data-automation#/definitions/BoundingBox" }, "confidence": { "type": "number" } } } }, "properties": {} } ``` # Optimize your blueprints with ground truth You can improve blueprint accuracy by providing example content assets with the correct expected results. Blueprint instruction optimization uses your examples to refine the natural language instructions in your blueprint fields, which improves your inference **Results** accuracy. Blueprint instruction optimization works best when you need to extract specific values that appear directly in your documents, such as invoice numbers, contract amounts, or tax form fields. We recommend providing 3 to 10 example assets that represent the documents you process in production, especially ones where you have encountered accuracy challenges. **How blueprint instruction optimization works** Blueprint instruction optimization analyzes the differences between your expected results and the initial inference results. The service iteratively refines the natural language instructions for each field of your blueprint until the instructions produce more accurate results across your example assets. This process completes in minutes without requiring any model training or fine-tuning. When you start your optimization process, you provide your example assets and the corresponding ground truth data—the correct values you expect to extract for each field. Blueprint instruction optimization compares these values against inference results and adjusts the field descriptions to improve accuracy. After optimization completes, you receive accuracy metrics that show the accuracy improvement, including exact match rates and F1 scores measured against your ground truth. **What you need before you start optimizing your blueprints** **A blueprint with defined fields**. Create a blueprint using the console or API. Your blueprint should include the field names and initial descriptions for the data you want to extract. **Example content assets**. Gather 3 to 10 documents assets that represent your production workload on documents. Choose examples that contain all the fields in your blueprint. **Expected results for your examples**. Prepare the correct values you want to extract from each example asset. You can enter these values manually during optimization or upload them using a manifest file. **An S3 bucket location**. Specify an S3 bucket where you want to store your example assets and ground truth data. You can provide your own bucket or allow the service to create one for you. **Step-by-step process to optimize your blueprint** To optimize your blueprint, start from the blueprint detail page in the Amazon Bedrock Data Automation console. Note this is only available for your document modality. Step 1. Select **Optimize blueprint** to begin the optimization workflow. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-blueprint-optimize-button.png) Step 2. **Upload your example assets**. Choose up to 10 content assets from your local device or from an S3 location. The service uploads your assets and displays thumbnails for each file. If you previously optimized this blueprint, you can add new examples or remove existing ones. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-optimize-files-selector.png) Step 3. **Provide ground truth for each asset**. Select an asset to open the ground truth editor. The editor displays your document preview on the left and a simplified table of your blueprint fields on the right. For each field, enter the correct value you expect to extract in the Ground Truth column. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-optimize-files-ground-truth.png) Step 4. To speed up ground truth entry, select **Auto-populate** to run initial inference on your assets and automatically populate the **Ground Truth** column from values in your **Results** column. Edit any incorrect values before proceeding. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-optimize-ground-truth-edit.png) Step 5. **Start optimization**. After you complete ground truth entry for all selected assets, choose **Start optimization**. Data automation analyzes your examples and refines the natural language instructions for each field. A progress indicator shows the optimization status with messages such as "Reading your assets" and "Iterating on blueprint natural language instructions." ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-start-optimization-button.png) Step 6. **Review the evaluation metrics**. When optimization completes, the **Metrics** section displays accuracy metrics for your blueprint. The metrics compare performance before optimization and after optimization. Review the overall F1 score, confidence score, and exact match rate to assess whether the blueprint meets your accuracy requirements. The **Metrics by sample file** tab shows field-level accuracy for each example asset. Use these metrics to identify which fields improved and which fields may need additional examples or manual refinement. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/bda-optimize-metrics.png) Step 7. **Complete optimization**. If the evaluation metrics meet your requirements, select **Save optimized blueprint** to promote the optimized blueprint to production. Your blueprint now uses the refined natural language instructions for all future inference requests. **Re-optimize your blueprint** You can re-optimize a blueprint at any time to improve accuracy further. Return to the blueprint detail page and select **Optimize blueprint**. The service displays the assets you previously used for optimization along with their ground truth values. To re-optimize, you can add new example assets, edit ground truth values for existing assets, or remove assets that no longer represent your workload. When you select **Start optimization**, blueprint instruction optimization calculates relative to your current blueprint instructions versus the new instructions. **Edit a blueprint after optimization** If you add or remove fields from an optimized blueprint, the service removes the optimization history and associated example assets. Before editing, download the manifest file that contains your asset locations and ground truth labels. The manifest file uses JSON format and includes all fields and ground truth values from your previous optimization. To preserve your optimization work, upload the manifest file when you re-optimize the edited blueprint. Data automation automatically applies ground truth values to matching fields. Fields that no longer exist in the blueprint are removed from the manifest. New fields do not have ground truth values until you provide them. **Manage optimization costs** Blueprint instruction optimization consumes incurs the inference costs as you would if manually edit your natural language instructions and iteratively test them against each sample document. For a rough calculation, the number of pages you supply as examples will be the number of pages that will be charged as you optimize your blueprint. Each optimization run processes your example assets multiple times to refine the instructions. To minimize costs, start with 3 to 5 examples for your initial optimization. Add more examples when you inspect the evaluation metrics and believe you need additional accuracy improvements. In addition, the optimized natural language instructions tend to be longer and more detailed than the original instructions, which can increase runtime inference costs. # Data Automation Library AWS Bedrock Data Automation Library enables you to enhance BDA's understanding of your content and generate more accurate insights from your data. A Data Automation Library serves as a container that stores entities and can be associated with BDA projects to improve extraction accuracy across multiple entity types and modalities for your specific use cases. Currently, Data Automation Library supports [Custom Vocabulary](bda-library-custom-vocabulary.md) to enhance extraction accuracy for audio and video content. Limits related to this feature are in the [Quotas and Limits](https://docs.aws.amazon.com/bedrock/latest/userguide/bda-limits.html) page. ## Key Benefits 1. **Improved Accuracy:** Tailor recognition to your specific needs. 1. **Cross modality support:** Enhance audio and video analysis. 1. **Flexible Management:** Easily create, update and associate to projects. 1. **Reusable Resources:** Create libraries once and use them across multiple projects. 1. **Easy Integration:** Simple API-driven workflow for library management. ## How Data Automation Library Works You can create a Data Automation Library and populate it with domain-specific entities, which enables BDA to apply your custom knowledge during content processing and improve extraction accuracy across your use cases. You can associate a library with a BDA project, which enables all jobs processed through that project to automatically leverage the library's entities without additional configuration per job. When ingestion is completed, a folder with the name of the job ID is created in the S3 URI provided in the ingestion API request. Both the input manifest and final ingestion results are uploaded in that folder. For example if the output bucket provided in the request is `s3://my-bucket/outputs/` and the jobId is `328c43e7-d226-41c9-9acb-e71a37022b99` then the input manifest and the final ingestion results are uploaded in `s3://my-bucket/outputs/328c43e7-d226-41c9-9acb-e71a37022b99` **Basic workflow:** 1. **Create a library** — Use [CreateDataAutomationLibrary](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_CreateDataAutomationLibrary.html) to initialize an empty library container. 1. **Add entities to your library** — Use [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) to add your domain-specific entities. 1. **Associate the library with a project** — Link the library during project creation with [CreateDataAutomationProject](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_CreateDataAutomationProject.html), or update an existing project with [UpdateDataAutomationProject](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_UpdateDataAutomationProject.html). 1. **Process your content** — Run jobs using [InvokeDataAutomationAsync](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomationAsync.html) through the associated project to apply enhanced extraction accuracy across your content. ## Key Concepts ### Data Automation Library A container that stores entities of one or more types. Libraries can be attached to multiple Data Automation projects and reused across different workloads. ### Data Automation Library Entity Type The type of content stored in the library. Currently, only [VOCABULARY](bda-library-custom-vocabulary.md) entity type is supported. ### Data Automation Library Entity A specific instance within an entity type. For Custom Vocabulary, an entity represents a collection of words and phrases for a specific language. ### Data Automation Library Ingestion Job An asynchronous operation that adds, updates, or deletes entities in a library. Jobs are processed sequentially to maintain data consistency. ### Project Association The link between a library and a BDA project. When you associate a library with a project, all jobs processed through that project apply the library's entities to improve extraction accuracy for your content. Note, that a project can only be associated with one library, but one library can be associated with multiple projects. ## Navigating to Data Automation Library page in the BDA Console 1. Navigate to the Amazon Bedrock service. 1. On the sidebar menu, select "Data Automation". 1. Select "Manage libraries" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-console-navigate-manage-libraries.png) # Custom Vocabulary You can provide domain-specific words and phrases per language, which enables BDA to improve extraction accuracy for your audio and video content. You can include terminology unique to your industry or domain — such as brand names, acronyms, proper nouns, and specialized terms — which enables BDA to recognize and extract those terms accurately from your processed content. **Important** You are responsible for the integrity of your own data when you use Bedrock Data Automation Library. Do not enter confidential information, personal information (PII), or protected health information (PHI) into a custom vocabulary. You can find the list of supported languages for Custom Vocabulary [here](https://docs.aws.amazon.com/bedrock/latest/userguide/bda-limits.html). Note that only the characters listed in your language's [character set](bda-library-character-sets.md) can be used in a custom vocabulary. ## How to use? You can use Custom Vocabulary in your BDA project by creating a Data Automation Library, adding domain-specific vocabulary entities/lists for your target language, and associating the library with your project during project creation or by updating an existing project, which enables all audio and video processing jobs executed through that project to apply your custom vocabulary. This results in improved extraction accuracy for your domain-specific terms across both standard output and custom blueprint output. **Considerations when using Custom Vocabulary:** + Focus on domain-specific terms that are frequently mis-recognized + Include common variations and spellings of important terms + Organize vocabulary by language for better management + Prioritize industry-specific jargon and technical terms + Include acronyms and abbreviations commonly used in your domain # Character Sets **Topics** + [English character set](#char-english) + [Spanish character set](#char-spanish) + [French character set](#char-french) + [German character set](#char-german) + [Italian character set](#char-italian) + [Portuguese character set](#char-portuguese) + [Japanese character set](#char-japanese) + [Korean character set](#char-korean) + [Chinese, Mandarin (Mainland China), Simplified character set](#char-chinese-cn) + [Chinese, Mandarin (Taiwan), Traditional character set](#char-chinese-tw) + [Chinese, Cantonese (Hong Kong), Traditional character set](#char-chinese-hk) ## English character set + a - z + A - Z + 0 - 9 + ' (apostrophe) + - (hyphen) + . (period) ## Spanish character set + a - z + A - Z + 0 - 9 + ' (apostrophe) + - (hyphen) + . (period) Refer to the table below for additional supported characters. | Character | Character | | --- | --- | | Á | á | | É | é | | Í | í | | Ó | ó | | Ú | ú | | Ñ | ñ | | ü | | ## French character set + a - z + A - Z + 0 - 9 + ' (apostrophe) + - (hyphen) + . (period) Refer to the table below for additional supported characters. | Character | Character | | --- | --- | | À | à | | Â | â | | Ç | ç | | È | è | | É | é | | Ê | ê | | Ë | ë | | Î | î | | Ï | ï | | Ô | ô | | Ö | ö | | Ù | ù | | Û | û | | Ü | ü | ## German character set + a - z + A - Z + 0 - 9 + ' (apostrophe) + - (hyphen) + . (period) Refer to the table below for additional supported characters. | Character | Character | | --- | --- | | Ä | ä | | Ö | ö | | Ü | ü | | ß | | ## Italian character set + a - z + A - Z + 0 - 9 + ' (apostrophe) + - (hyphen) + . (period) Refer to the table below for additional supported characters. | Character | Character | | --- | --- | | À | à | | Ä | ä | | Ç | ç | | È | è | | É | é | | Ê | ê | | Ë | ë | | Ì | ì | | Ò | ò | | Ù | ù | | Ü | ü | ## Portuguese character set + a - z + A - Z + 0 - 9 + ' (apostrophe) + - (hyphen) + . (period) Refer to the table below for additional supported characters. | Character | Character | | --- | --- | | Á | á | | À | à | | Â | â | | Ä | ä | | Ã | ã | | Ç | ç | | É | é | | Ê | ê | | È | è | | Ë | ë | | Í | í | | Ñ | ñ | | Ó | ó | | Ô | ô | | Ö | ö | | Õ | õ | | Ú | ú | | Ü | ü | ## Japanese character set [ja-jp-character-set](samples/ja-jp-character-set.zip) ## Korean character set [ko-kr-character-set](samples/ko-kr-character-set.zip) ## Chinese, Mandarin (Mainland China), Simplified character set [zh-cn-character-set](samples/zh-cn-character-set.zip) ## Chinese, Mandarin (Taiwan), Traditional character set [zh-tw-character-set](samples/zh-tw-character-set.zip) ## Chinese, Cantonese (Hong Kong), Traditional character set [zh-hk-character-set](samples/zh-hk-character-set.zip) # Managing Data Automation Library This section describes how to perform library-level operations for Data Automation Libraries. You can manage libraries through the AWS Management Console or AWS CLI. Library management operations include creating libraries, retrieving library details, listing all libraries in your account and region, updating library metadata such as descriptions, and deleting libraries when they're no longer needed. # Creating Library Use the [CreateDataAutomationLibrary](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_CreateDataAutomationLibrary.html) API to create a new library container. ## AWS CLI Example: **Request** ``` aws bedrock-data-automation create-data-automation-library \ --library-name "healthcare-vocabulary" \ --library-description "Medical terminology for transcription accuracy" \ --region us-east-1 ``` **Response** ``` { "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary", "status": "ACTIVE" } ``` ## AWS Console Example: 1. Navigate to "Manage libraries" page in BDA Console 1. Choose "Create library" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-create-console.png) ## Important Notes 1. Library creation is synchronous and returns immediately 1. Library names must be unique within your account and region 1. The library is empty until you run an entity ingestion job 1. Libraries can be associated with multiple projects # Getting Library Details Use the [GetDataAutomationLibrary](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_GetDataAutomationLibrary.html) API to retrieve information about an existing library. ## AWS CLI Example: **Request** ``` aws bedrock-data-automation get-data-automation-library \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" ``` **Response:** ``` { "library": { "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary", "libraryName": "healthcare-vocabulary", "libraryDescription": "Medical terminology for transcription accuracy", "status": "ACTIVE", "creationTime": "2026-01-01T00:00:00Z", "entityTypes": [ { "entityType": "VOCABULARY", "entityMetadata": "{\"entityCount\": 150}" } ] } } ``` ## AWS Console Example: 1. Navigate to "Manage libraries" page in BDA Console 1. Select the desired library from the list of libraries ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-get-details-console.png) # Listing Libraries Use the [ListDataAutomationLibraries](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_ListDataAutomationLibraries.html) api to retrieve the list of libraries. ## AWS CLI Example: **Request** ``` aws bedrock-data-automation list-data-automation-libraries \ --max-results 50 ``` **Response:** ``` { "libraries": [ { "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary", "creationTime": "2026-01-01T00:00:00Z", "libraryName": "healthcare-vocabulary" } ], "next_token": "" } ``` ## AWS Console Example: 1. Navigate to "Manage libraries" page in BDA Console. This page will list libraries associated in this account. ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-list-console.png) # Updating Library Metadata Use the [UpdateDataAutomationLibrary](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_UpdateDataAutomationLibrary.html) api to update library metadata ``` aws bedrock-data-automation update-data-automation-library \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --library-description "Updated: Medical terminology for healthcare transcription" ``` **Response:** ``` { "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary", "status": "ACTIVE" } ``` # Deleting a Library Use the [DeleteDataAutomationLibrary](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_DeleteDataAutomationLibrary.html) api to delete library with its entities. ## AWS CLI Example: **Request** ``` aws bedrock-data-automation delete-data-automation-library \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" ``` **Response:** ``` { "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary", "status": "DELETING" } ``` ## AWS Console Example: 1. Navigate to "Manage libraries" page in BDA Console 1. Choose the desired library from the list of libraries 1. Choose "Delete library" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-delete-console.png) ## Important: 1. To delete the library, first dissociate the library from the project and then delete the library. 1. You cannot delete a library with an active ingestion job 1. Deletion removes all entities and search indices 1. The operation may take several minutes for large libraries # Managing Library Entities This section describes how to manage entities within your Data Automation Libraries. Entities are domain-specific resources that enhance extraction accuracy for your content processing workloads. Currently, [Custom Vocabulary](bda-library-custom-vocabulary.md) is the only supported entity type, which improves speech recognition and transcription for audio and video content. You can perform entity-level operations using the AWS Management Console or AWS CLI. These operations include adding new entities via ingestion jobs with the vocabulary input provided either through S3 manifest files or inline payloads, monitoring status of such ingestion jobs, updating existing entities using UPSERT operations, deleting specific entities, and viewing entity details. Unlike library-level operations that manage the container itself, entity operations focus on the content within your libraries. Managing library entities can be done using these APIs: + [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) adds, updates, or deletes entities through asynchronous ingestion jobs using UPSERT or DELETE operations; + [GetDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_GetDataAutomationLibraryIngestionJob.html) checks the status and progress of ingestion jobs; + [ListDataAutomationLibraryIngestionJobs](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_ListDataAutomationLibraryIngestionJobs.html) retrieves a paginated list of all the ingestion jobs in account; + [ListDataAutomationLibraryEntities](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_ListDataAutomationLibraryEntities.html) retrieves a paginated list of all entities within a library; and + [GetDataAutomationLibraryEntity](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_GetDataAutomationLibraryEntity.html) retrieves detailed information about a specific entity including all vocabulary phrases. **Note**: Entity management is distinct from library management. For information about creating, updating, or deleting libraries, see [Managing Data Automation Library](bda-library-managing.md). # Managing Custom Vocabulary Entities This section describes how to add, update, delete, and view custom vocabulary entities/lists within your Data Automation Libraries. Custom vocabulary entities contain domain-specific terms and phrases that improve extraction accuracy for audio and video content in your target languages. You can manage vocabulary entities through asynchronous ingestion jobs using the AWS Management Console or AWS CLI. Operations include adding new vocabulary using S3 manifest files or inline payloads, monitoring job status and reviewing outputs, updating entities with UPSERT operations (which replace the entire entity), deleting specific entities, and viewing entity lists or detailed phrase information. All modifications are processed through the InvokeDataAutomationLibraryIngestionJob API with different operation types (UPSERT or DELETE). # Adding New Vocabulary Entities You can add vocabulary to your library using the [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) API. You can provide vocabulary through an S3 manifest file or inline payload. **Important** UPSERT operations use a clobber-style replacement at the entity level, meaning the entire entity is replaced rather than merged with existing content. ## Option 1: Using S3 Manifest File ### Step 1: Create a JSONL manifest file Example: `vocabulary-manifest.json` ``` {"entityId":"medical-en","description":"Medication terms in English language","phrases":[{"text":"paracetamol"},{"text":"ibuprofen"},{"text":"acetaminophen","displayAsText":"acetaminophen"}],"language":"EN"} {"entityId":"medical-es","description":"Medication terms in Spanish language","phrases":[{"text":"paracetamol"},{"text":"ibuprofen"},{"text":"acetaminophen","displayAsText":"acetaminophen"}],"language":"ES"} ``` **Manifest File Requirements:** + **File Format:** JSONL (JSON Lines) + **Entity JSON:** + **entityId** (required): Unique identifier (max 128 characters) + **description** (optional): Description of the entityId + **language** (required): ISO language code ([Supported languages](bda-library-character-sets.md)) + **phrases** (required): Array of text objects. Each object contains: + **text** (required): Individual word or phrase + **displayAsText** (optional): Use this to replace actual word in transcript (NOTE: Case sensitive) ### Step 2: Upload the manifest to S3 ``` aws s3 cp vocabulary-manifest.json s3://my-bucket/manifests/ ``` ### Step 3: Start the ingestion job Use the [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) to start a vocabulary ingestion job. **AWS CLI Example:** **Request** ``` aws bedrock-data-automation-data-automation invoke-data-automation-library-ingestion-job \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --operation-type "UPSERT" \ --input-configuration '{"s3Object":{"s3Uri":"s3://my-bucket/manifests/vocabulary-manifest.json"}}' \ --output-configuration '{"s3Uri":"s3://my-bucket/outputs/"}' ``` **Response:** ``` { "jobArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library-ingestion-job/job-12345" } ``` **AWS Console Example:** 1. Navigate to "Library details" page 1. Choose "Add custom vocabulary list" 1. Choose "Upload/select manifest" 1. Choose whether to upload the manifest file directly or from a S3 location ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-add-cv-manifest-console.png) ## Option 2: Using Inline Payload This option can be used for quick updates with up to 100 phrases. Use the [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) to start a vocabulary ingestion job. **AWS CLI Example:** **Request** ``` aws bedrock-data-automation-data-automation invoke-data-automation-library-ingestion-job \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --operation-type "UPSERT" \ --input-configuration '{"inlinePayload":{"upsertEntitiesInfo":[{"vocabulary":{"entityId":"medical-en","language":"EN","phrases":[{"text":"paracetamol"},{"text":"ibuprofen"}]}}]}}' \ --output-configuration '{"s3Uri":"s3://bda-data-bucket/output/"}' ``` **Response:** ``` { "jobArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library-ingestion-job/job-12345" } ``` **AWS Console Example:** 1. Navigate to "Library details" page 1. Choose "Add custom vocabulary list" 1. Choose "Add manually" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-add-cv-inline-console.png) # Updating Vocabulary Entities Use the [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) with "UPSERT" operation type to update existing vocabulary entities for the same entityId. This will replace the entire entity with the new content. **Important** UPSERT operations are clobber-style at the entity level. The entire entity is replaced, not merged. ## AWS CLI Example: *Note that this operation is same as [Adding New Vocabulary Entities](bda-library-adding-cv.md).* **Request** ``` aws bedrock-data-automation-data-automation invoke-data-automation-library-ingestion-job \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --operation-type "UPSERT" \ --input-configuration '{"s3Object":{"s3Uri":"s3://my-bucket/manifests/vocabulary-manifest.json"}}' \ --output-configuration '{"s3Uri":"s3://my-bucket/outputs/"}' ``` ## AWS Console Example: 1. Navigate to the "Library details" page for your library 1. Select the desired entity from the "Custom vocabulary lists" 1. Make the desired modifications: 1. Update: 1. Select the desired phrase 1. Update the phrase 1. Choose the check mark to the right of the phrase 1. Delete 1. Select the checkbox to the left of the desired phrase(s) 1. Choose "Delete phrases" 1. Choose "Save updates" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-update-cv-console.png) # Deleting Vocabulary List Use the [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) with "DELETE" operation type to remove specific entities from your library. ## AWS CLI Example: ### Option 1: Using S3 Manifest file #### Step 1: Create a JSONL manifest file Create a simple s3 manifest file with a list of entityIds. Example: `cv-delete-manifest.json` ``` {"entityIds": ["medical-en", "medical-es"]} ``` #### Step 2: Upload the manifest to S3 ``` aws s3 cp vocabulary-manifest.json s3://my-bucket/manifests/ ``` #### Step 3: Start the ingestion job Use the [InvokeDataAutomationLibraryIngestionJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_InvokeDataAutomationLibraryIngestionJob.html) to start a vocabulary ingestion job with an operation type DELETE **Request** ``` aws bedrock-data-automation invoke-data-automation-library-ingestion-job \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --operation-type "DELETE" \ --input-configuration '{"s3Object":{"s3Uri":"s3://my-bucket/manifests/cv-delete-manifest.json"}}' \ --output-configuration '{"s3Uri":"s3://my-bucket/outputs/"}' ``` **Response:** ``` { "jobArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library-ingestion-job/job-12345" } ``` ### Option 2: Using inline payload **Request** ``` aws bedrock-data-automation invoke-data-automation-library-ingestion-job \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --operation-type "DELETE" \ --input-configuration '{"inlinePayload":{"deleteEntitiesInfo":{"entityIds": ["medical-en"]}}}' \ --output-configuration '{"s3Uri": "s3://my-bucket/outputs/"}' ``` ## AWS Console Example: 1. Navigate to the "Library details" page for your library 1. Choose the desired entity from the "Custom vocabulary lists" 1. Choose "Delete custom vocabulary list" # Checking Vocabulary Ingestion Job Status ``` aws bedrock-data-automation get-data-automation-library-ingestion-job \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --job-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library-ingestion-job/job-12345" ``` ## Job Statuses: + **IN\$1PROGRESS:** Job is currently processing + **COMPLETED:** All entities processed successfully + **COMPLETED\$1WITH\$1ERRORS:** Some entities failed (check output for details) + **FAILED:** Job failed completely ## Understanding Job Output After job completion, check the S3 output location for detailed results. ``` { "metadata": { "jobId": "job-12345", "manifestS3Bucket": "my-bucket", "manifestS3Key": "manifests/vocabulary-manifest.json" }, "vocabulary": { "successful": [ { "entityId": "medical-en", "itemIndex": 0, "description": "Medical terms in English language", "phrases": [ { "text": "acetaminophen" }, { "text": "ibuprofen" }, { "text": "naproxen" } ], "language": "EN" } ], "failed": [ { "entityId": "medical-es", "itemIndex": 1, "errorType": "ValidationError", "errorMessage": "Entity validation failed: Missing required creator property 'language'" } ] } } ``` # Viewing Library Entities You can view data automation library entities using two operations: + [ListDataAutomationLibraryEntities](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_ListDataAutomationLibraryEntities.html): Retrieves a paginated list of all entities within a library, showing entity metadata. + [GetDataAutomationLibraryEntity](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_GetDataAutomationLibraryEntity.html): Retrieves detailed information about a specific entity. Both operations are available through the AWS Management Console and AWS CLI, requiring the library ARN and entity type as parameters. # Listing Library Entities Use the [ListDataAutomationLibraryEntities](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_ListDataAutomationLibraryEntities.html) api to retrieve the list of entities. ## AWS CLI Example: **Request** ``` aws bedrock-data-automation list-data-automation-library-entities \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --max-results 50 ``` **Response** ``` { "entities": [ { "vocabulary": { "entityId": "medical-en", "language": "EN", "numOfPhrases": 2, "lastModifiedTime": "2026-03-06T07:06:00.413000+00:00" } } ] } ``` ## AWS Console Example: 1. Navigate to the "Library details" page for your library ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-list-entities-console.png) # Getting Library Entity Details Use the [GetDataAutomationLibraryEntity](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_GetDataAutomationLibraryEntity.html) api to retrieve the list of vocabulary for an entity. ## AWS CLI Example: **Request** ``` aws bedrock-data-automation get-data-automation-library-entity \ --library-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" \ --entity-type "VOCABULARY" \ --entity-id "medical-en" ``` **Response** ``` { "entity": { "vocabulary": { "entityId": "medical-en", "language": "EN", "phrases": [ { "text": "paracetamol" }, { "text": "ibuprofen" } ], "lastModifiedTime": "2026-03-06T07:06:00.413000+00:00" } } } ``` ## AWS Console Example: 1. Navigate to the "Library details" page for your library 1. Choose the desired entity from the list ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-get-entity-details-console.png) # Using Libraries in Projects This section describes how to associate Data Automation Libraries with your BDA projects to enable library functionality for content processing. When you associate a library with a project, all audio and video processing jobs executed through that project leverage the entities defined in the library to improve extraction accuracy. # Associating Library with a Project To use custom vocabulary, associate your library with a BDA project. This can be done in two ways: ## Option 1: Create Project with Library: **AWS CLI Example:** ``` aws bedrock-data-automation create-data-automation-project \ --project-name "audio-transcription-project" \ --standard-output-configuration '{ "audio": { "extraction": { "category": { "state": "ENABLED", "types": ["TRANSCRIPTION"] } } } }' \ --data-automation-libraries '[{ "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" }]' ``` **AWS Console Example:** 1. Navigate to the Bedrock "Data Automation" page 1. Create a project 1. Navigate to the project detail page 1. Choose "Associated library" tab 1. Choose "Associate a library" button 1. Choose a library by selecting the radio button next to it and choose the "Associate library" button ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-associate-from-project-console.png) ## Option 2: Update Existing Project with library: ``` aws bedrock-data-automation update-data-automation-project \ --project-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-project/audio-transcription-project" \ --data-automation-libraries '[{ "libraryArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-library/healthcare-vocabulary" }]' ``` **AWS Console Example:** 1. Navigate to the "Library details" page for your library 1. Expand "Associated projects" 1. Choose "Associate projects" 1. Choose the projects to associate and choose "Associate project" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-associate-from-library-console.png) # Dissociation of Library from a Project You can dissociate a library from a project using the [UpdateDataAutomationProject](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation_UpdateDataAutomationProject.html) API. ## AWS CLI Example: ``` aws bedrock-data-automation update-data-automation-project \ --project-arn "arn:aws:bedrock:us-east-1:123456789012:data-automation-project/audio-transcription-project" \ --data-automation-libraries '[]' ``` ## AWS Console Example: 1. Navigate to the "Library details" page for your library 1. Expand "Associated projects" 1. Choose the desired project 1. Choose "Dissociate project" ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/library-dissociate-console.png) ## Regional Availability Data Automation Library is available in the following AWS Regions: | Region Name | Region Code | | --- | --- | | US East (N. Virginia) | us-east-1 | | US West (Oregon) | us-west-2 | | Europe (Ireland) | eu-west-1 | | Europe (London) | eu-west-2 | | Europe (Frankfurt) | eu-central-1 | | Asia Pacific (Mumbai) | ap-south-1 | | Asia Pacific (Sydney) | ap-southeast-2 | # Using the Bedrock Data Automation Console In Amazon Bedrock Data Automation (BDA), two major artifacts are used when processing information. Projects, which store output configurations, and Blueprints which let you customize the output format and extraction logic for your unstructured content. This section will discuss creating Projects and Blueprints in the BDA Console. For more information on how projects work in within BDA, see [Bedrock Data Automation projects](bda-projects.md). To learn more about Blueprints in BDA, see [How to create blueprints for custom outputs](bda-idp.md#how-to-create-blueprints). ## Projects in the BDA Console In the BDA Console, you can create and manage projects. Projects allow you to control which standard outputs are retrieved when running an inference operation, and control how custom outputs are handled during inference. Create a project by: 1. Navigate to the Amazon Bedrock service. From there, select "Data Automation" from the sidebar menu. 1. Select "Create project". 1. Give the project a name and then select "Create project" again. 1. This will take you to the Project Details page where you can see which standard outputs are enabled by your project. You can then control custom outputs by selecting the "Custom output" tab and then selecting the "Create blueprint" option. Additionally, you can add an existing Blueprint to the Project. ## Creating Blueprints in the BDA Console Alternatively, you can select "Data Automation" from the sidebar menu and navigate directly to Custom output setup. In the BDA dashboard, locate and click on the "Custom output setup" tab. This will take you to the Blueprints management page. ## Initiating Blueprint Creation 1. Click on the "Create Blueprint" button to start the process of creating a new Blueprint. 1. Select either "Upload from computer" or "Import from S3", and provide a file that's representative of the files you want to process. 1. Enter a Blueprint prompt for blueprint generation. When entering a prompt, you can specify any fields you would expect to find in your uploaded document. You can also specify data normalizations or validations. 1. Select "Generate blueprint". 1. Give the blueprint a name and choose "Create blueprint". 1. In the Custom blueprints section you can select the "Create blueprint" button. 1. Your blueprint will be created, and you will be able to see the extractions that have been identified based on your provided prompt. ## Previewing the Blueprint Use the preview feature to test your Blueprint with sample data. This allows you to verify that the extraction and formatting are working as expected. Select "save and exit blueprint prompt" to save your blueprint as a resource. ## Managing Blueprints After creation, you can manage your Blueprints from the Blueprints dashboard. Options include: + Editing existing Blueprints + Duplicating Blueprints + Deleting Blueprints + Viewing Blueprint version history You can also add a blueprint to a project by selecting the "Add to project" dropdown menu and selecting a project you have created. ## Using Your Blueprint Once created, you can use your Blueprint in BDA projects or directly in API calls to process your unstructured content. To use a Blueprint: + When making API calls, include the Blueprint ARN in your request parameters. See the section on calling the API for more info. + When making an API call, include the ARN of the Project containing the Blueprint. See the section on calling the API for more info. ## Processing Documents with Console The BDA Console also allows you to easily test and preview the insights that BDA can extract from your unstructured content. These tests can only be performed one document at a time. To process multiple documents, see using the API. You can upload sample documents, images, videos, or audio files, and the Console will display the default insights that BDA can generate, as well as the option to apply any Custom Blueprints you have created. You can test a blueprint you have created and applied to a project by going to the Projects section of Data Automation and then selecting a project. Once you are on the project details page, select "Test". On the Test page you will be able to select a file and choose to process the file with either standard output or a blueprint you have created. # Using the Bedrock Data Automation API The Amazon Bedrock Data Automation (BDA) feature provides a streamlined API workflow for processing your data. For all modalities, this workflow consists of three main steps: creating a project, invoking the analysis, and retrieving the results. To retrieve custom output for your processed data, you provide the Blueprint ARN when you invoke the analysis operation. ## Create a Data Automation Project To begin processing files with BDA, you first need to create a Data Automation Project. This can be done in two ways, with the CreateDataAutomationProject operation or the Amazon Amazon Bedrock Console. ### Using the API When using the API to create a project, you invoke the CreateDataAutomationProject. When creating a project, you must define your configuration settings for the type of file you tend to process (the modality you intend to use). Here's an example of how you might configure the standard output for images: ``` { "standardOutputConfiguration": { "image": { "state": "ENABLED", "extraction": { "category": { "state": "ENABLED", "types": [ "CONTENT_MODERATION", "TEXT_DETECTION" ] }, "boundingBox": { "state": "ENABLED" } }, "generativeField": { "state": "ENABLED", "types": [ "IMAGE_SUMMARY", "IAB" ] } } } } ``` The API validates the input configuration. It creates a new project with a unique ARN. The project settings are stored for future use. If a project is created with no parameters, the default settings will apply. For example, when processing images, image summarization and text detection will be enabled by default. There's a limit to the number of projects that can be created per AWS account. Certain combinations of settings may not be allowed or may require additional permissions. ------ #### [ Async ] **Invoke Data Automation Async** You have a project set up, you can start processing images using the [InvokeDataAutomationAsync](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomationAsync.html) operation. If using custom output, you can only submit a single blueprint ARN per request. This API call initiates the asynchronous processing of your files in a specified S3 bucket. The API accepts the project ARN and the file to be processed, then starts the asynchronous processing job. A job ID is returned for tracking the process. Errors will be raised if the project doesn't exist, or if the caller doesn't have the necessary permissions, or if the input files aren't in a supported format. The following is the structure of the JSON request: ``` { { "blueprints": [ { "blueprintArn": "string", "stage": "string", "version": "string" } ], "clientToken": "string", "dataAutomationConfiguration": { "dataAutomationProjectArn": "string", "stage": "string" }, "dataAutomationProfileArn": "string", "encryptionConfiguration": { "kmsEncryptionContext": { "string" : "string" }, "kmsKeyId": "string" }, "inputConfiguration": { "assetProcessingConfiguration": { "video": { "segmentConfiguration": { ... } } "s3Uri": "string" }, "notificationConfiguration": { "eventBridgeConfiguration": { "eventBridgeEnabled": boolean } }, "outputConfiguration": { "s3Uri": "string" }, "tags": [ { "key": "sstring", "value": "string" } ] } } ``` When you run `InvokeDataAutomationAsync` on a video file, you can set a 5 minute or longer section of a video that will be treated as a full video for the data extraction. This time is set with a timestamp of the starting millisecond and ending millisecond. This information is added in the `assetProcessingConfiguration` element. ------ #### [ Sync ] **Invoke Data Automation** Alternatively, you can use the [InvokeDataAutomation](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_data-automation-runtime_InvokeDataAutomation.html) operation. The `InvokeDataAutomation` operation only supports processing images. This API call initiates the synchronous processing of the provided via an S3 reference, or in the payload. The API accepts the project ARN and the file to be processed, and returns the structured insights in the response. Errors will be raised if the project doesn't exist, or if the caller doesn't have the necessary permissions, or if the input files aren't in a supported format. If the analyzed image is semantically classified as a document, this will also be raised as an error, because the InvokeDataAutomation only supports Images. To prevent this error, you may use Modality Routing on your project to force routing of all image file types as Images (see [Disabling modalities and routing file types](bda-routing-enablement.md)). Here is the structure of the JSON request for both image and document. Sync API request supports both image bytes and S3 bucket. To use image bytes, simply replace `“s3Uri”: “string”` in “inputConfiguration” section with `“bytes“: “base64-encoded string“` `outputConfiguration` is optional with default is inline output. If S3 uri is provided as outputConfiguration, encrypted output will be put into the specified S3 bucket. ``` { { "blueprints": [ { "blueprintArn": "string", //use for image "stage": "string", "version": "string" } ], "dataAutomationConfiguration": { "dataAutomationProjectArn": "string", "stage": "string" }, "dataAutomationProfileArn": "string", "inputConfiguration": { "s3Uri": "string" }, "outputConfiguration": { "s3Uri": "string" } } } ``` The output includes unique structures depending on both the file, operations, and custom output configuration specified in the call to InvokeDataAutomation. Note that this response includes both the standard and custom output responses. ------ Here is the structure of the JSON response with both standard and custom output configuration: ``` { "semanticModality": "IMAGE", "outputSegments": [ { "customOutputStatus": "MATCH", "standardOutput": { "image": { "summary": "This image shows a white Nike running shoe with a black Nike swoosh logo on the side. The shoe has a modern design with a thick, cushioned sole and a sleek upper part. The word \"ROUKEA\" is visible on the sole of the shoe, repeated twice. The shoe appears to be designed for comfort and performance, suitable for running or athletic activities. The background is plain and dark, highlighting the shoe.", "iab_categories": [ { "category": "Style and Fashion", "confidence": 0.9890000000000001, "taxonomy_level": 1, "parent_name": "", "id": "0ebe86c8-e9af-43f6-a7bb-182a61d2e1fd", "type": "IAB" }, { "category": "Men's Fashion", "confidence": 0.9890000000000001, "taxonomy_level": 2, "parent_name": "Style and Fashion", "id": "13bd456a-3e1b-4681-b0dd-f42a8d5e5ad5", "type": "IAB" }, { "category": "Style and Fashion", "confidence": 0.853, "taxonomy_level": 1, "parent_name": "", "id": "177b29a1-0e40-45c1-8540-5f49a3d7ded3", "type": "IAB" }, { "category": "Women's Fashion", "confidence": 0.853, "taxonomy_level": 2, "parent_name": "Style and Fashion", "id": "f0197ede-3ba6-498b-8f7b-43fecc5735ef", "type": "IAB" } ], "content_moderation": [], "logos": [ { "id": "2e109eb6-39f5-4782-826f-911b62d277fb", "type": "LOGOS", "confidence": 0.9170872209665809, "name": "nike", "locations": [ { "bounding_box": { "left": 0.3977411523719743, "top": 0.4922481227565456, "width": 0.2574246356942061, "height": 0.15461772197001689 } } ] } ], "text_words": [ { "id": "f70301df-5725-405e-b50c-612e352467bf", "type": "TEXT_WORD", "confidence": 0.10091366487951722, "text": "ROUKEA", "locations": [ { "bounding_box": { "left": 0.6486002310163024, "top": 0.6783271480251003, "width": 0.13219473954570082, "height": 0.05802226710963898 }, "polygon": [ { "x": 0.6486002310163024, "y": 0.7025876947351404 }, { "x": 0.7760931467045249, "y": 0.6783271480251003 }, { "x": 0.7807949705620032, "y": 0.7120888684246991 }, { "x": 0.6533020989743271, "y": 0.7363494151347393 } ] } ], "line_id": "9147fec0-d869-4d58-933e-93eb7164c404" } ], "text_lines": [ { "id": "9147fec0-d869-4d58-933e-93eb7164c404", "type": "TEXT_LINE", "confidence": 0.10091366487951722, "text": "ROUKEA", "locations": [ { "bounding_box": { "left": 0.6486002310163024, "top": 0.6783271480251003, "width": 0.13219473954570082, "height": 0.05802226710963898 }, "polygon": [ { "x": 0.6486002310163024, "y": 0.7025876947351404 }, { "x": 0.7760931467045249, "y": 0.6783271480251003 }, { "x": 0.7807949705620032, "y": 0.7120888684246991 }, { "x": 0.6533020989743271, "y": 0.7363494151347393 } ] } ] } ] }, "statistics": { "iab_category_count": 4, "content_moderation_count": 0, "logo_count": 1, "line_count": 1, "word_count": 1 }, "metadata": { "semantic_modality": "IMAGE", "image_width_pixels": 173, "image_height_pixels": 148, "image_encoding": "jpeg", "s3_bucket": "test-bucket", "s3_key": "uploads/test-image.jpeg" } }, "customOutput": { "matched_blueprint": { "arn": "arn:aws:bedrock:us-east-1:123456789012:blueprint/test", "version": "1", "name": "test-blueprint", "confidence": 1.0 }, "inference_result": { "product_details": { "product_category": "footwear" }, "image_sentiment": "Positive", "image_background": "Solid color", "image_style": "Product image", "image_humor": false } } } ] } ``` ## Get Data Automation Status To check the status of your processing job and retrieve results, use GetDataAutomationStatus. The GetDataAutomationStatus API allows you to monitor the progress of your job and access the results once processing is complete. The API accepts the invocation ARN returned by InvokeDataAutomationAsync. It checks the current status of the job and returns relevant information. Once the job is complete, it provides the location of the results in S3. If the job is still in progress, it returns the current state (e.g., "InProgress"). If the job is complete, it returns "Success" along with the S3 location of the results. If there was an error, it returns "ServiceError" or "ClientError" with error details. The following is the format of the request JSON: ``` { "InvocationArn": "string" // Arn } ``` ## Async Output Response The results of the file processing are stored in the S3 bucket configured for the input images. The output includes unique structures depending on both the file modality and the operation types specified in the call to InvokeDataAutomationAsync. For information on the standard outputs for a given modality, see [Standard output in Bedrock Data Automation](bda-standard-output.md). As an example, for images it can include information on the following: + Image Summarization: A descriptive summary or caption of the image. + IAB Classification: Categorization based on the IAB taxonomy. + Image Text Detection: Extracted text with bounding box information. + Content Moderation: Detects inappropriate, unwanted, or offensive content in an image. The following is an example snippet of the output for image processing: ``` { "metadata": { "id": "image_123", "semantic_modality": "IMAGE", "s3_bucket": "my-s3-bucket", "s3_prefix": "images/", "image_width_pixels": 1920, "image_height_pixels": 1080 }, "image": { "summary": "A lively party scene with colorful decorations and supplies", "iab_categories": [ { "category": "Party Supplies", "confidence": 0.9, "parent_name": "Events & Attractions" } ], "content_moderation": [ { "category": "Drugs & Tobacco Paraphernalia & Use", "confidence": 0.7 } ], "text_words": [ { "id": "word_1", "text": "lively", "confidence": 0.9, "line_id": "line_1", "locations": [ { "bounding_box": { "left": 100, "top": 200, "width": 50, "height": 20 }, "polygon": [ { "x": 100, "y": 200 }, { "x": 150, "y": 200 }, { "x": 150, "y": 220 }, { "x": 100, "y": 220 } ] } ] } ] } } ``` This structured output allows for easy integration with downstream applications and further analysis. ## Blueprint Optimization APIs ### InvokeBlueprintOptimizationAsync You can improve blueprint accuracy by providing example content assets with the correct expected results. Blueprint instruction optimization uses your examples to refine the natural language instructions in your blueprint fields, which improves your inference *Results* accuracy. For a blueprint, you can call InvokeBlueprintOptimizationAsync API which initiates the asynchronous optimization job to improve Blueprint field instructions based on ground truth data. **Request Body** ``` { "blueprint": { "blueprintArn": "arn:aws:bedrock:us-east-1:123456789012:blueprint/my-document-processor", "stage": "DEVELOPMENT" }, "samples": [ { "assetS3Object": { "s3Uri": "s3://my-optimization-bucket/samples/document1.pdf" }, "groundTruthS3Object": { "s3Uri": "s3://my-optimization-bucket/ground-truth/document1-expected.json" } } ], "outputConfiguration": { "s3Object": { "s3Uri": "s3://my-optimization-bucket/results/optimization-output" } }, "dataAutomationProfileArn": "arn:aws:bedrock:us-east-1:123456789012:data-automation-profile/my-profile" } ``` **Response** ``` { "invocationArn": "arn:aws:bedrock:us-east-1:123456789012:blueprint-optimization-invocation/opt-12345abcdef" } ``` **Important** Save the invocationArn to monitor the optimization job status. ### GetBlueprintOptimizationStatus Retrieves the current status and results of a blueprint optimization job outputted by calling the InvokeBlueprintOptimizationAsync async API. GetBlueprintOptimizationStatus accepts the invocation ARN returned by InvokeBlueprintOptimizationAsync. **Response** ``` { "status": "Success", "outputConfiguration": { "s3Object": { "s3Uri": "s3://my-optimization-bucket/results/optimization-output" } } } ``` **Status Values:** + Created - Job has been created + InProgress - Optimization is running + Success - Optimization completed successfully + ServiceError - Internal service error occurred + ClientError - Invalid request parameters ### CopyBlueprintStage Copies the Blueprint from source stage to the target stage (eg. DEVELOPMENT stage to LIVE stage). This will be used to sync all the configuration including the optimizationSamples field between stages. **Request Body** ``` { "blueprintArn": "arn:aws:bedrock:us-east-1:123456789012:blueprint/my-document-processor", "sourceStage": "DEVELOPMENT", "targetStage": "LIVE" } ``` **Stage Values:** + DEVELOPMENT - Development/testing stage + LIVE - Production stage **Response** ``` {} ``` **Warning** This operation overwrites the target stage configuration and cannot be easily undone. Ensure thorough testing before copying to LIVE stage. # Tagging Inferences and Resources in Bedrock Data Automation To help you manage your BDA resources and inferences, you can assign metadata to either as tags. A tag is a label that you assign to an AWS resource, such as a project, or blueprint. Each tag consists of a key and a value. Tagging resources enables you to catagorize your AWS resources in different ways, for example, by purpose, owner, or application. Tagging inferences works slightly differently. This kind of tagging allows you to catagorize not just projects or blueprints, but specific calls of the BDA API. For more information, see Tagging your AWS resources. On top of catagorization you can use inference tags to see costs allocations each month. For more information, see Use cost allocation tags in the AWS Billing and Cost Management User Guide. Tags help you do the following: + Identify and organize your AWS resources. Many AWS resources support tagging, so you can assign the same tag to resources in different services to indicate that the resources are the same. + Control access to your resources. You can use tags with Amazon Bedrock to create policies to control access to Amazon Bedrock resources. These policies can be attached to an IAM role or user to enable tag-based access control. + Allocate costs. You activate tags on the AWS Billing and Cost Management dashboard. AWS uses the tags to categorize your costs and deliver a monthly cost allocation report to you. Usable with inference tagging only. For more information on tagging resoruces and inferences, such as how to use API tagging operations, see [Tagging Amazon Bedrock resources](tagging.md). # Prerequisites for using Bedrock Data Automation Files for BDA need to meet certain requirements in order to be processed. The following tables show what those requirements are for different file types. ------ #### [ Async ] **Async document file requirements** The following tables show the requirements for files processed using the Invoke Data Automation Async API. **Document file requirements** | Requirement Description | Requirement Details | | --- | --- | | (Console) Maximum number of pages per document file | 20 | | Maximum Number of pages per document while splitter is enabled | 3000 | | (Console) Maximum file size (MB) | 200 | | Maximum file size (MB) | 500 | | Supported File Formats | PDF, TIFF, JPEG, PNG, DOCX | | PDF Specific Limits | The maximum height and width is 40 inches and 9000 points. PDFs cannot be password protected. PDFs can contain JPEG 2000 formatted images. | | Document Rotation and Image Size | BDA supports all in-plane document rotations, for example 45-degree in-plane rotation. BDA supports images with a resolution less than or equal to 10000 pixels on all sides. | | Text Alignment | Text can be text aligned horizontally within the document. Horizontally arrayed text can be read regardless of the degree of rotation of a document. BDA does not support vertical text (text written vertically, as is common in languages like Japanese and Chinese) alignment within the document. | | Character Size | The minimum height for text to be detected is 15 pixels. At 150 DPI, this would be the same as 8 point font. | | Character Type | BDA supports both handwritten and printed character recognition. | **Note** To process DOCX files, they are converted into PDFs. This means page number mapping will not work for DOCX files. Images of the converted PDFs will be uploaded to your output bucket if the JSON\$1 option and page granularity are selected. Blueprint instruction optimization supports all the limits above for documents with the following differences: + A total of 10 document asset examples + 20 pages per document asset example on console and API + 200MB for the total document asset example + Only PDF, DOCX, and TIFF document file formats ------ #### [ Sync ] **Sync document file requirements** The following tables show the requirements for files processed using the Invoke Data Automation API. **Document file requirements** | Requirement Description | Requirement Details | | --- | --- | | (Console) Maximum number of pages per document file | 10 | | Maximum number of pages per document file (splitter is not available) | 10 | | (Console) Maximum file size (MB) | 50 | | Maximum file size (MB) | 50 | | Supported File Formats | PDF, TIFF, JPEG, PNG | | PDF Specific Limits | The maximum height and width is 40 inches and 9000 points. PDFs cannot be password protected. PDFs can contain JPEG 2000 formatted images. | | Document Rotation and Image Size | BDA supports all in-plane document rotations, for example 45-degree in-plane rotation. BDA supports images with a resolution less than or equal to 10000 pixels on all sides. | | Text Alignment | Text can be text aligned horizontally within the document. Horizontally arrayed text can be read regardless of the degree of rotation of a document. BDA does not support vertical text (text written vertically, as is common in languages like Japanese and Chinese) alignment within the document. | | Character Size | The minimum height for text to be detected is 15 pixels. At 150 DPI, this would be the same as 8 point font. | | Character Type | BDA supports both handwritten and printed character recognition. | **Note** Figure captioning works on 20 images per 10 page docuement (sync) and 20 images per page (async). **Tip** Tips to speed up sync API processing: Disable Generative fields unless absolutely required. Select only the granularity and Output text format that you need (vs selecting multiple). Simplify your Blueprint to reduce the number of fields extracted as much as possible. Reduce the number of table and list fields in your blueprint where possible. ------ **Blueprint requirements** | Requirement Description | Requirement Details | | --- | --- | | Maximum number of blueprints per project | 40 | | Maximum number of projects per account | 100 | | Maximum number of blueprints per account | 1000 | | Maximum number of blueprint versions | 100 | | Maximum number of blueprint leaf fields | 100 | | Maximum number of blueprint list leaf fields | 30 | | Maximum blueprint name length | 60 characters | | Maximum blueprint field description length | 600 characters (document), 500 characters (image/video/audio) | | Maximum blueprint field name length | 60 characters | | Maximum blueprint size | 100,000 characters (JSON formatted) | **Image file requirements** | Requirement Description | Requirement Details | | --- | --- | | Maximum File Size (MB) | 5 | | Maximum Resolution | 8k | | Supported File Formats | JPEG, PNG | **Video file requirements** | Requirement Description | Requirement Details | | --- | --- | | Maximum File Size (MB) | 10240 | | Maximum Video Length (Minutes) | 240 | | Supported File Formats | MP4, MOV, AVI, MKV, or WEBM container formats with H.264, H.265/HEVC, VP8, VP9, AV1, or MPEG-4 Visual video codecs | | Maximum Video Blueprints per Project | 1 | | Maximum Video Blueprints per Start Inference request | 1 | | Minimum resolution | 224 | | Maximum resolution | 7680 | | Minimum framerate (Frames per second) | 1 | | Maximum framerate (Frames per second) | 60 | **Audio file requirements** | Requirement Description | Requirement Details | | --- | --- | | Supported Input Languages | English, Germany, Spanish, French, Italian, Portuguese, Japanese, Korean, Chinese, Taiwanese and Cantonese. *\$1All locales supported of the above languages.* | | Supported Output Languages | English, or the dominant language of the audio. | | Minimum Audio Sample Rate (Hz) | 8000 | | Maximum Audio Sample Rate (Hz) | 48000 | | Maximum File Size (MB) | 2048 | | Maximum Audio Length (Minutes) | 240 | | Minimum Audio Length (Milliseconds) | 500 | | Supported File Formats | AMR, FLAC, M4A, MP3, Ogg, WAV | | Maximum Audio Blueprints per Project | 1 | | Maximum Audio Blueprints per Start Inference request | 1 | | Maximum Audio Channels for Audio files | 2 | **Data automation library requirements** | Resource | Limit | | --- | --- | | Maximum number of data automation libraries per account | 10 | | Maximum number of custom vocabulary phrases that can be configured per library (all languages) | 500 | | Maximum total number of characters across all vocabulary phrases per library | 30000 | | Maximum number of concurrent vocabulary ingestion jobs that can run per library | 1 | # Using Amazon Bedrock Data Automation CLI The Amazon Bedrock Data Automation (BDA) feature provides a streamlined CLI workflow for processing your data. For all modalities, this workflow consists of three main steps: creating a project, creating Blueprints for custom output, and processing documents. This guide walks you through the key CLI commands for working with BDA. ## Create your first Data Automation project To begin working with BDA, first create a project using the `create-data-automation-project` command. Consider this sample passport that we'll process: ![\[alt text not found\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/passport2.png) When creating a project, you must define your configuration settings for the type of file you intend to process. The following command represents a minimal working example for creating an image processing project: ``` aws bedrock-data-automation create-data-automation-project \ --project-name "ImageProcessingProject" \ --standard-output-configuration '{ "image": { "extraction": { "category": { "state": "ENABLED", "types": ["TEXT_DETECTION"] }, "boundingBox": { "state": "ENABLED" } }, "generativeField": { "state": "ENABLED" } } }' ``` The command validates the input configuration and creates a new project with a unique ARN. A response would include the project ARN and stage: ``` { "projectArn": "Amazon Resource Name (ARN)", "projectStage": "DEVELOPMENT", "status": "IN_PROGRESS" } ``` If a project is created with no parameters, the default settings will apply. For example, when processing images, image summarization and text detection will be enabled by default. ## Complete parameter reference The following table shows all available parameters for the `create-data-automation-project` command: **Parameters for create-data-automation-project** | Parameter | Required | Default | Description | | --- | --- | --- | --- | | --project-name | Yes | N/A | Name for the Data Automation project | | --project-type | No | The type of the project defines which runtime processing API it may be used with. ASYNC projects may only be used with the invoke-bedrock-data-automation-async API, whereas SYNC projects may only be used with the invoke-bedrock-data-automation API. | | --project-stage | No | LIVE | Stage for the project (DEVELOPMENT or LIVE) | | --standard-output-configuration | Yes | N/A | JSON configuration for standard output processing | | --custom-output-configuration | No | N/A | JSON configuration for custom output processing | | --encryption-configuration | No | N/A | Encryption settings for the project | | --client-token | No | Auto-generated | Unique identifier for request idempotency | ## Creating a Blueprint After creating a project, you can create a Blueprint to define the structure of your data processing using the `create-blueprint` command. Here's a minimal working example for creating a Blueprint tailored to passport processing: ``` aws bedrock-data-automation create-blueprint \ --blueprint-name "passport-blueprint" \ --type "IMAGE" \ --blueprint-stage "DEVELOPMENT" \ --schema '{ "class": "Passport", "description": "Blueprint for processing passport images", "properties": { "passport_number": { "type": "string", "inferenceType": "explicit", "instruction": "The passport identification number" }, "full_name": { "type": "string", "inferenceType": "explicit", "instruction": "The full name of the passport holder" } } }' ``` The command creates a new Blueprint with the specified schema. You can then use this Blueprint when processing documents to extract structured data according to your defined schema. ## Using your Blueprint ### Adding a Blueprint to a project To add a Blueprint to your project, use the `update-data-automation-project` command: ``` aws bedrock-data-automation update-data-automation-project \ --project-arn "Amazon Resource Name (ARN)" \ --standard-output-configuration '{ "image": { "extraction": { "category": { "state": "ENABLED", "types": ["TEXT_DETECTION"] }, "boundingBox": { "state": "ENABLED" } }, "generativeField": { "state": "ENABLED", "types": ["IMAGE_SUMMARY"] } } }' \ --custom-output-configuration '{ "blueprints": [ { "blueprintArn": "Amazon Resource Name (ARN)", "blueprintVersion": "1", "blueprintStage": "LIVE" } ] }' ``` ### Verifying Blueprint integration You can verify the Blueprint integration using the `get-data-automation-project` command: ``` aws bedrock-data-automation get-data-automation-project \ --project-arn "Amazon Resource Name (ARN)" ``` ### Managing multiple Blueprints Use the `list-blueprints` command to view all of your Blueprints: ``` aws bedrock-data-automation list-blueprints ``` ## Process Documents Asynchronously Before processing documents with BDA, you must first upload your documents to an S3 bucket.Once you have a project set up, you can process documents using the `invoke-data-automation-async` command: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://my-bda-documents/invoices/invoice-123.pdf" }' \ --output-configuration '{ "s3Uri": "s3://my-bda-documents/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` The command returns an invocation ARN that you can use to check the processing status: ``` { "invocationArn": "Amazon Resource Name (ARN)" } ``` ## Check Processing Status To check the status of your processing job, use the `get-data-automation-status` command: ``` aws bedrock-data-automation-runtime get-data-automation-status \ --invocation-arn "Amazon Resource Name (ARN)" ``` The command returns the current status of the processing job: ``` { "status": "COMPLETED", "creationTime": "2025-07-09T12:34:56.789Z", "lastModifiedTime": "2025-07-09T12:45:12.345Z", "outputLocation": "s3://my-bda-documents/output/efgh5678/" } ``` Possible status values include: + `IN_PROGRESS`: The processing job is currently running. + `COMPLETED`: The processing job has successfully completed. + `FAILED`: The processing job has failed. Check the response for error details. + `STOPPED`: The processing job was manually stopped. ## Retrieve Results Once processing is complete, you can list the output files in your S3 bucket: ``` aws s3 ls s3://my-bda-documents/output/efgh5678/ ``` To download the results to your local machine: ``` aws s3 cp s3://my-bda-documents/output/efgh5678/ ~/Downloads/bda-results/ --recursive ``` The output includes structured data based on your project configuration and any Blueprints you've applied. ## Process Documents Synchronously Before processing documents with BDA, you must first upload your documents to an S3 bucket. Sync API srupports both input via S3 bucket or image bytes (i.e. processing documents without S3). The command returns structured data based on your project configuration and any Blueprints you've applied: ``` aws bedrock-data-automation-runtime invoke-data-automation \ --input-configuration '{ "s3Uri": "s3://my-bda-documents/invoices/invoice-123.pdf" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` ## Process Images Synchronously The command returns structured data based on your project configuration and any Blueprints you've applied: ``` aws bedrock-data-automation-runtime invoke-data-automation \ --input-configuration '{ "s3Uri": "s3://my-bda-documents/invoices/advertisement_latest.jpeg" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` # Blueprint Operations CLI This guide covers Blueprint operations available through the AWS Command Line Interface (CLI) for Amazon Bedrock Data Automation (BDA). ## Creating Blueprints Blueprints define the structure and properties of data you want to extract from your documents, images, audio, or video files. Use the create-blueprint command to define a new Blueprint. The following command creates a new Blueprint tailored to extract data from a passport image. **Syntax** ``` aws bedrock-data-automation create-blueprint \ --blueprint-name "passport-blueprint" \ --type "IMAGE" \ --blueprint-stage "DEVELOPMENT" \ --schema '{ "class": "Passport", "description": "Blueprint for processing passport images", "properties": { "passport_number": { "type": "string", "inferenceType": "explicit", "instruction": "The passport identification number" }, "full_name": { "type": "string", "inferenceType": "explicit", "instruction": "The full name of the passport holder" }, "expiration_date": { "type": "string", "inferenceType": "explicit", "instruction": "The passport expiration date" } } }' ``` ## Complete parameter reference The following table shows all available parameters for the `create-blueprint` command: **Parameters for create-blueprint** | Parameter | Required | Default | Description | | --- | --- | --- | --- | | --blueprint-name | Yes | N/A | Name for the Blueprint | | --type | Yes | N/A | Type of content (IMAGE, DOCUMENT, AUDIO, VIDEO) | | --blueprint-stage | No | LIVE | Stage for the Blueprint (DEVELOPMENT or LIVE) | | --schema | Yes | N/A | JSON schema defining the Blueprint structure | | --client-token | No | Auto-generated | Unique identifier for request idempotency | ## Viewing Blueprint configurations **List all Blueprints** Use the list-blueprints command to retrieve a list of all Blueprints associated to your account. **Syntax** ``` aws bedrock-data-automation list-blueprints ``` **View Blueprint details** To see detailed information about a specific Blueprint, including its schema and configuration, use the get-blueprint command. **Syntax** ``` aws bedrock-data-automation get-blueprint \ --blueprint-arn "Amazon Resource Name (ARN)" ``` **Inspect specific version** When working with versioned Blueprints, use the get-blueprint command with the --blueprint-version option to view a particular version. **Syntax** ``` aws bedrock-data-automation get-blueprint \ --blueprint-arn "Amazon Resource Name (ARN)" \ --blueprint-version "version-number" ``` **Inspect specific stage** To view Blueprints in either DEVELOPMENT or LIVE stage, use: ``` aws bedrock-data-automation get-blueprint \ --blueprint-arn "Amazon Resource Name (ARN)" \ --blueprint-stage "LIVE" ``` ## Editing Blueprint specifications **Update Blueprint settings** To modify an existing Blueprint's schema or properties, use the update-blueprint command. **Syntax** ``` aws bedrock-data-automation update-blueprint \ --blueprint-arn "Amazon Resource Name (ARN)" \ --schema '{ "class": "Passport", "description": "Updated blueprint for processing passport images", "properties": { "passport_number": { "type": "string", "inferenceType": "explicit", "instruction": "The passport identification number" }, "full_name": { "type": "string", "inferenceType": "explicit", "instruction": "The full name of the passport holder" }, "expiration_date": { "type": "string", "inferenceType": "explicit", "instruction": "The passport expiration date" } } }' ``` **Note:** When updating a Blueprint, you must provide the complete schema, even for fields you're not changing. **Promote to LIVE** To move a Blueprint from DEVELOPMENT to LIVE stage for production, use the update-blueprint command with the --blueprint-stage option. **Syntax** ``` aws bedrock-data-automation update-blueprint \ --blueprint-arn "Amazon Resource Name (ARN)" \ --blueprint-stage "LIVE" ``` **Blueprint versioning** Create a new version of your Blueprint to preserve its current state before making significant changes using the create-blueprint-version command. **Syntax** ``` aws bedrock-data-automation create-blueprint-version \ --blueprint-arn "Amazon Resource Name (ARN)" ``` ## Managing Blueprint tags Tags help users organize and categorize Blueprints for simplified management. **Add tags** Apply metadata to your Blueprint by adding tags. **Syntax** ``` aws bedrock-data-automation tag-resource \ --resource-arn "Amazon Resource Name (ARN)" \ --tags '{"Department":"Finance","Project":"PassportProcessing"}' ``` **Remove tags** Remove specific tags from your Blueprint with the untag-resource command. **Syntax** ``` aws bedrock-data-automation untag-resource \ --resource-arn "Amazon Resource Name (ARN)" \ --tag-keys '["Department","Project"]' ``` **View tags** List all tags associated with your Blueprint using the list-tags-for-resource command. **Syntax** ``` aws bedrock-data-automation list-tags-for-resource \ --resource-arn "Amazon Resource Name (ARN)" ``` ## Deleting Blueprints **Delete an entire Blueprint** Use the delete-blueprint command to permanently remove a Blueprint and all its versions. **Syntax** ``` aws bedrock-data-automation delete-blueprint \ --blueprint-arn "Amazon Resource Name (ARN)" ``` **Caution:** This command permanently deletes a Blueprint and cannot recover it. **Important:** You cannot delete a Blueprint that's currently in use by any projects. Before deleting, ensure the Blueprint isn't referenced in any project's custom output configuration. ## Blueprint Optimization ### Invoking Blueprint Optimization Start an asynchronous blueprint optimization job to improve blueprint's instructions of each of your blueprint fields and result accuracy. **Syntax** ``` aws bedrock-data-automation invoke-blueprint-optimization-async \ --blueprint blueprintArn="arn:aws:bedrock:::blueprint/",stage="DEVELOPMENT" \ --samples '[ { "assetS3Object": { "s3Uri": "s3://my-optimization-bucket/samples/document1.pdf" }, "groundTruthS3Object": { "s3Uri": "s3://my-optimization-bucket/ground-truth/document1-expected.json" } } ]' \ --output-configuration s3Object='{s3Uri="s3://my-optimization-bucket/results/optimization-output"}' \ --data-automation-profile-arn "Amazon Resource Name (ARN):data-automation-profile/default" ``` ### Checking Blueprint Optimization Status Monitor the progress and results of a blueprint optimization job. **Syntax** ``` aws bedrock-data-automation get-blueprint-optimization-status \ --invocation-arn "arn:aws:bedrock:::blueprint-optimization-invocation/opt-12345abcdef" ``` Use this command to track the optimization job status. The response includes the current status (Created, InProgress, Success, ServiceError, or ClientError) and output configuration details when completed. ### Copying Blueprint Stages Copy a Blueprint from one stage to another **Syntax** ``` aws bedrock-data-automation copy-blueprint-stage \ --blueprint-arn "arn:aws:bedrock:::blueprint/" \ --source-stage "DEVELOPMENT" \ --target-stage "LIVE" ``` **Caution:** This command copies the entire Blueprint configuration from the source stage to the target stage, overwriting any existing configuration in the target stage. **Important:** Ensure the Blueprint is thoroughly tested in the source stage before copying to production (LIVE) stage. This operation cannot be easily undone. # Processing through CLI Before processing documents with BDA, you must first upload your documents to an S3 bucket: **Syntax** ``` aws s3 cp [--options] ``` Example: ``` aws s3 cp /local/path/document.pdf s3://my-bda-bucket/input/document.pdf ``` ------ #### [ Async ] **Basic processing command structure** Use the `invoke-data-automation-async` command to process files: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://amzn-s3-demo-bucket/sample-images/sample-image.jpg" }' \ --output-configuration '{ "s3Uri": "s3://amzn-s3-demo-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` **Advanced processing command structure** **Video processing with time segments** For video files, you can specify time segments to process: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://my-bucket/video.mp4", "assetProcessingConfiguration": { "video": { "segmentConfiguration": { "timestampSegment": { "startTimeMillis": 0, "endTimeMillis": 300000 } } } } }' \ --output-configuration '{ "s3Uri": "s3://my-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` **Using custom blueprints** You can specify custom blueprints directly in the command: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://my-bucket/document.pdf" }' \ --output-configuration '{ "s3Uri": "s3://my-bucket/output/" }' \ --blueprints '[ { "blueprintArn": "Amazon Resource Name (ARN)", "version": "1", "stage": "LIVE" } ]' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` **Adding encryption configuration** For enhanced security, you can add encryption configuration: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://my-bucket/document.pdf" }' \ --output-configuration '{ "s3Uri": "s3://my-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --encryption-configuration '{ "kmsKeyId": "Amazon Resource Name (ARN)", "kmsEncryptionContext": { "Department": "Finance", "Project": "DocumentProcessing" } }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` **Event notifications** Enable EventBridge notifications for processing completion: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://my-bucket/document.pdf" }' \ --output-configuration '{ "s3Uri": "s3://my-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --notification-configuration '{ "eventBridgeConfiguration": { "eventBridgeEnabled": true } }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` **Checking processing status** Use the `get-data-automation-status` command to check the status of your processing job: ``` aws bedrock-data-automation-runtime get-data-automation-status \ --invocation-arn "Amazon Resource Name (ARN)" ``` The response will include the current status: ``` { "status": "COMPLETED", "creationTime": "2025-07-24T12:34:56.789Z", "lastModifiedTime": "2025-07-24T12:45:12.345Z", "outputLocation": "s3://my-bucket/output/abcd1234/" } ``` **Retrieve processing results** **Locating output files in S3** List the output files in your S3 bucket: ``` aws s3 ls s3://amzn-s3-demo-bucket/output/ ``` Download the results to your local machine: ``` aws s3 cp s3://amzn-s3-demo-bucket/output/ ~/Downloads/bda-results/ --recursive ``` **Understanding output structure** The output typically includes: + `standard-output.json`: Contains standard extraction results + `custom-output.json`: Contains results from custom blueprints + `metadata.json`: Contains processing metadata and confidence scores **Common response fields** Standard output typically includes: + `extractedData`: The main extracted information + `confidence`: Confidence scores for each extracted field + `metadata`: Processing information including timestamps and model details + `boundingBoxes`: Location information for detected elements (if enabled) **Error handling and troubleshooting** Common error scenarios and solutions: + **Invalid S3 URI**: Ensure your S3 bucket exists and you have proper permissions + **Missing data-automation-profile-arn**: This parameter is required for all processing requests + **Project not found**: Verify your project ARN is correct and the project exists + **Unsupported file format**: Check that your file format is supported by BDA **Adding tags to processing jobs** You can add tags to help organize and track your processing jobs: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://my-bucket/document.pdf" }' \ --output-configuration '{ "s3Uri": "s3://my-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --tags '[ { "key": "Department", "value": "Finance" }, { "key": "Project", "value": "InvoiceProcessing" } ]' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" ``` ------ #### [ Sync ] **Basic processing command structure** Use the `invoke-data-automation` command to process files: ``` aws bedrock-data-automation-runtime invoke-data-automation \ --input-configuration '{ "s3Uri": "s3://amzn-s3-demo-bucket/sample-images/sample-image.jpg" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" --region "aws-region" ``` **Advanced processing command structure** Output to S3 bucket ``` aws bedrock-data-automation-runtime invoke-data-automation \ --input-configuration '{ "s3Uri": "s3://amzn-s3-demo-bucket/sample-images/sample-image.jpg" }' \ --output-configuration '{"s3Uri": "s3://amzn-s3-demo-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" --region "aws-region" //document only ``` Use byte input ``` aws bedrock-data-automation-runtime invoke-data-automation \ --input-configuration '{ "bytes": #blob input }' \ --output-configuration '{"s3Uri": "s3://amzn-s3-demo-bucket/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" --region "aws-region" ``` **Note** **Bytes** A blob of base64-encoded document bytes. The maximum size of a document that's provided in a blob of bytes is 50 MB. Type should be Base64-encoded binary data object. **Use custom blueprints (only for image)** ``` aws bedrock-data-automation-runtime invoke-data-automation \ --input-configuration '{ "s3Uri": "s3://amzn-s3-demo-bucket/sample-images/sample-image.jpg" }' \ --blueprints '[{"blueprintArn": "Amazon Resource Name (ARN)", "version": "1", "stage": "LIVE" } ]' \ --data-automation-profile-arn "Amazon Resource Name (ARN)" --region "aws-region" ``` ------ # Processing use cases Amazon Bedrock Data Automation allows you to process documents, images, audio, and video. through the command line interface (CLI). For each modality, the workflow consists of creating a project, invoking the analysis, and retrieving the result. Choose the tab for your preferred method, and then follow the steps: ------ #### [ Documents ] **Extracting data from a W2** ![\[Sample W2 form with standard fields, demonstrating layout and data fields that will be extracted.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/W2.png) When processing a W2 form, an example schema would be the following: ``` { "class": "W2TaxForm", "description": "Simple schema for extracting key information from W2 tax forms", "properties": { "employerName": { "type": "string", "inferenceType": "explicit", "instruction": "The employer's company name" }, "employeeSSN": { "type": "string", "inferenceType": "explicit", "instruction": "The employee's Social Security Number (SSN)" }, "employeeName": { "type": "string", "inferenceType": "explicit", "instruction": "The employee's full name" }, "wagesAndTips": { "type": "number", "inferenceType": "explicit", "instruction": "Wages, tips, other compensation (Box 1)" }, "federalIncomeTaxWithheld": { "type": "number", "inferenceType": "explicit", "instruction": "Federal income tax withheld (Box 2)" }, "taxYear": { "type": "string", "inferenceType": "explicit", "instruction": "The tax year for this W2 form" } } } ``` The command to invoke the processing of the W2 would be similar to the following: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://w2-processing-bucket-301678011486/input/W2.png" }' \ --output-configuration '{ "s3Uri": "s3://w2-processing-bucket-301678011486/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN):data-automation-profile/default" ``` An example of the expected output is: ``` { "documentType": "W2TaxForm", "extractedData": { "employerName": "The Big Company", "employeeSSN": "123-45-6789", "employeeName": "Jane Doe", "wagesAndTips": 48500.00, "federalIncomeTaxWithheld": 6835.00, "taxYear": "2014" }, "confidence": { "employerName": 0.99, "employeeSSN": 0.97, "employeeName": 0.99, "wagesAndTips": 0.98, "federalIncomeTaxWithheld": 0.97, "taxYear": 0.99 }, "metadata": { "processingTimestamp": "2025-07-23T23:15:30Z", "documentId": "w2-12345", "modelId": "amazon.titan-document-v1", "pageCount": 1 } } ``` ------ #### [ Images ] **Travel advertisement example** ![\[Sample image, demonstrating how users can extract information from advertisements.\]](http://docs.aws.amazon.com/bedrock/latest/userguide/images/bda/TravelAdvertisement.jpg) An example schema for travel advertisements would be the following: ``` { "class": "TravelAdvertisement", "description": "Schema for extracting information from travel advertisement images", "properties": { "destination": { "type": "string", "inferenceType": "explicit", "instruction": "The name of the travel destination being advertised" }, "tagline": { "type": "string", "inferenceType": "explicit", "instruction": "The main promotional text or tagline in the advertisement" }, "landscapeType": { "type": "string", "inferenceType": "explicit", "instruction": "The type of landscape shown (e.g., mountains, beach, forest, etc.)" }, "waterFeatures": { "type": "string", "inferenceType": "explicit", "instruction": "Description of any water features visible in the image (ocean, lake, river, etc.)" }, "dominantColors": { "type": "string", "inferenceType": "explicit", "instruction": "The dominant colors present in the image" }, "advertisementType": { "type": "string", "inferenceType": "explicit", "instruction": "The type of travel advertisement (e.g., destination promotion, tour package, etc.)" } } } ``` The command to invoke the processing of the travel advertisement would be similar to the following: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://travel-ads-bucket-301678011486/input/TravelAdvertisement.jpg" }' \ --output-configuration '{ "s3Uri": "s3://travel-ads-bucket-301678011486/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN):data-automation-profile/default" ``` An example of the expected output is: ``` { "documentType": "TravelAdvertisement", "extractedData": { "destination": "Kauai", "tagline": "Travel to KAUAI", "landscapeType": "Coastal mountains with steep cliffs and valleys", "waterFeatures": "Turquoise ocean with white surf along the coastline", "dominantColors": "Green, blue, turquoise, brown, white", "advertisementType": "Destination promotion" }, "confidence": { "destination": 0.98, "tagline": 0.99, "landscapeType": 0.95, "waterFeatures": 0.97, "dominantColors": 0.96, "advertisementType": 0.92 }, "metadata": { "processingTimestamp": "2025-07-23T23:45:30Z", "documentId": "travel-ad-12345", "modelId": "amazon.titan-image-v1", "imageWidth": 1920, "imageHeight": 1080 } } ``` ------ #### [ Audio ] **Transcribing a phone call** An example schema for a phone call would be the following: ``` { "class": "AudioRecording", "description": "Schema for extracting information from AWS customer call recordings", "properties": { "callType": { "type": "string", "inferenceType": "explicit", "instruction": "The type of call (e.g., technical support, account management, consultation)" }, "participants": { "type": "string", "inferenceType": "explicit", "instruction": "The number and roles of participants in the call" }, "mainTopics": { "type": "string", "inferenceType": "explicit", "instruction": "The main topics or AWS services discussed during the call" }, "customerIssues": { "type": "string", "inferenceType": "explicit", "instruction": "Any customer issues or pain points mentioned during the call" }, "actionItems": { "type": "string", "inferenceType": "explicit", "instruction": "Action items or next steps agreed upon during the call" }, "callDuration": { "type": "string", "inferenceType": "explicit", "instruction": "The duration of the call" }, "callSummary": { "type": "string", "inferenceType": "explicit", "instruction": "A brief summary of the entire call" } } } ``` The command to invoke the processing of a phone call would be similar to the following: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://audio-analysis-bucket-301678011486/input/AWS_TCA-Call-Recording-2.wav" }' \ --output-configuration '{ "s3Uri": "s3://audio-analysis-bucket-301678011486/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN):data-automation-profile/default" ``` An example of the expected output is: ``` { "documentType": "AudioRecording", "extractedData": { "callType": "Technical consultation", "participants": "3 participants: AWS Solutions Architect, AWS Technical Account Manager, and Customer IT Director", "mainTopics": "AWS Bedrock implementation, data processing pipelines, model fine-tuning, and cost optimization", "customerIssues": "Integration challenges with existing ML infrastructure, concerns about latency for real-time processing, questions about data security compliance", "actionItems": [ "AWS team to provide documentation on Bedrock data processing best practices", "Customer to share their current ML architecture diagrams", "Schedule follow-up meeting to review implementation plan", "AWS to provide cost estimation for proposed solution" ], "callDuration": "45 minutes and 23 seconds", "callSummary": "Technical consultation call between AWS team and customer regarding implementation of AWS Bedrock for their machine learning workloads. Discussion covered integration approaches, performance optimization, security considerations, and next steps for implementation planning." }, "confidence": { "callType": 0.94, "participants": 0.89, "mainTopics": 0.92, "customerIssues": 0.87, "actionItems": 0.85, "callDuration": 0.99, "callSummary": 0.93 }, "metadata": { "processingTimestamp": "2025-07-24T00:30:45Z", "documentId": "audio-12345", "modelId": "amazon.titan-audio-v1", "audioDuration": "00:45:23", "audioFormat": "WAV", "sampleRate": "44.1 kHz" }, "transcript": { "segments": [ { "startTime": "00:00:03", "endTime": "00:00:10", "speaker": "Speaker 1", "text": "Hello everyone, thank you for joining today's call about implementing AWS Bedrock for your machine learning workloads." }, { "startTime": "00:00:12", "endTime": "00:00:20", "speaker": "Speaker 2", "text": "Thanks for having us. We're really interested in understanding how Bedrock can help us streamline our document processing pipeline." }, { "startTime": "00:00:22", "endTime": "00:00:35", "speaker": "Speaker 3", "text": "Yes, and specifically we'd like to discuss integration with our existing systems and any potential latency concerns for real-time processing requirements." } // Additional transcript segments would continue here ] } } ``` ------ #### [ Video ] **Processing a video** An example schema for videos would be the following: ``` { "class": "VideoContent", "description": "Schema for extracting information from video content", "properties": { "title": { "type": "string", "inferenceType": "explicit", "instruction": "The title or name of the video content" }, "contentType": { "type": "string", "inferenceType": "explicit", "instruction": "The type of content (e.g., tutorial, competition, documentary, advertisement)" }, "mainSubject": { "type": "string", "inferenceType": "explicit", "instruction": "The main subject or focus of the video" }, "keyPersons": { "type": "string", "inferenceType": "explicit", "instruction": "Key people appearing in the video (hosts, participants, etc.)" }, "keyScenes": { "type": "string", "inferenceType": "explicit", "instruction": "Description of important scenes or segments in the video" }, "audioElements": { "type": "string", "inferenceType": "explicit", "instruction": "Description of notable audio elements (music, narration, dialogue)" }, "summary": { "type": "string", "inferenceType": "explicit", "instruction": "A brief summary of the video content" } } } ``` The command to invoke the processing of the video would be similar to the following: ``` aws bedrock-data-automation-runtime invoke-data-automation-async \ --input-configuration '{ "s3Uri": "s3://video-analysis-bucket-301678011486/input/MakingTheCut.mp4", "assetProcessingConfiguration": { "video": { "segmentConfiguration": { "timestampSegment": { "startTimeMillis": 0, "endTimeMillis": 300000 } } } } }' \ --output-configuration '{ "s3Uri": "s3://video-analysis-bucket-301678011486/output/" }' \ --data-automation-configuration '{ "dataAutomationProjectArn": "Amazon Resource Name (ARN)", "stage": "LIVE" }' \ --data-automation-profile-arn "Amazon Resource Name (ARN):data-automation-profile/default" ``` An example of the expected output is: ``` { "documentType": "VideoContent", "extractedData": { "title": "Making the Cut", "contentType": "Fashion design competition", "mainSubject": "Fashion designers competing to create the best clothing designs", "keyPersons": "Heidi Klum, Tim Gunn, and various fashion designer contestants", "keyScenes": [ "Introduction of the competition and contestants", "Design challenge announcement", "Designers working in their studios", "Runway presentation of designs", "Judges' critique and elimination decision" ], "audioElements": "Background music, host narration, contestant interviews, and design feedback discussions", "summary": "An episode of 'Making the Cut' fashion competition where designers compete in a challenge to create innovative designs. The episode includes the challenge announcement, design process, runway presentation, and judging." }, "confidence": { "title": 0.99, "contentType": 0.95, "mainSubject": 0.92, "keyPersons": 0.88, "keyScenes": 0.90, "audioElements": 0.87, "summary": 0.94 }, "metadata": { "processingTimestamp": "2025-07-24T00:15:30Z", "documentId": "video-12345", "modelId": "amazon.titan-video-v1", "videoDuration": "00:45:23", "analyzedSegment": "00:00:00 - 00:05:00", "resolution": "1920x1080" }, "transcript": { "segments": [ { "startTime": "00:00:05", "endTime": "00:00:12", "speaker": "Heidi Klum", "text": "Welcome to Making the Cut, where we're searching for the next great global fashion brand." }, { "startTime": "00:00:15", "endTime": "00:00:25", "speaker": "Tim Gunn", "text": "Designers, for your first challenge, you'll need to create a look that represents your brand and can be sold worldwide." } // Additional transcript segments would continue here ] } } ``` ------