# Amazon Bedrock Runtime
The following actions are supported by Amazon Bedrock Runtime:
+ [ApplyGuardrail](API_runtime_ApplyGuardrail.md)
+ [Converse](API_runtime_Converse.md)
+ [ConverseStream](API_runtime_ConverseStream.md)
+ [CountTokens](API_runtime_CountTokens.md)
+ [GetAsyncInvoke](API_runtime_GetAsyncInvoke.md)
+ [InvokeModel](API_runtime_InvokeModel.md)
+ [InvokeModelWithBidirectionalStream](API_runtime_InvokeModelWithBidirectionalStream.md)
+ [InvokeModelWithResponseStream](API_runtime_InvokeModelWithResponseStream.md)
+ [ListAsyncInvokes](API_runtime_ListAsyncInvokes.md)
+ [StartAsyncInvoke](API_runtime_StartAsyncInvoke.md)
# ApplyGuardrail
The action to apply a guardrail.
For troubleshooting some of the common errors you might encounter when using the `ApplyGuardrail` API, see [Troubleshooting Amazon Bedrock API Error Codes](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html) in the Amazon Bedrock User Guide
## Request Syntax
```
POST /guardrail/guardrailIdentifier/version/guardrailVersion/apply HTTP/1.1
Content-type: application/json
{
"content": [
{ ... }
],
"outputScope": "string",
"source": "string"
}
```
## URI Request Parameters
The request uses the following URI parameters.
** [guardrailIdentifier](#API_runtime_ApplyGuardrail_RequestSyntax) **
The guardrail identifier used in the request to apply the guardrail.
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: `(|([a-z0-9]+)|(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:guardrail/[a-z0-9]+))`
Required: Yes
** [guardrailVersion](#API_runtime_ApplyGuardrail_RequestSyntax) **
The guardrail version used in the request to apply the guardrail.
Pattern: `(|([1-9][0-9]{0,7})|(DRAFT))`
Required: Yes
## Request Body
The request accepts the following data in JSON format.
** [content](#API_runtime_ApplyGuardrail_RequestSyntax) **
The content details used in the request to apply the guardrail.
Type: Array of [GuardrailContentBlock](API_runtime_GuardrailContentBlock.md) objects
Required: Yes
** [outputScope](#API_runtime_ApplyGuardrail_RequestSyntax) **
Specifies the scope of the output that you get in the response. Set to `FULL` to return the entire output, including any detected and non-detected entries in the response for enhanced debugging.
Note that the full output scope doesn't apply to word filters or regex in sensitive information filters. It does apply to all other filtering policies, including sensitive information with filters that can detect personally identifiable information (PII).
Type: String
Valid Values: `INTERVENTIONS | FULL`
Required: No
** [source](#API_runtime_ApplyGuardrail_RequestSyntax) **
The source of data used in the request to apply the guardrail.
Type: String
Valid Values: `INPUT | OUTPUT`
Required: Yes
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"action": "string",
"actionReason": "string",
"assessments": [
{
"appliedGuardrailDetails": {
"guardrailArn": "string",
"guardrailId": "string",
"guardrailOrigin": [ "string" ],
"guardrailOwnership": "string",
"guardrailVersion": "string"
},
"automatedReasoningPolicy": {
"findings": [
{ ... }
]
},
"contentPolicy": {
"filters": [
{
"action": "string",
"confidence": "string",
"detected": boolean,
"filterStrength": "string",
"type": "string"
}
]
},
"contextualGroundingPolicy": {
"filters": [
{
"action": "string",
"detected": boolean,
"score": number,
"threshold": number,
"type": "string"
}
]
},
"invocationMetrics": {
"guardrailCoverage": {
"images": {
"guarded": number,
"total": number
},
"textCharacters": {
"guarded": number,
"total": number
}
},
"guardrailProcessingLatency": number,
"usage": {
"automatedReasoningPolicies": number,
"automatedReasoningPolicyUnits": number,
"contentPolicyImageUnits": number,
"contentPolicyUnits": number,
"contextualGroundingPolicyUnits": number,
"sensitiveInformationPolicyFreeUnits": number,
"sensitiveInformationPolicyUnits": number,
"topicPolicyUnits": number,
"wordPolicyUnits": number
}
},
"sensitiveInformationPolicy": {
"piiEntities": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
],
"regexes": [
{
"action": "string",
"detected": boolean,
"match": "string",
"name": "string",
"regex": "string"
}
]
},
"topicPolicy": {
"topics": [
{
"action": "string",
"detected": boolean,
"name": "string",
"type": "string"
}
]
},
"wordPolicy": {
"customWords": [
{
"action": "string",
"detected": boolean,
"match": "string"
}
],
"managedWordLists": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
]
}
}
],
"guardrailCoverage": {
"images": {
"guarded": number,
"total": number
},
"textCharacters": {
"guarded": number,
"total": number
}
},
"outputs": [
{
"text": "string"
}
],
"usage": {
"automatedReasoningPolicies": number,
"automatedReasoningPolicyUnits": number,
"contentPolicyImageUnits": number,
"contentPolicyUnits": number,
"contextualGroundingPolicyUnits": number,
"sensitiveInformationPolicyFreeUnits": number,
"sensitiveInformationPolicyUnits": number,
"topicPolicyUnits": number,
"wordPolicyUnits": number
}
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [action](#API_runtime_ApplyGuardrail_ResponseSyntax) **
The action taken in the response from the guardrail.
Type: String
Valid Values: `NONE | GUARDRAIL_INTERVENED`
** [actionReason](#API_runtime_ApplyGuardrail_ResponseSyntax) **
The reason for the action taken when harmful content is detected.
Type: String
** [assessments](#API_runtime_ApplyGuardrail_ResponseSyntax) **
The assessment details in the response from the guardrail.
Type: Array of [GuardrailAssessment](API_runtime_GuardrailAssessment.md) objects
** [guardrailCoverage](#API_runtime_ApplyGuardrail_ResponseSyntax) **
The guardrail coverage details in the apply guardrail response.
Type: [GuardrailCoverage](API_runtime_GuardrailCoverage.md) object
** [outputs](#API_runtime_ApplyGuardrail_ResponseSyntax) **
The output details in the response from the guardrail.
Type: Array of [GuardrailOutputContent](API_runtime_GuardrailOutputContent.md) objects
** [usage](#API_runtime_ApplyGuardrail_ResponseSyntax) **
The usage details in the response from the guardrail.
Type: [GuardrailUsage](API_runtime_GuardrailUsage.md) object
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceQuotaExceededException **
Your request exceeds the service quota for your account. You can view your quotas at [Viewing service quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/gs-request-quota.html). You can resubmit your request later.
HTTP Status Code: 400
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/ApplyGuardrail)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/ApplyGuardrail)
# Converse
Sends messages to the specified Amazon Bedrock model. `Converse` provides a consistent interface that works with all models that support messages. This allows you to write code once and use it with different models. If a model has unique inference parameters, you can also pass those unique parameters to the model.
Amazon Bedrock doesn't store any text, images, or documents that you provide as content. The data is only used to generate the response.
You can submit a prompt by including it in the `messages` field, specifying the `modelId` of a foundation model or inference profile to run inference on it, and including any other fields that are relevant to your use case.
You can also submit a prompt from Prompt management by specifying the ARN of the prompt version and including a map of variables to values in the `promptVariables` field. You can append more messages to the prompt by using the `messages` field. If you use a prompt from Prompt management, you can't include the following fields in the request: `additionalModelRequestFields`, `inferenceConfig`, `system`, or `toolConfig`. Instead, these fields must be defined through Prompt management. For more information, see [Test a prompt using Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-test.html).
For information about the Converse API, see [Use the Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html). To use a guardrail, see [Use a guardrail with the Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html). To use a tool with a model, see [Tool use (Function calling)](https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html).
For example code, see [Converse API examples](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#message-inference-examples).
This operation requires permission for the `bedrock:InvokeModel` action.
**Important**
To deny all inference access to resources that you specify in the modelId field, you need to deny access to the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` actions. Doing this also denies access to the resource through the base inference actions ([InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html)). For more information see [Deny access for inference on specific models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference).
For troubleshooting some of the common errors you might encounter when using the `Converse` API, see [Troubleshooting Amazon Bedrock API Error Codes](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html) in the Amazon Bedrock User Guide
## Request Syntax
```
POST /model/modelId/converse HTTP/1.1
Content-type: application/json
{
"additionalModelRequestFields": JSON value,
"additionalModelResponseFieldPaths": [ "string" ],
"guardrailConfig": {
"guardrailIdentifier": "string",
"guardrailVersion": "string",
"trace": "string"
},
"inferenceConfig": {
"maxTokens": number,
"stopSequences": [ "string" ],
"temperature": number,
"topP": number
},
"messages": [
{
"content": [
{ ... }
],
"role": "string"
}
],
"outputConfig": {
"textFormat": {
"structure": { ... },
"type": "string"
}
},
"performanceConfig": {
"latency": "string"
},
"promptVariables": {
"string" : { ... }
},
"requestMetadata": {
"string" : "string"
},
"serviceTier": {
"type": "string"
},
"system": [
{ ... }
],
"toolConfig": {
"toolChoice": { ... },
"tools": [
{ ... }
]
}
}
```
## URI Request Parameters
The request uses the following URI parameters.
** [modelId](#API_runtime_Converse_RequestSyntax) **
Specifies the model or throughput with which to run inference, or the prompt resource to use in inference. The value depends on the resource that you use:
+ If you use a base model, specify the model ID or its ARN. For a list of model IDs for base models, see [Amazon Bedrock base model IDs (on-demand throughput)](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) in the Amazon Bedrock User Guide.
+ If you use an Amazon Bedrock Marketplace model, specify the ID or ARN of the marketplace endpoint that you created. For more information about Amazon Bedrock Marketplace and setting up an endpoint, see [Amazon Bedrock Marketplace](https://docs.aws.amazon.com/bedrock/latest/userguide/amazon-bedrock-marketplace.html) in the Amazon Bedrock User Guide.
+ If you use an inference profile, specify the inference profile ID or its ARN. For a list of inference profile IDs, see [Supported Regions and models for cross-region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference-support.html) in the Amazon Bedrock User Guide.
+ If you use a prompt created through [Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html), specify the ARN of the prompt version. For more information, see [Test a prompt using Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-test.html).
+ If you use a provisioned model, specify the ARN of the Provisioned Throughput. For more information, see [Run inference using a Provisioned Throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-thru-use.html) in the Amazon Bedrock User Guide.
+ If you use a custom model, specify the ARN of the custom model deployment (for on-demand inference) or the ARN of your provisioned model (for Provisioned Throughput). For more information, see [Use a custom model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-use.html) in the Amazon Bedrock User Guide.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:(([0-9]{12}:custom-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}/[a-z0-9]{12})|(:foundation-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|([0-9]{12}:imported-model/[a-z0-9]{12})|([0-9]{12}:provisioned-model/[a-z0-9]{12})|([0-9]{12}:custom-model-deployment/[a-z0-9]{12})|([0-9]{12}:(inference-profile|application-inference-profile)/[a-zA-Z0-9-:.]+)))|([a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|(([0-9a-zA-Z][_-]?)+)|([a-zA-Z0-9-:.]+)|(^(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:prompt/[0-9a-zA-Z]{10}(?::[0-9]{1,5})?))$|(^arn:aws:sagemaker:[a-z0-9-]+:[0-9]{12}:endpoint/[a-zA-Z0-9-]+$)|(^arn:aws(-[^:]+)?:bedrock:([0-9a-z-]{1,20}):([0-9]{12}):(default-)?prompt-router/[a-zA-Z0-9-:.]+$)`
Required: Yes
## Request Body
The request accepts the following data in JSON format.
** [additionalModelRequestFields](#API_runtime_Converse_RequestSyntax) **
Additional inference parameters that the model supports, beyond the base set of inference parameters that `Converse` and `ConverseStream` support in the `inferenceConfig` field. For more information, see [Model parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
Type: JSON value
Required: No
** [additionalModelResponseFieldPaths](#API_runtime_Converse_RequestSyntax) **
Additional model parameters field paths to return in the response. `Converse` and `ConverseStream` return the requested fields as a JSON Pointer object in the `additionalModelResponseFields` field. The following is example JSON for `additionalModelResponseFieldPaths`.
`[ "/stop_sequence" ]`
For information about the JSON Pointer syntax, see the [Internet Engineering Task Force (IETF)](https://datatracker.ietf.org/doc/html/rfc6901) documentation.
`Converse` and `ConverseStream` reject an empty JSON Pointer or incorrectly structured JSON Pointer with a `400` error code. if the JSON Pointer is valid, but the requested field is not in the model response, it is ignored by `Converse`.
Type: Array of strings
Array Members: Minimum number of 0 items. Maximum number of 10 items.
Length Constraints: Minimum length of 1. Maximum length of 256.
Required: No
** [guardrailConfig](#API_runtime_Converse_RequestSyntax) **
Configuration information for a guardrail that you want to use in the request. If you include `guardContent` blocks in the `content` field in the `messages` field, the guardrail operates only on those messages. If you include no `guardContent` blocks, the guardrail operates on all messages in the request body and in any included prompt resource.
Type: [GuardrailConfiguration](API_runtime_GuardrailConfiguration.md) object
Required: No
** [inferenceConfig](#API_runtime_Converse_RequestSyntax) **
Inference parameters to pass to the model. `Converse` and `ConverseStream` support a base set of inference parameters. If you need to pass additional parameters that the model supports, use the `additionalModelRequestFields` request field.
Type: [InferenceConfiguration](API_runtime_InferenceConfiguration.md) object
Required: No
** [messages](#API_runtime_Converse_RequestSyntax) **
The messages that you want to send to the model.
Type: Array of [Message](API_runtime_Message.md) objects
Required: No
** [outputConfig](#API_runtime_Converse_RequestSyntax) **
Output configuration for a model response.
Type: [OutputConfig](API_runtime_OutputConfig.md) object
Required: No
** [performanceConfig](#API_runtime_Converse_RequestSyntax) **
Model performance settings for the request.
Type: [PerformanceConfiguration](API_runtime_PerformanceConfiguration.md) object
Required: No
** [promptVariables](#API_runtime_Converse_RequestSyntax) **
Contains a map of variables in a prompt from Prompt management to objects containing the values to fill in for them when running model invocation. This field is ignored if you don't specify a prompt resource in the `modelId` field.
Type: String to [PromptVariableValues](API_runtime_PromptVariableValues.md) object map
Required: No
** [requestMetadata](#API_runtime_Converse_RequestSyntax) **
Key-value pairs that you can use to filter invocation logs.
Type: String to string map
Map Entries: Maximum number of 16 items.
Key Length Constraints: Minimum length of 1. Maximum length of 256.
Key Pattern: `[a-zA-Z0-9\s:_@$#=/+,-.]{1,256}`
Value Length Constraints: Minimum length of 0. Maximum length of 256.
Value Pattern: `[a-zA-Z0-9\s:_@$#=/+,-.]{0,256}`
Required: No
** [serviceTier](#API_runtime_Converse_RequestSyntax) **
Specifies the processing tier configuration used for serving the request.
Type: [ServiceTier](API_runtime_ServiceTier.md) object
Required: No
** [system](#API_runtime_Converse_RequestSyntax) **
A prompt that provides instructions or context to the model about the task it should perform, or the persona it should adopt during the conversation.
Type: Array of [SystemContentBlock](API_runtime_SystemContentBlock.md) objects
Required: No
** [toolConfig](#API_runtime_Converse_RequestSyntax) **
Configuration information for the tools that the model can use when generating a response.
For information about models that support tool use, see [Supported models and model features](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features).
Type: [ToolConfiguration](API_runtime_ToolConfiguration.md) object
Required: No
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"additionalModelResponseFields": JSON value,
"metrics": {
"latencyMs": number
},
"output": { ... },
"performanceConfig": {
"latency": "string"
},
"serviceTier": {
"type": "string"
},
"stopReason": "string",
"trace": {
"guardrail": {
"actionReason": "string",
"inputAssessment": {
"string" : {
"appliedGuardrailDetails": {
"guardrailArn": "string",
"guardrailId": "string",
"guardrailOrigin": [ "string" ],
"guardrailOwnership": "string",
"guardrailVersion": "string"
},
"automatedReasoningPolicy": {
"findings": [
{ ... }
]
},
"contentPolicy": {
"filters": [
{
"action": "string",
"confidence": "string",
"detected": boolean,
"filterStrength": "string",
"type": "string"
}
]
},
"contextualGroundingPolicy": {
"filters": [
{
"action": "string",
"detected": boolean,
"score": number,
"threshold": number,
"type": "string"
}
]
},
"invocationMetrics": {
"guardrailCoverage": {
"images": {
"guarded": number,
"total": number
},
"textCharacters": {
"guarded": number,
"total": number
}
},
"guardrailProcessingLatency": number,
"usage": {
"automatedReasoningPolicies": number,
"automatedReasoningPolicyUnits": number,
"contentPolicyImageUnits": number,
"contentPolicyUnits": number,
"contextualGroundingPolicyUnits": number,
"sensitiveInformationPolicyFreeUnits": number,
"sensitiveInformationPolicyUnits": number,
"topicPolicyUnits": number,
"wordPolicyUnits": number
}
},
"sensitiveInformationPolicy": {
"piiEntities": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
],
"regexes": [
{
"action": "string",
"detected": boolean,
"match": "string",
"name": "string",
"regex": "string"
}
]
},
"topicPolicy": {
"topics": [
{
"action": "string",
"detected": boolean,
"name": "string",
"type": "string"
}
]
},
"wordPolicy": {
"customWords": [
{
"action": "string",
"detected": boolean,
"match": "string"
}
],
"managedWordLists": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
]
}
}
},
"modelOutput": [ "string" ],
"outputAssessments": {
"string" : [
{
"appliedGuardrailDetails": {
"guardrailArn": "string",
"guardrailId": "string",
"guardrailOrigin": [ "string" ],
"guardrailOwnership": "string",
"guardrailVersion": "string"
},
"automatedReasoningPolicy": {
"findings": [
{ ... }
]
},
"contentPolicy": {
"filters": [
{
"action": "string",
"confidence": "string",
"detected": boolean,
"filterStrength": "string",
"type": "string"
}
]
},
"contextualGroundingPolicy": {
"filters": [
{
"action": "string",
"detected": boolean,
"score": number,
"threshold": number,
"type": "string"
}
]
},
"invocationMetrics": {
"guardrailCoverage": {
"images": {
"guarded": number,
"total": number
},
"textCharacters": {
"guarded": number,
"total": number
}
},
"guardrailProcessingLatency": number,
"usage": {
"automatedReasoningPolicies": number,
"automatedReasoningPolicyUnits": number,
"contentPolicyImageUnits": number,
"contentPolicyUnits": number,
"contextualGroundingPolicyUnits": number,
"sensitiveInformationPolicyFreeUnits": number,
"sensitiveInformationPolicyUnits": number,
"topicPolicyUnits": number,
"wordPolicyUnits": number
}
},
"sensitiveInformationPolicy": {
"piiEntities": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
],
"regexes": [
{
"action": "string",
"detected": boolean,
"match": "string",
"name": "string",
"regex": "string"
}
]
},
"topicPolicy": {
"topics": [
{
"action": "string",
"detected": boolean,
"name": "string",
"type": "string"
}
]
},
"wordPolicy": {
"customWords": [
{
"action": "string",
"detected": boolean,
"match": "string"
}
],
"managedWordLists": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
]
}
}
]
}
},
"promptRouter": {
"invokedModelId": "string"
}
},
"usage": {
"cacheDetails": [
{
"inputTokens": number,
"ttl": "string"
}
],
"cacheReadInputTokens": number,
"cacheWriteInputTokens": number,
"inputTokens": number,
"outputTokens": number,
"totalTokens": number
}
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [additionalModelResponseFields](#API_runtime_Converse_ResponseSyntax) **
Additional fields in the response that are unique to the model.
Type: JSON value
** [metrics](#API_runtime_Converse_ResponseSyntax) **
Metrics for the call to `Converse`.
Type: [ConverseMetrics](API_runtime_ConverseMetrics.md) object
** [output](#API_runtime_Converse_ResponseSyntax) **
The result from the call to `Converse`.
Type: [ConverseOutput](API_runtime_ConverseOutput.md) object
**Note: **This object is a Union. Only one member of this object can be specified or returned.
** [performanceConfig](#API_runtime_Converse_ResponseSyntax) **
Model performance settings for the request.
Type: [PerformanceConfiguration](API_runtime_PerformanceConfiguration.md) object
** [serviceTier](#API_runtime_Converse_ResponseSyntax) **
Specifies the processing tier configuration used for serving the request.
Type: [ServiceTier](API_runtime_ServiceTier.md) object
** [stopReason](#API_runtime_Converse_ResponseSyntax) **
The reason why the model stopped generating output.
Type: String
Valid Values: `end_turn | tool_use | max_tokens | stop_sequence | guardrail_intervened | content_filtered | malformed_model_output | malformed_tool_use | model_context_window_exceeded`
** [trace](#API_runtime_Converse_ResponseSyntax) **
A trace object that contains information about the Guardrail behavior.
Type: [ConverseTrace](API_runtime_ConverseTrace.md) object
** [usage](#API_runtime_Converse_ResponseSyntax) **
The total number of tokens used in the call to `Converse`. The total includes the tokens input to the model and the tokens generated by the model.
Type: [TokenUsage](API_runtime_TokenUsage.md) object
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ModelErrorException **
The request failed due to an error while processing the model.
** originalStatusCode **
The original status code.
** resourceName **
The resource name.
HTTP Status Code: 424
** ModelNotReadyException **
The model specified in the request is not ready to serve inference requests. The AWS SDK will automatically retry the operation up to 5 times. For information about configuring automatic retries, see [Retry behavior](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html) in the *AWS SDKs and Tools* reference guide.
HTTP Status Code: 429
** ModelTimeoutException **
The request took too long to process. Processing time exceeded the model timeout length.
HTTP Status Code: 408
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## Examples
### Send a message to a model
Send a messsage to Anthropic Claude Sonnet with `Converse`.
#### Sample Request
```
POST /model/anthropic.claude-3-sonnet-20240229-v1:0/converse HTTP/1.1
Content-type: application/json
{
"messages": [
{
"role": "user",
"content": [
{
"text": "Write an article about impact of high inflation to GDP of a country"
}
]
}
],
"system": [{"text" : "You are an economist with access to lots of data"}],
"inferenceConfig": {
"maxTokens": 1000,
"temperature": 0.5
}
}
```
### Example response
Response for the above request.
#### Sample Request
```
HTTP/1.1 200
Content-type: application/json
{
"output": {
"message": {
"content": [
{
"text": ""
}
],
"role": "assistant"
}
},
"stopReason": "end_turn",
"usage": {
"inputTokens": 30,
"outputTokens": 628,
"totalTokens": 658
},
"metrics": {
"latencyMs": 1275
}
}
```
### Send a message with additional model fields
In the following example, the request passess a field (`top_k`) that the `Converse` field doesn't support. You pass the additional field in the `additionalModelRequestFields` field. The example also shows how to set the paths for the additional fields sent in the response from the model.
#### Sample Request
```
POST /model/anthropic.claude-3-sonnet-20240229-v1:0/converse HTTP/1.1
Content-type: application/json
{
"messages": [
{
"role": "user",
"content": [
{
"text": "Provide general steps to debug a BSOD on a Windows laptop."
}
]
}
],
"system": [{"text" : "You are a tech support expert who helps resolve technical issues. Signal 'SUCCESS' if you can resolve the issue, otherwise 'FAILURE'"}],
"inferenceConfig": {
"stopSequences": [ "SUCCESS", "FAILURE" ]
},
"additionalModelRequestFields": {
"top_k": 200
},
"additionalModelResponseFieldPaths": [
"/stop_sequence"
]
}
```
### Example response
Response for the above example.
#### Sample Request
```
HTTP/1.1 200
Content-type: application/json
{
"output": {
"message": {
"content": [
{
"text": ""
}
],
"role": "assistant"
}
},
"additionalModelResponseFields": {
"stop_sequence": "SUCCESS"
},
"stopReason": "stop_sequence",
"usage": {
"inputTokens": 51,
"outputTokens": 442,
"totalTokens": 493
},
"metrics": {
"latencyMs": 7944
}
}
```
### Use an inference profile in a conversation
The following request calls the US Anthropic Claude 3.5 Sonnet inference profile to route traffic to the us-east-1 and us-west-2 regions.
#### Sample Request
```
POST /model/us.anthropic.claude-3-5-sonnet-20240620-v1:0/converse HTTP/1.1
{
"messages": [
{
"role": "user",
"content": [
{
"text": "Hello world"
}
]
}
]
}
```
### Run inference on a prompt resource from Prompt management
Send the following request to run inference on version 1 of a prompt resource from Prompt management whose ID is `PROMPT12345`. Suppose the prompt contains a variable called `{{genre}}`. This request would fill in the variable with the value `pop`. Check that you have `bedrock:RenderPrompt` permissions for the prompt resource. For more information, see [Prerequisites for Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-prereq.html).
#### Sample Request
```
POST /model/arn:aws:bedrock:us-west-2:123456789012:prompt/PROMPT12345:1/converse HTTP/1.1
Content-type: application/json
{
"promptVariables": {
"genre": {
"text": "pop"
}
}
}
```
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/Converse)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/Converse)
# ConverseStream
Sends messages to the specified Amazon Bedrock model and returns the response in a stream. `ConverseStream` provides a consistent API that works with all Amazon Bedrock models that support messages. This allows you to write code once and use it with different models. Should a model have unique inference parameters, you can also pass those unique parameters to the model.
To find out if a model supports streaming, call [GetFoundationModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetFoundationModel.html) and check the `responseStreamingSupported` field in the response.
**Note**
The AWS CLI doesn't support streaming operations in Amazon Bedrock, including `ConverseStream`.
Amazon Bedrock doesn't store any text, images, or documents that you provide as content. The data is only used to generate the response.
You can submit a prompt by including it in the `messages` field, specifying the `modelId` of a foundation model or inference profile to run inference on it, and including any other fields that are relevant to your use case.
You can also submit a prompt from Prompt management by specifying the ARN of the prompt version and including a map of variables to values in the `promptVariables` field. You can append more messages to the prompt by using the `messages` field. If you use a prompt from Prompt management, you can't include the following fields in the request: `additionalModelRequestFields`, `inferenceConfig`, `system`, or `toolConfig`. Instead, these fields must be defined through Prompt management. For more information, see [Test a prompt using Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-test.html).
For information about the Converse API, see [Use the Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html). To use a guardrail, see [Use a guardrail with the Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html). To use a tool with a model, see [Tool use (Function calling)](https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html).
For example code, see [Conversation streaming example](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#messages-streaming-inference-example).
This operation requires permission for the `bedrock:InvokeModelWithResponseStream` action.
**Important**
To deny all inference access to resources that you specify in the modelId field, you need to deny access to the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` actions. Doing this also denies access to the resource through the base inference actions ([InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html)). For more information see [Deny access for inference on specific models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference).
For troubleshooting some of the common errors you might encounter when using the `ConverseStream` API, see [Troubleshooting Amazon Bedrock API Error Codes](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html) in the Amazon Bedrock User Guide
## Request Syntax
```
POST /model/modelId/converse-stream HTTP/1.1
Content-type: application/json
{
"additionalModelRequestFields": JSON value,
"additionalModelResponseFieldPaths": [ "string" ],
"guardrailConfig": {
"guardrailIdentifier": "string",
"guardrailVersion": "string",
"streamProcessingMode": "string",
"trace": "string"
},
"inferenceConfig": {
"maxTokens": number,
"stopSequences": [ "string" ],
"temperature": number,
"topP": number
},
"messages": [
{
"content": [
{ ... }
],
"role": "string"
}
],
"outputConfig": {
"textFormat": {
"structure": { ... },
"type": "string"
}
},
"performanceConfig": {
"latency": "string"
},
"promptVariables": {
"string" : { ... }
},
"requestMetadata": {
"string" : "string"
},
"serviceTier": {
"type": "string"
},
"system": [
{ ... }
],
"toolConfig": {
"toolChoice": { ... },
"tools": [
{ ... }
]
}
}
```
## URI Request Parameters
The request uses the following URI parameters.
** [modelId](#API_runtime_ConverseStream_RequestSyntax) **
Specifies the model or throughput with which to run inference, or the prompt resource to use in inference. The value depends on the resource that you use:
+ If you use a base model, specify the model ID or its ARN. For a list of model IDs for base models, see [Amazon Bedrock base model IDs (on-demand throughput)](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) in the Amazon Bedrock User Guide.
+ If you use an Amazon Bedrock Marketplace model, specify the ID or ARN of the marketplace endpoint that you created. For more information about Amazon Bedrock Marketplace and setting up an endpoint, see [Amazon Bedrock Marketplace](https://docs.aws.amazon.com/bedrock/latest/userguide/amazon-bedrock-marketplace.html) in the Amazon Bedrock User Guide.
+ If you use an inference profile, specify the inference profile ID or its ARN. For a list of inference profile IDs, see [Supported Regions and models for cross-region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference-support.html) in the Amazon Bedrock User Guide.
+ If you use a prompt created through [Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html), specify the ARN of the prompt version. For more information, see [Test a prompt using Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-test.html).
+ If you use a provisioned model, specify the ARN of the Provisioned Throughput. For more information, see [Run inference using a Provisioned Throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-thru-use.html) in the Amazon Bedrock User Guide.
+ If you use a custom model, specify the ARN of the custom model deployment (for on-demand inference) or the ARN of your provisioned model (for Provisioned Throughput). For more information, see [Use a custom model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-use.html) in the Amazon Bedrock User Guide.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:(([0-9]{12}:custom-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}/[a-z0-9]{12})|(:foundation-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|([0-9]{12}:imported-model/[a-z0-9]{12})|([0-9]{12}:provisioned-model/[a-z0-9]{12})|([0-9]{12}:custom-model-deployment/[a-z0-9]{12})|([0-9]{12}:(inference-profile|application-inference-profile)/[a-zA-Z0-9-:.]+)))|([a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|(([0-9a-zA-Z][_-]?)+)|([a-zA-Z0-9-:.]+)|(^(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:prompt/[0-9a-zA-Z]{10}(?::[0-9]{1,5})?))$|(^arn:aws:sagemaker:[a-z0-9-]+:[0-9]{12}:endpoint/[a-zA-Z0-9-]+$)|(^arn:aws(-[^:]+)?:bedrock:([0-9a-z-]{1,20}):([0-9]{12}):(default-)?prompt-router/[a-zA-Z0-9-:.]+$)`
Required: Yes
## Request Body
The request accepts the following data in JSON format.
** [additionalModelRequestFields](#API_runtime_ConverseStream_RequestSyntax) **
Additional inference parameters that the model supports, beyond the base set of inference parameters that `Converse` and `ConverseStream` support in the `inferenceConfig` field. For more information, see [Model parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
Type: JSON value
Required: No
** [additionalModelResponseFieldPaths](#API_runtime_ConverseStream_RequestSyntax) **
Additional model parameters field paths to return in the response. `Converse` and `ConverseStream` return the requested fields as a JSON Pointer object in the `additionalModelResponseFields` field. The following is example JSON for `additionalModelResponseFieldPaths`.
`[ "/stop_sequence" ]`
For information about the JSON Pointer syntax, see the [Internet Engineering Task Force (IETF)](https://datatracker.ietf.org/doc/html/rfc6901) documentation.
`Converse` and `ConverseStream` reject an empty JSON Pointer or incorrectly structured JSON Pointer with a `400` error code. if the JSON Pointer is valid, but the requested field is not in the model response, it is ignored by `Converse`.
Type: Array of strings
Array Members: Minimum number of 0 items. Maximum number of 10 items.
Length Constraints: Minimum length of 1. Maximum length of 256.
Required: No
** [guardrailConfig](#API_runtime_ConverseStream_RequestSyntax) **
Configuration information for a guardrail that you want to use in the request. If you include `guardContent` blocks in the `content` field in the `messages` field, the guardrail operates only on those messages. If you include no `guardContent` blocks, the guardrail operates on all messages in the request body and in any included prompt resource.
Type: [GuardrailStreamConfiguration](API_runtime_GuardrailStreamConfiguration.md) object
Required: No
** [inferenceConfig](#API_runtime_ConverseStream_RequestSyntax) **
Inference parameters to pass to the model. `Converse` and `ConverseStream` support a base set of inference parameters. If you need to pass additional parameters that the model supports, use the `additionalModelRequestFields` request field.
Type: [InferenceConfiguration](API_runtime_InferenceConfiguration.md) object
Required: No
** [messages](#API_runtime_ConverseStream_RequestSyntax) **
The messages that you want to send to the model.
Type: Array of [Message](API_runtime_Message.md) objects
Required: No
** [outputConfig](#API_runtime_ConverseStream_RequestSyntax) **
Output configuration for a model response.
Type: [OutputConfig](API_runtime_OutputConfig.md) object
Required: No
** [performanceConfig](#API_runtime_ConverseStream_RequestSyntax) **
Model performance settings for the request.
Type: [PerformanceConfiguration](API_runtime_PerformanceConfiguration.md) object
Required: No
** [promptVariables](#API_runtime_ConverseStream_RequestSyntax) **
Contains a map of variables in a prompt from Prompt management to objects containing the values to fill in for them when running model invocation. This field is ignored if you don't specify a prompt resource in the `modelId` field.
Type: String to [PromptVariableValues](API_runtime_PromptVariableValues.md) object map
Required: No
** [requestMetadata](#API_runtime_ConverseStream_RequestSyntax) **
Key-value pairs that you can use to filter invocation logs.
Type: String to string map
Map Entries: Maximum number of 16 items.
Key Length Constraints: Minimum length of 1. Maximum length of 256.
Key Pattern: `[a-zA-Z0-9\s:_@$#=/+,-.]{1,256}`
Value Length Constraints: Minimum length of 0. Maximum length of 256.
Value Pattern: `[a-zA-Z0-9\s:_@$#=/+,-.]{0,256}`
Required: No
** [serviceTier](#API_runtime_ConverseStream_RequestSyntax) **
Specifies the processing tier configuration used for serving the request.
Type: [ServiceTier](API_runtime_ServiceTier.md) object
Required: No
** [system](#API_runtime_ConverseStream_RequestSyntax) **
A prompt that provides instructions or context to the model about the task it should perform, or the persona it should adopt during the conversation.
Type: Array of [SystemContentBlock](API_runtime_SystemContentBlock.md) objects
Required: No
** [toolConfig](#API_runtime_ConverseStream_RequestSyntax) **
Configuration information for the tools that the model can use when generating a response.
For information about models that support streaming tool use, see [Supported models and model features](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features).
Type: [ToolConfiguration](API_runtime_ToolConfiguration.md) object
Required: No
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"contentBlockDelta": {
"contentBlockIndex": number,
"delta": { ... }
},
"contentBlockStart": {
"contentBlockIndex": number,
"start": { ... }
},
"contentBlockStop": {
"contentBlockIndex": number
},
"internalServerException": {
},
"messageStart": {
"role": "string"
},
"messageStop": {
"additionalModelResponseFields": JSON value,
"stopReason": "string"
},
"metadata": {
"metrics": {
"latencyMs": number
},
"performanceConfig": {
"latency": "string"
},
"serviceTier": {
"type": "string"
},
"trace": {
"guardrail": {
"actionReason": "string",
"inputAssessment": {
"string" : {
"appliedGuardrailDetails": {
"guardrailArn": "string",
"guardrailId": "string",
"guardrailOrigin": [ "string" ],
"guardrailOwnership": "string",
"guardrailVersion": "string"
},
"automatedReasoningPolicy": {
"findings": [
{ ... }
]
},
"contentPolicy": {
"filters": [
{
"action": "string",
"confidence": "string",
"detected": boolean,
"filterStrength": "string",
"type": "string"
}
]
},
"contextualGroundingPolicy": {
"filters": [
{
"action": "string",
"detected": boolean,
"score": number,
"threshold": number,
"type": "string"
}
]
},
"invocationMetrics": {
"guardrailCoverage": {
"images": {
"guarded": number,
"total": number
},
"textCharacters": {
"guarded": number,
"total": number
}
},
"guardrailProcessingLatency": number,
"usage": {
"automatedReasoningPolicies": number,
"automatedReasoningPolicyUnits": number,
"contentPolicyImageUnits": number,
"contentPolicyUnits": number,
"contextualGroundingPolicyUnits": number,
"sensitiveInformationPolicyFreeUnits": number,
"sensitiveInformationPolicyUnits": number,
"topicPolicyUnits": number,
"wordPolicyUnits": number
}
},
"sensitiveInformationPolicy": {
"piiEntities": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
],
"regexes": [
{
"action": "string",
"detected": boolean,
"match": "string",
"name": "string",
"regex": "string"
}
]
},
"topicPolicy": {
"topics": [
{
"action": "string",
"detected": boolean,
"name": "string",
"type": "string"
}
]
},
"wordPolicy": {
"customWords": [
{
"action": "string",
"detected": boolean,
"match": "string"
}
],
"managedWordLists": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
]
}
}
},
"modelOutput": [ "string" ],
"outputAssessments": {
"string" : [
{
"appliedGuardrailDetails": {
"guardrailArn": "string",
"guardrailId": "string",
"guardrailOrigin": [ "string" ],
"guardrailOwnership": "string",
"guardrailVersion": "string"
},
"automatedReasoningPolicy": {
"findings": [
{ ... }
]
},
"contentPolicy": {
"filters": [
{
"action": "string",
"confidence": "string",
"detected": boolean,
"filterStrength": "string",
"type": "string"
}
]
},
"contextualGroundingPolicy": {
"filters": [
{
"action": "string",
"detected": boolean,
"score": number,
"threshold": number,
"type": "string"
}
]
},
"invocationMetrics": {
"guardrailCoverage": {
"images": {
"guarded": number,
"total": number
},
"textCharacters": {
"guarded": number,
"total": number
}
},
"guardrailProcessingLatency": number,
"usage": {
"automatedReasoningPolicies": number,
"automatedReasoningPolicyUnits": number,
"contentPolicyImageUnits": number,
"contentPolicyUnits": number,
"contextualGroundingPolicyUnits": number,
"sensitiveInformationPolicyFreeUnits": number,
"sensitiveInformationPolicyUnits": number,
"topicPolicyUnits": number,
"wordPolicyUnits": number
}
},
"sensitiveInformationPolicy": {
"piiEntities": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
],
"regexes": [
{
"action": "string",
"detected": boolean,
"match": "string",
"name": "string",
"regex": "string"
}
]
},
"topicPolicy": {
"topics": [
{
"action": "string",
"detected": boolean,
"name": "string",
"type": "string"
}
]
},
"wordPolicy": {
"customWords": [
{
"action": "string",
"detected": boolean,
"match": "string"
}
],
"managedWordLists": [
{
"action": "string",
"detected": boolean,
"match": "string",
"type": "string"
}
]
}
}
]
}
},
"promptRouter": {
"invokedModelId": "string"
}
},
"usage": {
"cacheDetails": [
{
"inputTokens": number,
"ttl": "string"
}
],
"cacheReadInputTokens": number,
"cacheWriteInputTokens": number,
"inputTokens": number,
"outputTokens": number,
"totalTokens": number
}
},
"modelStreamErrorException": {
},
"serviceUnavailableException": {
},
"throttlingException": {
},
"validationException": {
}
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [contentBlockDelta](#API_runtime_ConverseStream_ResponseSyntax) **
The messages output content block delta.
Type: [ContentBlockDeltaEvent](API_runtime_ContentBlockDeltaEvent.md) object
** [contentBlockStart](#API_runtime_ConverseStream_ResponseSyntax) **
Start information for a content block.
Type: [ContentBlockStartEvent](API_runtime_ContentBlockStartEvent.md) object
** [contentBlockStop](#API_runtime_ConverseStream_ResponseSyntax) **
Stop information for a content block.
Type: [ContentBlockStopEvent](API_runtime_ContentBlockStopEvent.md) object
** [internalServerException](#API_runtime_ConverseStream_ResponseSyntax) **
An internal server error occurred. Retry your request.
Type: Exception
HTTP Status Code: 500
** [messageStart](#API_runtime_ConverseStream_ResponseSyntax) **
Message start information.
Type: [MessageStartEvent](API_runtime_MessageStartEvent.md) object
** [messageStop](#API_runtime_ConverseStream_ResponseSyntax) **
Message stop information.
Type: [MessageStopEvent](API_runtime_MessageStopEvent.md) object
** [metadata](#API_runtime_ConverseStream_ResponseSyntax) **
Metadata for the converse output stream.
Type: [ConverseStreamMetadataEvent](API_runtime_ConverseStreamMetadataEvent.md) object
** [modelStreamErrorException](#API_runtime_ConverseStream_ResponseSyntax) **
A streaming error occurred. Retry your request.
Type: Exception
HTTP Status Code: 424
** [serviceUnavailableException](#API_runtime_ConverseStream_ResponseSyntax) **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
Type: Exception
HTTP Status Code: 503
** [throttlingException](#API_runtime_ConverseStream_ResponseSyntax) **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide.
Type: Exception
HTTP Status Code: 429
** [validationException](#API_runtime_ConverseStream_ResponseSyntax) **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide.
Type: Exception
HTTP Status Code: 400
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ModelErrorException **
The request failed due to an error while processing the model.
** originalStatusCode **
The original status code.
** resourceName **
The resource name.
HTTP Status Code: 424
** ModelNotReadyException **
The model specified in the request is not ready to serve inference requests. The AWS SDK will automatically retry the operation up to 5 times. For information about configuring automatic retries, see [Retry behavior](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html) in the *AWS SDKs and Tools* reference guide.
HTTP Status Code: 429
** ModelTimeoutException **
The request took too long to process. Processing time exceeded the model timeout length.
HTTP Status Code: 408
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## Examples
### Send a message to a model and stream the response.
Send a message to Anthropic Claude Sonnet with `ConverseStream` and stream the response.
#### Sample Request
```
POST /model/anthropic.claude-3-sonnet-20240229-v1:0/converse-stream HTTP/1.1
{
"messages": [
{
"role": "user",
"content": [
{
"text": "Write an article about impact of high inflation to GDP of a country"
}
]
}
],
"system": [{"text" : "You are an economist with access to lots of data"}],
"inferenceConfig": {
"maxTokens": 1000,
"temperature": 0.5
}
}
```
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/ConverseStream)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/ConverseStream)
# CountTokens
Returns the token count for a given inference request. This operation helps you estimate token usage before sending requests to foundation models by returning the token count that would be used if the same input were sent to the model in an inference request.
Token counting is model-specific because different models use different tokenization strategies. The token count returned by this operation will match the token count that would be charged if the same input were sent to the model in an `InvokeModel` or `Converse` request.
You can use this operation to:
+ Estimate costs before sending inference requests.
+ Optimize prompts to fit within token limits.
+ Plan for token usage in your applications.
This operation accepts the same input formats as `InvokeModel` and `Converse`, allowing you to count tokens for both raw text inputs and structured conversation formats.
The following operations are related to `CountTokens`:
+ [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) - Sends inference requests to foundation models
+ [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) - Sends conversation-based inference requests to foundation models
## Request Syntax
```
POST /model/modelId/count-tokens HTTP/1.1
Content-type: application/json
{
"input": { ... }
}
```
## URI Request Parameters
The request uses the following URI parameters.
** [modelId](#API_runtime_CountTokens_RequestSyntax) **
The unique identifier or ARN of the foundation model to use for token counting. Each model processes tokens differently, so the token count is specific to the model you specify.
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: `[a-zA-Z_\.\-/0-9:]+`
Required: Yes
## Request Body
The request accepts the following data in JSON format.
** [input](#API_runtime_CountTokens_RequestSyntax) **
The input for which to count tokens. The structure of this parameter depends on whether you're counting tokens for an `InvokeModel` or `Converse` request:
+ For `InvokeModel` requests, provide the request body in the `invokeModel` field
+ For `Converse` requests, provide the messages and system content in the `converse` field
The input format must be compatible with the model specified in the `modelId` parameter.
Type: [CountTokensInput](API_runtime_CountTokensInput.md) object
**Note: **This object is a Union. Only one member of this object can be specified or returned.
Required: Yes
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"inputTokens": number
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [inputTokens](#API_runtime_CountTokens_ResponseSyntax) **
The number of tokens in the provided input according to the specified model's tokenization rules. This count represents the number of input tokens that would be processed if the same input were sent to the model in an inference request. Use this value to estimate costs and ensure your inputs stay within model token limits.
Type: Integer
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/CountTokens)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/CountTokens)
# GetAsyncInvoke
Retrieve information about an asynchronous invocation.
## Request Syntax
```
GET /async-invoke/invocationArn HTTP/1.1
```
## URI Request Parameters
The request uses the following URI parameters.
** [invocationArn](#API_runtime_GetAsyncInvoke_RequestSyntax) **
The invocation's ARN.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:async-invoke/[a-z0-9]{12}`
Required: Yes
## Request Body
The request does not have a request body.
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"clientRequestToken": "string",
"endTime": "string",
"failureMessage": "string",
"invocationArn": "string",
"lastModifiedTime": "string",
"modelArn": "string",
"outputDataConfig": { ... },
"status": "string",
"submitTime": "string"
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [clientRequestToken](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
The invocation's idempotency token.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: `[!-~]*`
** [endTime](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
When the invocation ended.
Type: Timestamp
** [failureMessage](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
An error message.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 2048.
** [invocationArn](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
The invocation's ARN.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:async-invoke/[a-z0-9]{12}`
** [lastModifiedTime](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
The invocation's last modified time.
Type: Timestamp
** [modelArn](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
The invocation's model ARN.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `arn:[a-z0-9\-]+:bedrock:[a-z0-9\-]*:[0-9]*:(provisioned-model|foundation-model)/.+`
** [outputDataConfig](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
Output data settings.
Type: [AsyncInvokeOutputDataConfig](API_runtime_AsyncInvokeOutputDataConfig.md) object
**Note: **This object is a Union. Only one member of this object can be specified or returned.
** [status](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
The invocation's status.
Type: String
Valid Values: `InProgress | Completed | Failed`
** [submitTime](#API_runtime_GetAsyncInvoke_ResponseSyntax) **
When the invocation request was submitted.
Type: Timestamp
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/GetAsyncInvoke)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/GetAsyncInvoke)
# InvokeModel
Invokes the specified Amazon Bedrock model to run inference using the prompt and inference parameters provided in the request body. You use model inference to generate text, images, and embeddings.
For example code, see [Invoke model code examples](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-invoke.html#inference-example-invoke).
This operation requires permission for the `bedrock:InvokeModel` action.
**Important**
To deny all inference access to resources that you specify in the modelId field, you need to deny access to the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` actions. Doing this also denies access to the resource through the Converse API actions ([Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) and [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)). For more information see [Deny access for inference on specific models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference).
For troubleshooting some of the common errors you might encounter when using the `InvokeModel` API, see [Troubleshooting Amazon Bedrock API Error Codes](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html) in the Amazon Bedrock User Guide
## Request Syntax
```
POST /model/modelId/invoke HTTP/1.1
Content-Type: contentType
Accept: accept
X-Amzn-Bedrock-Trace: trace
X-Amzn-Bedrock-GuardrailIdentifier: guardrailIdentifier
X-Amzn-Bedrock-GuardrailVersion: guardrailVersion
X-Amzn-Bedrock-PerformanceConfig-Latency: performanceConfigLatency
X-Amzn-Bedrock-Service-Tier: serviceTier
body
```
## URI Request Parameters
The request uses the following URI parameters.
** [accept](#API_runtime_InvokeModel_RequestSyntax) **
The desired MIME type of the inference body in the response. The default value is `application/json`.
** [contentType](#API_runtime_InvokeModel_RequestSyntax) **
The MIME type of the input data in the request. You must specify `application/json`.
** [guardrailIdentifier](#API_runtime_InvokeModel_RequestSyntax) **
The unique identifier of the guardrail that you want to use. If you don't provide a value, no guardrail is applied to the invocation.
An error will be thrown in the following situations.
+ You don't provide a guardrail identifier but you specify the `amazon-bedrock-guardrailConfig` field in the request body.
+ You enable the guardrail but the `contentType` isn't `application/json`.
+ You provide a guardrail identifier, but `guardrailVersion` isn't specified.
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: `(|([a-z0-9]+)|(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:guardrail/[a-z0-9]+))`
** [guardrailVersion](#API_runtime_InvokeModel_RequestSyntax) **
The version number for the guardrail. The value can also be `DRAFT`.
Pattern: `(|([1-9][0-9]{0,7})|(DRAFT))`
** [modelId](#API_runtime_InvokeModel_RequestSyntax) **
Specifies the model or throughput with which to run inference, or the prompt resource to use in inference. The value depends on the resource that you use:
+ If you use a base model, specify the model ID or its ARN. For a list of model IDs for base models, see [Amazon Bedrock base model IDs (on-demand throughput)](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) in the Amazon Bedrock User Guide.
+ If you use an Amazon Bedrock Marketplace model, specify the ID or ARN of the marketplace endpoint that you created. For more information about Amazon Bedrock Marketplace and setting up an endpoint, see [Amazon Bedrock Marketplace](https://docs.aws.amazon.com/bedrock/latest/userguide/amazon-bedrock-marketplace.html) in the Amazon Bedrock User Guide.
+ If you use an inference profile, specify the inference profile ID or its ARN. For a list of inference profile IDs, see [Supported Regions and models for cross-region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference-support.html) in the Amazon Bedrock User Guide.
+ If you use a prompt created through [Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html), specify the ARN of the prompt version. For more information, see [Test a prompt using Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-test.html).
+ If you use a provisioned model, specify the ARN of the Provisioned Throughput. For more information, see [Run inference using a Provisioned Throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-thru-use.html) in the Amazon Bedrock User Guide.
+ If you use a custom model, specify the ARN of the custom model deployment (for on-demand inference) or the ARN of your provisioned model (for Provisioned Throughput). For more information, see [Use a custom model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-use.html) in the Amazon Bedrock User Guide.
+ If you use an [imported model](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html), specify the ARN of the imported model. You can get the model ARN from a successful call to [CreateModelImportJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelImportJob.html) or from the Imported models page in the Amazon Bedrock console.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:(([0-9]{12}:custom-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}/[a-z0-9]{12})|(:foundation-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|([0-9]{12}:imported-model/[a-z0-9]{12})|([0-9]{12}:provisioned-model/[a-z0-9]{12})|([0-9]{12}:custom-model-deployment/[a-z0-9]{12})|([0-9]{12}:(inference-profile|application-inference-profile)/[a-zA-Z0-9-:.]+)))|([a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|(([0-9a-zA-Z][_-]?)+)|([a-zA-Z0-9-:.]+)$|(^(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:prompt/[0-9a-zA-Z]{10}(?::[0-9]{1,5})?))$|(^arn:aws:sagemaker:[a-z0-9-]+:[0-9]{12}:endpoint/[a-zA-Z0-9-]+$)|(^arn:aws(-[^:]+)?:bedrock:([0-9a-z-]{1,20}):([0-9]{12}):(default-)?prompt-router/[a-zA-Z0-9-:.]+$)`
Required: Yes
** [performanceConfigLatency](#API_runtime_InvokeModel_RequestSyntax) **
Model performance settings for the request.
Valid Values: `standard | optimized`
** [serviceTier](#API_runtime_InvokeModel_RequestSyntax) **
Specifies the processing tier type used for serving the request.
Valid Values: `priority | default | flex | reserved`
** [trace](#API_runtime_InvokeModel_RequestSyntax) **
Specifies whether to enable or disable the Bedrock trace. If enabled, you can see the full Bedrock trace.
Valid Values: `ENABLED | DISABLED | ENABLED_FULL`
## Request Body
The request accepts the following binary data.
** [body](#API_runtime_InvokeModel_RequestSyntax) **
The prompt and inference parameters in the format specified in the `contentType` in the header. You must provide the body in JSON format. To see the format and content of the request and response bodies for different models, refer to [Inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html). For more information, see [Run inference](https://docs.aws.amazon.com/bedrock/latest/userguide/api-methods-run.html) in the Bedrock User Guide.
Length Constraints: Minimum length of 0. Maximum length of 25000000.
## Response Syntax
```
HTTP/1.1 200
Content-Type: contentType
X-Amzn-Bedrock-PerformanceConfig-Latency: performanceConfigLatency
X-Amzn-Bedrock-Service-Tier: serviceTier
body
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The response returns the following HTTP headers.
** [contentType](#API_runtime_InvokeModel_ResponseSyntax) **
The MIME type of the inference result.
** [performanceConfigLatency](#API_runtime_InvokeModel_ResponseSyntax) **
Model performance settings for the request.
Valid Values: `standard | optimized`
** [serviceTier](#API_runtime_InvokeModel_ResponseSyntax) **
Specifies the processing tier type used for serving the request.
Valid Values: `priority | default | flex | reserved`
The response returns the following as the HTTP body.
** [body](#API_runtime_InvokeModel_ResponseSyntax) **
Inference response from the model in the format specified in the `contentType` header. To see the format and content of the request and response bodies for different models, refer to [Inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html).
Length Constraints: Minimum length of 0. Maximum length of 25000000.
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ModelErrorException **
The request failed due to an error while processing the model.
** originalStatusCode **
The original status code.
** resourceName **
The resource name.
HTTP Status Code: 424
** ModelNotReadyException **
The model specified in the request is not ready to serve inference requests. The AWS SDK will automatically retry the operation up to 5 times. For information about configuring automatic retries, see [Retry behavior](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html) in the *AWS SDKs and Tools* reference guide.
HTTP Status Code: 429
** ModelTimeoutException **
The request took too long to process. Processing time exceeded the model timeout length.
HTTP Status Code: 408
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceQuotaExceededException **
Your request exceeds the service quota for your account. You can view your quotas at [Viewing service quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/gs-request-quota.html). You can resubmit your request later.
HTTP Status Code: 400
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## Examples
### Run inference on a text model
Send an invoke request to run inference on a Titan Text G1 - Express model. We set the `accept` parameter to accept any content type in the response.
#### Sample Request
```
POST https://bedrock-runtime.us-east-1.amazonaws.com/model/amazon.titan-text-express-v1/invoke
-H accept: */*
-H content-type: application/json
Payload
{"inputText": "Hello world"}
```
### Run inference on an image model
In the following example, the request sets the `accept` parameter to `image/png`.
#### Sample Request
```
POST https://bedrock-runtime.us-east-1.amazonaws.com/model/stability.stable-diffusion-xl-v1/invoke
-H accept: image/png
-H content-type: application/json
Payload
{"inputText": "Picture of a bird"}
```
### Use a guardrail
This example shows how to use a guardrail with `InvokeModel`.
#### Sample Request
```
POST /model/modelId/invoke HTTP/1.1
Accept: accept
Content-Type: contentType
X-Amzn-Bedrock-GuardrailIdentifier: guardrailIdentifier
X-Amzn-Bedrock-GuardrailVersion: guardrailVersion
X-Amzn-Bedrock-GuardrailTrace: guardrailTrace
X-Amzn-Bedrock-Trace: trace
body
// body
{
"amazon-bedrock-guardrailConfig": {
"tagSuffix": "string"
}
}
```
### Example response
This is an example response from `InvokeModel` when using a guardrail.
#### Sample Request
```
HTTP/1.1 200
Content-Type: contentType
body
// body
{
"amazon-bedrock-guardrailAction": "INTERVENED | NONE",
"amazon-bedrock-trace": {
"guardrails": {
// Detailed guardrail trace
}
}
}
```
### Use an inference profile in model invocation
The following request calls the US Anthropic Claude 3.5 Sonnet inference profile to route traffic to the us-east-1 and us-west-2 regions.
#### Sample Request
```
POST /model/us.anthropic.claude-3-5-sonnet-20240620-v1:0/invoke HTTP/1.1
{
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": 1024,
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Hello world"
}
]
}
]
}
```
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/InvokeModel)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/InvokeModel)
# InvokeModelWithBidirectionalStream
Invoke the specified Amazon Bedrock model to run inference using the bidirectional stream. The response is returned in a stream that remains open for 8 minutes. A single session can contain multiple prompts and responses from the model. The prompts to the model are provided as audio files and the model's responses are spoken back to the user and transcribed.
It is possible for users to interrupt the model's response with a new prompt, which will halt the response speech. The model will retain contextual awareness of the conversation while pivoting to respond to the new prompt.
This operation requires permission for the `bedrock:InvokeModel` action.
Note that to log `InvokeModelWithBidirectionalStream` API calls in CloudTrail, you need to configure advanced event selectors to record data events for the `AWS::Bedrock::Model` resource type and `AWS:Bedrock::AsyncInvoke`. To learn more, see [Monitor Amazon Bedrock API calls using CloudTrail](https://docs.aws.amazon.com/bedrock/latest/userguide/logging-using-cloudtrail.html).
**Important**
To deny all inference access to resources that you specify in the modelId field, you need to deny access to the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` actions. Doing this also denies access to the resource through the Converse API actions ([Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) and [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)). For more information see [Deny access for inference on specific models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference).
For more information, see [Using the Amazon Nova Sonic Speech-to-Speech model](https://docs.aws.amazon.com/nova/latest/userguide/speech.html).
## Request Syntax
```
POST /model/modelId/invoke-with-bidirectional-stream HTTP/1.1
Content-type: application/json
{
"chunk": {
"bytes": blob
}
}
```
## URI Request Parameters
The request uses the following URI parameters.
** [modelId](#API_runtime_InvokeModelWithBidirectionalStream_RequestSyntax) **
The model ID or ARN of the model ID to use. Currently, only `amazon.nova-sonic-v1:0` is supported.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:(([0-9]{12}:custom-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}/[a-z0-9]{12})|(:foundation-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|([0-9]{12}:imported-model/[a-z0-9]{12})|([0-9]{12}:provisioned-model/[a-z0-9]{12})|([0-9]{12}:custom-model-deployment/[a-z0-9]{12})|([0-9]{12}:(inference-profile|application-inference-profile)/[a-zA-Z0-9-:.]+)))|([a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|(([0-9a-zA-Z][_-]?)+)|([a-zA-Z0-9-:.]+)$|(^(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:prompt/[0-9a-zA-Z]{10}(?::[0-9]{1,5})?))$|(^arn:aws:sagemaker:[a-z0-9-]+:[0-9]{12}:endpoint/[a-zA-Z0-9-]+$)|(^arn:aws(-[^:]+)?:bedrock:([0-9a-z-]{1,20}):([0-9]{12}):(default-)?prompt-router/[a-zA-Z0-9-:.]+$)`
Required: Yes
## Request Body
The request accepts the following data in JSON format.
** [chunk](#API_runtime_InvokeModelWithBidirectionalStream_RequestSyntax) **
The audio chunk that is used as input for the invocation step.
Type: [BidirectionalInputPayloadPart](API_runtime_BidirectionalInputPayloadPart.md) object
Required: No
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"chunk": {
"bytes": blob
},
"internalServerException": {
},
"modelStreamErrorException": {
},
"modelTimeoutException": {
},
"serviceUnavailableException": {
},
"throttlingException": {
},
"validationException": {
}
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [chunk](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The speech chunk that was provided as output from the invocation step.
Type: [BidirectionalOutputPayloadPart](API_runtime_BidirectionalOutputPayloadPart.md) object
** [internalServerException](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The request encountered an unknown internal error.
Type: Exception
HTTP Status Code: 500
** [modelStreamErrorException](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The request encountered an error with the model stream.
Type: Exception
HTTP Status Code: 424
** [modelTimeoutException](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The connection was closed because a request was not received within the timeout period.
Type: Exception
HTTP Status Code: 408
** [serviceUnavailableException](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The request has failed due to a temporary failure of the server.
Type: Exception
HTTP Status Code: 503
** [throttlingException](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The request was denied due to request throttling.
Type: Exception
HTTP Status Code: 429
** [validationException](#API_runtime_InvokeModelWithBidirectionalStream_ResponseSyntax) **
The input fails to satisfy the constraints specified by an AWS service.
Type: Exception
HTTP Status Code: 400
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ModelErrorException **
The request failed due to an error while processing the model.
** originalStatusCode **
The original status code.
** resourceName **
The resource name.
HTTP Status Code: 424
** ModelNotReadyException **
The model specified in the request is not ready to serve inference requests. The AWS SDK will automatically retry the operation up to 5 times. For information about configuring automatic retries, see [Retry behavior](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html) in the *AWS SDKs and Tools* reference guide.
HTTP Status Code: 429
** ModelStreamErrorException **
An error occurred while streaming the response. Retry your request.
** originalMessage **
The original message.
** originalStatusCode **
The original status code.
HTTP Status Code: 424
** ModelTimeoutException **
The request took too long to process. Processing time exceeded the model timeout length.
HTTP Status Code: 408
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceQuotaExceededException **
Your request exceeds the service quota for your account. You can view your quotas at [Viewing service quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/gs-request-quota.html). You can resubmit your request later.
HTTP Status Code: 400
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/InvokeModelWithBidirectionalStream)
# InvokeModelWithResponseStream
Invoke the specified Amazon Bedrock model to run inference using the prompt and inference parameters provided in the request body. The response is returned in a stream.
To see if a model supports streaming, call [GetFoundationModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetFoundationModel.html) and check the `responseStreamingSupported` field in the response.
**Note**
The AWS CLI doesn't support streaming operations in Amazon Bedrock, including `InvokeModelWithResponseStream`.
For example code, see [Invoke model with streaming code example](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-invoke.html#inference-examples-stream).
This operation requires permissions to perform the `bedrock:InvokeModelWithResponseStream` action.
**Important**
To deny all inference access to resources that you specify in the modelId field, you need to deny access to the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` actions. Doing this also denies access to the resource through the Converse API actions ([Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) and [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)). For more information see [Deny access for inference on specific models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference).
For troubleshooting some of the common errors you might encounter when using the `InvokeModelWithResponseStream` API, see [Troubleshooting Amazon Bedrock API Error Codes](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html) in the Amazon Bedrock User Guide
## Request Syntax
```
POST /model/modelId/invoke-with-response-stream HTTP/1.1
Content-Type: contentType
X-Amzn-Bedrock-Accept: accept
X-Amzn-Bedrock-Trace: trace
X-Amzn-Bedrock-GuardrailIdentifier: guardrailIdentifier
X-Amzn-Bedrock-GuardrailVersion: guardrailVersion
X-Amzn-Bedrock-PerformanceConfig-Latency: performanceConfigLatency
X-Amzn-Bedrock-Service-Tier: serviceTier
body
```
## URI Request Parameters
The request uses the following URI parameters.
** [accept](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
The desired MIME type of the inference body in the response. The default value is `application/json`.
** [contentType](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
The MIME type of the input data in the request. You must specify `application/json`.
** [guardrailIdentifier](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
The unique identifier of the guardrail that you want to use. If you don't provide a value, no guardrail is applied to the invocation.
An error is thrown in the following situations.
+ You don't provide a guardrail identifier but you specify the `amazon-bedrock-guardrailConfig` field in the request body.
+ You enable the guardrail but the `contentType` isn't `application/json`.
+ You provide a guardrail identifier, but `guardrailVersion` isn't specified.
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: `(|([a-z0-9]+)|(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:guardrail/[a-z0-9]+))`
** [guardrailVersion](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
The version number for the guardrail. The value can also be `DRAFT`.
Pattern: `(|([1-9][0-9]{0,7})|(DRAFT))`
** [modelId](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
Specifies the model or throughput with which to run inference, or the prompt resource to use in inference. The value depends on the resource that you use:
+ If you use a base model, specify the model ID or its ARN. For a list of model IDs for base models, see [Amazon Bedrock base model IDs (on-demand throughput)](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) in the Amazon Bedrock User Guide.
+ If you use an Amazon Bedrock Marketplace model, specify the ID or ARN of the marketplace endpoint that you created. For more information about Amazon Bedrock Marketplace and setting up an endpoint, see [Amazon Bedrock Marketplace](https://docs.aws.amazon.com/bedrock/latest/userguide/amazon-bedrock-marketplace.html) in the Amazon Bedrock User Guide.
+ If you use an inference profile, specify the inference profile ID or its ARN. For a list of inference profile IDs, see [Supported Regions and models for cross-region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference-support.html) in the Amazon Bedrock User Guide.
+ If you use a prompt created through [Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management.html), specify the ARN of the prompt version. For more information, see [Test a prompt using Prompt management](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-management-test.html).
+ If you use a provisioned model, specify the ARN of the Provisioned Throughput. For more information, see [Run inference using a Provisioned Throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-thru-use.html) in the Amazon Bedrock User Guide.
+ If you use a custom model, specify the ARN of the custom model deployment (for on-demand inference) or the ARN of your provisioned model (for Provisioned Throughput). For more information, see [Use a custom model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-use.html) in the Amazon Bedrock User Guide.
+ If you use an [imported model](https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-import-model.html), specify the ARN of the imported model. You can get the model ARN from a successful call to [CreateModelImportJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelImportJob.html) or from the Imported models page in the Amazon Bedrock console.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:(([0-9]{12}:custom-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}/[a-z0-9]{12})|(:foundation-model/[a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|([0-9]{12}:imported-model/[a-z0-9]{12})|([0-9]{12}:provisioned-model/[a-z0-9]{12})|([0-9]{12}:custom-model-deployment/[a-z0-9]{12})|([0-9]{12}:(inference-profile|application-inference-profile)/[a-zA-Z0-9-:.]+)))|([a-z0-9-]{1,63}[.]{1}[a-z0-9-]{1,63}([.:]?[a-z0-9-]{1,63}))|(([0-9a-zA-Z][_-]?)+)|([a-zA-Z0-9-:.]+)$|(^(arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:prompt/[0-9a-zA-Z]{10}(?::[0-9]{1,5})?))$|(^arn:aws:sagemaker:[a-z0-9-]+:[0-9]{12}:endpoint/[a-zA-Z0-9-]+$)|(^arn:aws(-[^:]+)?:bedrock:([0-9a-z-]{1,20}):([0-9]{12}):(default-)?prompt-router/[a-zA-Z0-9-:.]+$)`
Required: Yes
** [performanceConfigLatency](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
Model performance settings for the request.
Valid Values: `standard | optimized`
** [serviceTier](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
Specifies the processing tier type used for serving the request.
Valid Values: `priority | default | flex | reserved`
** [trace](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
Specifies whether to enable or disable the Bedrock trace. If enabled, you can see the full Bedrock trace.
Valid Values: `ENABLED | DISABLED | ENABLED_FULL`
## Request Body
The request accepts the following binary data.
** [body](#API_runtime_InvokeModelWithResponseStream_RequestSyntax) **
The prompt and inference parameters in the format specified in the `contentType` in the header. You must provide the body in JSON format. To see the format and content of the request and response bodies for different models, refer to [Inference parameters](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html). For more information, see [Run inference](https://docs.aws.amazon.com/bedrock/latest/userguide/api-methods-run.html) in the Bedrock User Guide.
Length Constraints: Minimum length of 0. Maximum length of 25000000.
## Response Syntax
```
HTTP/1.1 200
X-Amzn-Bedrock-Content-Type: contentType
X-Amzn-Bedrock-PerformanceConfig-Latency: performanceConfigLatency
X-Amzn-Bedrock-Service-Tier: serviceTier
Content-type: application/json
{
"chunk": {
"bytes": blob
},
"internalServerException": {
},
"modelStreamErrorException": {
},
"modelTimeoutException": {
},
"serviceUnavailableException": {
},
"throttlingException": {
},
"validationException": {
}
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The response returns the following HTTP headers.
** [contentType](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
The MIME type of the inference result.
** [performanceConfigLatency](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
Model performance settings for the request.
Valid Values: `standard | optimized`
** [serviceTier](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
Specifies the processing tier type used for serving the request.
Valid Values: `priority | default | flex | reserved`
The following data is returned in JSON format by the service.
** [chunk](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
Content included in the response.
Type: [PayloadPart](API_runtime_PayloadPart.md) object
** [internalServerException](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
An internal server error occurred. Retry your request.
Type: Exception
HTTP Status Code: 500
** [modelStreamErrorException](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
An error occurred while streaming the response. Retry your request.
Type: Exception
HTTP Status Code: 424
** [modelTimeoutException](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
The request took too long to process. Processing time exceeded the model timeout length.
Type: Exception
HTTP Status Code: 408
** [serviceUnavailableException](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
The service isn't available. Try again later.
Type: Exception
HTTP Status Code: 503
** [throttlingException](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
Your request was throttled because of service-wide limitations. Resubmit your request later or in a different region. You can also purchase [Provisioned Throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html) to increase the rate or number of tokens you can process.
Type: Exception
HTTP Status Code: 429
** [validationException](#API_runtime_InvokeModelWithResponseStream_ResponseSyntax) **
Input validation failed. Check your request parameters and retry the request.
Type: Exception
HTTP Status Code: 400
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ModelErrorException **
The request failed due to an error while processing the model.
** originalStatusCode **
The original status code.
** resourceName **
The resource name.
HTTP Status Code: 424
** ModelNotReadyException **
The model specified in the request is not ready to serve inference requests. The AWS SDK will automatically retry the operation up to 5 times. For information about configuring automatic retries, see [Retry behavior](https://docs.aws.amazon.com/sdkref/latest/guide/feature-retry-behavior.html) in the *AWS SDKs and Tools* reference guide.
HTTP Status Code: 429
** ModelStreamErrorException **
An error occurred while streaming the response. Retry your request.
** originalMessage **
The original message.
** originalStatusCode **
The original status code.
HTTP Status Code: 424
** ModelTimeoutException **
The request took too long to process. Processing time exceeded the model timeout length.
HTTP Status Code: 408
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceQuotaExceededException **
Your request exceeds the service quota for your account. You can view your quotas at [Viewing service quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/gs-request-quota.html). You can resubmit your request later.
HTTP Status Code: 400
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## Examples
### Run inference with streaming on a text model
For streaming, you can set `x-amzn-bedrock-accept-type` in the header to contain the desired content type of the response. In this example, we set it to accept any content type. The default value is `application/json`.
#### Sample Request
```
POST https://bedrock-runtime.us-east-1.amazonaws.com/model/amazon.titan-text-express-v1/invoke-with-response-stream
-H accept: application/vnd.amazon.eventstream
-H content-type: application/json
-H x-amzn-bedrock-accept: */*
Payload
{"inputText": "Hello world"}
```
### Example response
For streaming, the content type in the response is always set to `application/vnd.amazon.eventstream`. The response includes an additional header (x-amzn-bedrock-content-type), which contains the actual content type of the response.
#### Sample Request
```
-H content-type: application/vnd.amazon.eventstream
-H x-amzn-bedrock-content-type: application/json
Payload (stream events)
```
### Use a guardrail
This examples show how to use a guardrail with `InvokeModelWithResponseStream`.
```
POST /model/modelId/invoke-with-response-stream HTTP/1.1
X-Amzn-Bedrock-Accept: accept
Content-Type: contentType
X-Amzn-Bedrock-GuardrailIdentifier: guardrailIdentifier
X-Amzn-Bedrock-GuardrailVersion: guardrailVersion
X-Amzn-Bedrock-GuardrailTrace: guardrailTrace
X-Amzn-Bedrock-Trace: trace
body
// body
{
"amazon-bedrock-guardrailConfig": {
"tagSuffix": "string",
"streamProcessingMode": "string"
}
}
```
### Example response
This examples shows the response from a call to `InvokeModelWithResponseStream` when using a guardrail.
```
HTTP/1.1 200
X-Amzn-Bedrock-Content-Type: contentType
Content-type: application/json
// chunk 1
{
"completion": "...",
"amazon-bedrock-guardrailAction": "INTERVENED | NONE"
}
// chunk 2
{
"completion": "...",
"amazon-bedrock-guardrailAction": "INTERVENED | NONE"
}
// last chunk
{
"completion": "...",
"amazon-bedrock-guardrailAction": "INTERVENED | NONE",
"amazon-bedrock-trace": {
"guardrail": {
... // Detailed guardrail trace
}
}
}
```
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/InvokeModelWithResponseStream)
# ListAsyncInvokes
Lists asynchronous invocations.
## Request Syntax
```
GET /async-invoke?maxResults=maxResults&nextToken=nextToken&sortBy=sortBy&sortOrder=sortOrder&statusEquals=statusEquals&submitTimeAfter=submitTimeAfter&submitTimeBefore=submitTimeBefore HTTP/1.1
```
## URI Request Parameters
The request uses the following URI parameters.
** [maxResults](#API_runtime_ListAsyncInvokes_RequestSyntax) **
The maximum number of invocations to return in one page of results.
Valid Range: Minimum value of 1. Maximum value of 1000.
** [nextToken](#API_runtime_ListAsyncInvokes_RequestSyntax) **
Specify the pagination token from a previous request to retrieve the next page of results.
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `\S*`
** [sortBy](#API_runtime_ListAsyncInvokes_RequestSyntax) **
How to sort the response.
Valid Values: `SubmissionTime`
** [sortOrder](#API_runtime_ListAsyncInvokes_RequestSyntax) **
The sorting order for the response.
Valid Values: `Ascending | Descending`
** [statusEquals](#API_runtime_ListAsyncInvokes_RequestSyntax) **
Filter invocations by status.
Valid Values: `InProgress | Completed | Failed`
** [submitTimeAfter](#API_runtime_ListAsyncInvokes_RequestSyntax) **
Include invocations submitted after this time.
** [submitTimeBefore](#API_runtime_ListAsyncInvokes_RequestSyntax) **
Include invocations submitted before this time.
## Request Body
The request does not have a request body.
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"asyncInvokeSummaries": [
{
"clientRequestToken": "string",
"endTime": "string",
"failureMessage": "string",
"invocationArn": "string",
"lastModifiedTime": "string",
"modelArn": "string",
"outputDataConfig": { ... },
"status": "string",
"submitTime": "string"
}
],
"nextToken": "string"
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [asyncInvokeSummaries](#API_runtime_ListAsyncInvokes_ResponseSyntax) **
A list of invocation summaries.
Type: Array of [AsyncInvokeSummary](API_runtime_AsyncInvokeSummary.md) objects
** [nextToken](#API_runtime_ListAsyncInvokes_ResponseSyntax) **
Specify the pagination token from a previous request to retrieve the next page of results.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `\S*`
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/ListAsyncInvokes)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/ListAsyncInvokes)
# StartAsyncInvoke
Starts an asynchronous invocation.
This operation requires permission for the `bedrock:InvokeModel` action.
**Important**
To deny all inference access to resources that you specify in the modelId field, you need to deny access to the `bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` actions. Doing this also denies access to the resource through the Converse API actions ([Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) and [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)). For more information see [Deny access for inference on specific models](https://docs.aws.amazon.com/bedrock/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-deny-inference).
## Request Syntax
```
POST /async-invoke HTTP/1.1
Content-type: application/json
{
"clientRequestToken": "string",
"modelId": "string",
"modelInput": JSON value,
"outputDataConfig": { ... },
"tags": [
{
"key": "string",
"value": "string"
}
]
}
```
## URI Request Parameters
The request does not use any URI parameters.
## Request Body
The request accepts the following data in JSON format.
** [clientRequestToken](#API_runtime_StartAsyncInvoke_RequestSyntax) **
Specify idempotency token to ensure that requests are not duplicated.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: `[!-~]*`
Required: No
** [modelId](#API_runtime_StartAsyncInvoke_RequestSyntax) **
The model to invoke.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 256.
Pattern: `[a-zA-Z_\.\-/0-9:]+`
Required: Yes
** [modelInput](#API_runtime_StartAsyncInvoke_RequestSyntax) **
Input to send to the model.
Type: JSON value
Required: Yes
** [outputDataConfig](#API_runtime_StartAsyncInvoke_RequestSyntax) **
Where to store the output.
Type: [AsyncInvokeOutputDataConfig](API_runtime_AsyncInvokeOutputDataConfig.md) object
**Note: **This object is a Union. Only one member of this object can be specified or returned.
Required: Yes
** [tags](#API_runtime_StartAsyncInvoke_RequestSyntax) **
Tags to apply to the invocation.
Type: Array of [Tag](API_runtime_Tag.md) objects
Array Members: Minimum number of 0 items. Maximum number of 200 items.
Required: No
## Response Syntax
```
HTTP/1.1 200
Content-type: application/json
{
"invocationArn": "string"
}
```
## Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
** [invocationArn](#API_runtime_StartAsyncInvoke_ResponseSyntax) **
The ARN of the invocation.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 2048.
Pattern: `arn:aws(-[^:]+)?:bedrock:[a-z0-9-]{1,20}:[0-9]{12}:async-invoke/[a-z0-9]{12}`
## Errors
For information about the errors that are common to all actions, see [Common Error Types](CommonErrors.md).
** AccessDeniedException **
The request is denied because you do not have sufficient permissions to perform the requested action. For troubleshooting this error, see [AccessDeniedException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-access-denied) in the Amazon Bedrock User Guide
HTTP Status Code: 403
** ConflictException **
Error occurred because of a conflict while performing an operation.
HTTP Status Code: 400
** InternalServerException **
An internal server error occurred. For troubleshooting this error, see [InternalFailure](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-internal-failure) in the Amazon Bedrock User Guide
HTTP Status Code: 500
** ResourceNotFoundException **
The specified resource ARN was not found. For troubleshooting this error, see [ResourceNotFound](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-resource-not-found) in the Amazon Bedrock User Guide
HTTP Status Code: 404
** ServiceQuotaExceededException **
Your request exceeds the service quota for your account. You can view your quotas at [Viewing service quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/gs-request-quota.html). You can resubmit your request later.
HTTP Status Code: 400
** ServiceUnavailableException **
The service isn't currently available. For troubleshooting this error, see [ServiceUnavailable](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-service-unavailable) in the Amazon Bedrock User Guide
HTTP Status Code: 503
** ThrottlingException **
Your request was denied due to exceeding the account quotas for *Amazon Bedrock*. For troubleshooting this error, see [ThrottlingException](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-throttling-exception) in the Amazon Bedrock User Guide
HTTP Status Code: 429
** ValidationException **
The input fails to satisfy the constraints specified by *Amazon Bedrock*. For troubleshooting this error, see [ValidationError](https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html#ts-validation-error) in the Amazon Bedrock User Guide
HTTP Status Code: 400
## See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:
+ [AWS Command Line Interface V2](https://docs.aws.amazon.com/goto/cli2/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for .NET V4](https://docs.aws.amazon.com/goto/DotNetSDKV4/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for C\$1\$1](https://docs.aws.amazon.com/goto/SdkForCpp/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for Go v2](https://docs.aws.amazon.com/goto/SdkForGoV2/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for Java V2](https://docs.aws.amazon.com/goto/SdkForJavaV2/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for JavaScript V3](https://docs.aws.amazon.com/goto/SdkForJavaScriptV3/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for Kotlin](https://docs.aws.amazon.com/goto/SdkForKotlin/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for PHP V3](https://docs.aws.amazon.com/goto/SdkForPHPV3/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for Python](https://docs.aws.amazon.com/goto/boto3/bedrock-runtime-2023-09-30/StartAsyncInvoke)
+ [AWS SDK for Ruby V3](https://docs.aws.amazon.com/goto/SdkForRubyV3/bedrock-runtime-2023-09-30/StartAsyncInvoke)