Amazon Titan Text 模型 - Amazon Bedrock

本文属于机器翻译版本。若本译文内容与英语原文存在差异,则一律以英文原文为准。

Amazon Titan Text 模型

Amazon Titan Text 模型支持以下推理参数。

有关 Titan Text 提示工程准则的更多信息,请参阅 Titan Text Prompt Engineering Guidelines

有关 Titan 模型的更多信息,请参阅 Amazon Titan 模型概述

请求和响应

请求正文在 InvokeModelInvokeModelWithResponseStream 请求的 body 字段中传递。

Request
{ "inputText": string, "textGenerationConfig": { "temperature": float, "topP": float, "maxTokenCount": int, "stopSequences": [string] } }

以下参数为必需参数:

  • inputText – 为生成响应提供模型的提示。要以对话方式生成回复,请使用以下格式封装提示:

    "inputText": "User: <prompt>\nBot:

textGenerationConfig 是可选项。您可以使用它来配置以下推理参数

  • temperature – 使用较低的值可降低响应中的随机性。

    默认 最小值 最大值
    0.7 0.0 1.0
  • topP – 使用较低的值可忽略可能性较低的选项并减少响应的多样性。

    默认 最小值 最大值
    0.9 0.0 1.0
  • maxTokenCount – 指定要在生成的响应中使用的最大词元数。最大词元限制会被严格执行。

    模型 默认 最小值 最大值
    Titan Text Lite 512 0 4,096
    Titan Text Express 512 0 8192
    Titan Text 高级版 512 0 3,072
  • stopSequences – 指定字符序列以指示模型应在何处停止。

InvokeModel Response
{ "inputTextTokenCount": int, "results": [{ "tokenCount": int, "outputText": "\n<response>\n", "completionReason": "string" }] }

响应正文包含以下字段:

  • inputTextTokenCount – 提示中的词元数量。

  • results – 由一个项目组成的数组,一个包含以下字段的对象:

    • tokenCount – 响应中的词元数量。

    • outputText – 响应中的文本。

    • completionReason – 响应结束生成操作的原因。可能有以下原因;

      • FINISHED – 响应已完全生成。

      • LENGTH – 由于您设置的响应长度,响应被截断。

      • STOP_CRITERIA_MET – 由于达到停止标准,响应被截断。

      • RAG_QUERY_WHEN_RAG_DISABLED – 该功能已禁用,无法完成查询。

      • CONTENT_FILTERED – 所应用的内容筛选条件筛选或删除了内容。

InvokeModelWithResponseStream Response

响应流正文中的每个文本块的格式如下。您必须对 bytes 字段进行解码(有关示例,请参阅 使用以下命令提交单个提示 InvokeModel)。

{ "chunk": { "bytes": b'{ "index": int, "inputTextTokenCount": int, "totalOutputTextTokenCount": int, "outputText": "<response-chunk>", "completionReason": "string" }' } }
  • index – 流式传输响应中数据块的索引。

  • inputTextTokenCount – 提示中的词元数量。

  • totalOutputTextTokenCount – 响应中的词元数量。

  • outputText – 响应中的文本。

  • completionReason – 响应结束生成操作的原因。可能有以下原因。

    • FINISHED – 响应已完全生成。

    • LENGTH – 由于您设置的响应长度,响应被截断。

    • STOP_CRITERIA_MET – 由于达到停止标准,响应被截断。

    • RAG_QUERY_WHEN_RAG_DISABLED – 该功能已禁用,无法完成查询。

    • CONTENT_FILTERED – 所应用的筛选条件筛选或删除了内容。

代码示例

以下示例展示了如何使用 Python SDK 对 Amazon Titan Text Premier 模型运行推理。

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to create a list of action items from a meeting transcript with the Amazon Titan Text model (on demand). """ import json import logging import boto3 from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon Titan Text models" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_text(model_id, body): """ Generate text using Amazon Titan Text models on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: response (json): The response from the model. """ logger.info( "Generating text with Amazon Titan Text model %s", model_id) bedrock = boto3.client(service_name='bedrock-runtime') accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Text generation error. Error is {finish_reason}") logger.info( "Successfully generated text with Amazon Titan Text model %s", model_id) return response_body def main(): """ Entrypoint for Amazon Titan Text model example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") # You can replace the model_id with any other Titan Text Models # Titan Text Model family model_id is as mentioned below: # amazon.titan-text-premier-v1:0, amazon.titan-text-express-v1, amazon.titan-text-lite-v1 model_id = 'amazon.titan-text-premier-v1:0' prompt = """Meeting transcript: Miguel: Hi Brant, I want to discuss the workstream for our new product launch Brant: Sure Miguel, is there anything in particular you want to discuss? Miguel: Yes, I want to talk about how users enter into the product. Brant: Ok, in that case let me add in Namita. Namita: Hey everyone Brant: Hi Namita, Miguel wants to discuss how users enter into the product. Miguel: its too complicated and we should remove friction. for example, why do I need to fill out additional forms? I also find it difficult to find where to access the product when I first land on the landing page. Brant: I would also add that I think there are too many steps. Namita: Ok, I can work on the landing page to make the product more discoverable but brant can you work on the additonal forms? Brant: Yes but I would need to work with James from another team as he needs to unblock the sign up workflow. Miguel can you document any other concerns so that I can discuss with James only once? Miguel: Sure. From the meeting transcript above, Create a list of action items for each person. """ body = json.dumps({ "inputText": prompt, "textGenerationConfig": { "maxTokenCount": 3072, "stopSequences": [], "temperature": 0.7, "topP": 0.9 } }) response_body = generate_text(model_id, body) print(f"Input token count: {response_body['inputTextTokenCount']}") for result in response_body['results']: print(f"Token count: {result['tokenCount']}") print(f"Output text: {result['outputText']}") print(f"Completion reason: {result['completionReason']}") except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating text with the Amazon Titan Text Premier model {model_id}.") if __name__ == "__main__": main()

以下示例展示了如何使用 Python SDK 对 Amazon Titan Text G1 - Express 模型运行推理。

# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. # SPDX-License-Identifier: Apache-2.0 """ Shows how to create a list of action items from a meeting transcript with the Amazon &titan-text-express; model (on demand). """ import json import logging import boto3 from botocore.exceptions import ClientError class ImageError(Exception): "Custom exception for errors returned by Amazon &titan-text-express; model" def __init__(self, message): self.message = message logger = logging.getLogger(__name__) logging.basicConfig(level=logging.INFO) def generate_text(model_id, body): """ Generate text using Amazon &titan-text-express; model on demand. Args: model_id (str): The model ID to use. body (str) : The request body to use. Returns: response (json): The response from the model. """ logger.info( "Generating text with Amazon &titan-text-express; model %s", model_id) bedrock = boto3.client(service_name='bedrock-runtime') accept = "application/json" content_type = "application/json" response = bedrock.invoke_model( body=body, modelId=model_id, accept=accept, contentType=content_type ) response_body = json.loads(response.get("body").read()) finish_reason = response_body.get("error") if finish_reason is not None: raise ImageError(f"Text generation error. Error is {finish_reason}") logger.info( "Successfully generated text with Amazon &titan-text-express; model %s", model_id) return response_body def main(): """ Entrypoint for Amazon &titan-text-express; example. """ try: logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") model_id = 'amazon.titan-text-express-v1' prompt = """Meeting transcript: Miguel: Hi Brant, I want to discuss the workstream for our new product launch Brant: Sure Miguel, is there anything in particular you want to discuss? Miguel: Yes, I want to talk about how users enter into the product. Brant: Ok, in that case let me add in Namita. Namita: Hey everyone Brant: Hi Namita, Miguel wants to discuss how users enter into the product. Miguel: its too complicated and we should remove friction. for example, why do I need to fill out additional forms? I also find it difficult to find where to access the product when I first land on the landing page. Brant: I would also add that I think there are too many steps. Namita: Ok, I can work on the landing page to make the product more discoverable but brant can you work on the additonal forms? Brant: Yes but I would need to work with James from another team as he needs to unblock the sign up workflow. Miguel can you document any other concerns so that I can discuss with James only once? Miguel: Sure. From the meeting transcript above, Create a list of action items for each person. """ body = json.dumps({ "inputText": prompt, "textGenerationConfig": { "maxTokenCount": 4096, "stopSequences": [], "temperature": 0, "topP": 1 } }) response_body = generate_text(model_id, body) print(f"Input token count: {response_body['inputTextTokenCount']}") for result in response_body['results']: print(f"Token count: {result['tokenCount']}") print(f"Output text: {result['outputText']}") print(f"Completion reason: {result['completionReason']}") except ClientError as err: message = err.response["Error"]["Message"] logger.error("A client error occurred: %s", message) print("A client error occured: " + format(message)) except ImageError as err: logger.error(err.message) print(err.message) else: print( f"Finished generating text with the Amazon &titan-text-express; model {model_id}.") if __name__ == "__main__": main()