本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。
retrieve-and-generate评估任务需要使用 JSON 行格式的提示数据集。您的数据集中最多可以有 1000 个提示
为 Amazon Bedrock 调用您的知识库的 retrieve-and-generate评估任务准备数据集
要创建 Amazon Bedrock 调用您的知识库的仅限检索的评估任务，您的提示数据集必须包含以下键值对：
-
referenceResponses— 此父键用于指定您期RetrieveAndGenerate望返回的基本真相响应。在text密钥中指定基本真相。referenceResponses如果您在评估作业中选择上下文覆盖率指标，则为必填项。 -
prompt— 此父键用于指定在评估作业运行时您希望模型响应的提示（用户查询）。
下面是一个包含 6 个输入并使用了 JSON 行格式的自定义数据集示例。
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you want to use during inference"}]},"referenceResponses":[{"content":[{"text":"Specify a ground-truth response"}]}]}]}为清楚起见，扩展了以下提示。在实际的提示数据集中，每行（提示）都必须是有效的 JSON 对象。
{
"conversationTurns": [
{
"prompt": {
"content": [
{
"text": "What is the recommended service interval for your product?"
}
]
},
"referenceResponses": [
{
"content": [
{
"text": "The recommended service interval for our product is two years."
}
]
}
]
}
]
}
使用您自己的推理响应数据为 retrieve-and-generate评估任务准备数据集
要创建在其中提供自己的推理响应数据的 retrieve-and-generate评估作业，您的提示数据集是对话回合的列表，其中包含每回合的以下内容。每个作业只能评估一个 RAG 来源。
-
prompt— 您向模型提供的生成结果的提示。 -
referenceResponses— 此父密钥用于指定 LLM 在提取检索结果和输入查询后对最终输出所期望的基本真相响应。 -
referenceContexts（可选）— 此可选的父密钥用于指定您希望从 RAG 源中检索到的基本真相段落。仅当您想在自己的自定义评估指标中使用此密钥时，才需要包含该密钥。Amazon Bedrock 提供的内置指标不使用此属性。 -
output— 来自你的 RAG 源的输出，包括以下内容：-
text— RAG 系统中 LLM 的最终输出。 -
retrievedPassages— 此父密钥用于指定您的 RAG 来源检索到的内容。
-
您的output数据还必须包含用于定义用于生成推理响应的 RAG 源的字符串knowledgeBaseIdentifier。您还可以包含一个可选modelIdentifier字符串来标识您使用的 LLM。对于retrievalResults和retrievedReferences，您可以提供可选的名称和元数据。
下面是一个包含 6 个输入并使用了 JSON 行格式的自定义数据集示例。
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
{"conversationTurns":[{"prompt":{"content":[{"text":"Provide the prompt you used to generate the response"}]},"referenceResponses":[{"content":[{"text":"A ground truth for the final response generated by the LLM"}]}],"referenceContexts":[{"content":[{"text":"A ground truth for a received passage"}]}],"output":{"text":"The output of the LLM","modelIdentifier":"(Optional) a string identifying your model","knowledgeBaseIdentifier":"A string identifying your RAG source","retrievedPassages":{"retrievalResults":[{"name":"(Optional) a name for your retrieval","content":{"text":"The retrieved content"},"metadata":{"(Optional) a key for your metadata":"(Optional) a value for your metadata"}}]}}}]}
为了清晰起见，以下显示了扩展的提示数据集格式。在实际的提示数据集中，每行（提示）都必须是有效的 JSON 对象。
{
"conversationTurns": [
{
"prompt": {
"content": [
{
"text": "Provide the prompt you used to generate the responses"
}
]
},
"referenceResponses": [
{
"content": [
{
"text": "A ground truth for the final response generated by the LLM"
}
]
}
],
"referenceContexts": [
{
"content": [
{
"text": "A ground truth for a received passage"
}
]
}
],
"output": {
"text": "The output of the LLM",
"modelIdentifier": "(Optional) a string identifying your model",
"knowledgeBaseIdentifier": "A string identifying your RAG source",
"retrievedPassages": {
"retrievalResults": [
{
"name": "(Optional) a name for your retrieval",
"content": {
"text": "The retrieved content"
},
"metadata": {
"(Optional) a key for your metadata": "(Optional) a value for your metadata"
}
}
]
}
}
}
]
}
