本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。

数据集架构

一个数据集包含一个或多个场景。每个场景都代表与代理的对话（会话）。按需和批处理数据集运行器都使用相同的数据集格式。

S AgentCore DK 支持两种场景类型：

FileDatasetProvider从 JSON 结构中自动检测场景类型：带turns字段的场景按预定义加载；带actor_profile字段（无字段turns）的场景以模拟方式加载。

预定义的场景

预定义的场景指定了具有已知输入和可选预期输出的固定转弯顺序。

Single-turn 示例

每个场景都会发送一个提示并检查响应：

{
  "scenarios": [
    {
      "scenario_id": "math-question",
      "turns": [
        {
          "input": "What is 15 + 27?",
          "expected_response": "15 + 27 = 42"
        }
      ],
      "expected_trajectory": ["calculator"],
      "assertions": ["Agent used the calculator tool to compute the result"]
    },
    {
      "scenario_id": "weather-check",
      "turns": [
        {
          "input": "What's the weather?",
          "expected_response": "The weather is sunny"
        }
      ],
      "expected_trajectory": ["weather"],
      "assertions": ["Agent used the weather tool"]
    }
  ]
}

Multi-turn 示例

Multi-turn 每个场景都有多个回合。回合在同一个会话中按顺序执行，保持对话上下文。每个回合都有自己的回合 expected_responseassertions，同时expected_trajectory适用于整个会话：

{
  "scenarios": [
    {
      "scenario_id": "math-then-weather",
      "turns": [
        {
          "input": "What is 15 + 27?",
          "expected_response": "15 + 27 = 42"
        },
        {
          "input": "What's the weather?",
          "expected_response": "The weather is sunny"
        }
      ],
      "expected_trajectory": ["calculator", "weather"],
      "assertions": [
        "Agent used the calculator tool for the math question",
        "Agent used the weather tool when asked about weather"
      ]
    }
  ]
}

场景字段

转弯字段

模拟场景

模拟场景定义了演员概况和初始输入。演员动态生成后续回合：

{
  "scenarios": [
    {
      "scenario_id": "geography-student",
      "scenario_description": "A curious student asks geography questions",
      "actor_profile": {
        "traits": {"expertise": "novice", "tone": "curious"},
        "context": "A student studying world geography who wants to learn about capitals",
        "goal": "Find out the capital cities of at least two different countries"
      },
      "input": "Hi! I'm studying geography. Can you help me learn about world capitals?",
      "max_turns": 5,
      "assertions": [
        "Agent provides accurate capital city information",
        "Agent is helpful and encouraging to the student"
      ]
    }
  ]
}

场景字段

实况测绘

两位运行者都会自动将数据集字段映射到使用它们的评估器：

有关实地真值字段及其如何与评估 API 配合使用的更多详细信息，请参阅实况评估。

内联数据集构建

您可以直接在 Python 中构造数据集，而不是从 JSON 文件中加载：

from bedrock_agentcore.evaluation import Dataset, PredefinedScenario, Turn

dataset = Dataset(
    scenarios=[
        PredefinedScenario(
            scenario_id="math-question",
            turns=[
                Turn(
                    input="What is 15 + 27?",
                    expected_response="15 + 27 = 42",
                ),
            ],
            expected_trajectory=["calculator"],
            assertions=["Agent used the calculator tool"],
        ),
        PredefinedScenario(
            scenario_id="weather-check",
            turns=[
                Turn(input="What's the weather?"),
            ],
            expected_trajectory=["weather"],
        ),
    ]
)

或者从 JSON 文件中加载：

from bedrock_agentcore.evaluation import FileDatasetProvider

dataset = FileDatasetProvider("dataset.json").get_dataset()