Evaluator prompts used in a RAG evaluation job

Focus mode

Evaluator prompts used in a RAG evaluation job - Amazon Bedrock

The same prompts are used for retrieve-only and retrieve-and-generate evaluation jobs. All prompts contain an optional chat_history component. If conversationTurns is specified, then chat_history is included in the prompt.

Double curly braces {{}} are used to indicate where data from your prompt dataset is inserted.

{{chat_history}} – This represents the history of the conversation denoted in conversationTurns. For each turn, the next prompt is amended to the chat_history.
{{prompt}} – The prompt from your prompt dataset
{{ground_truth}} – The ground truth from your prompt dataset
{{prediction}} – The final output from the LLM in your RAG system