Choose the best performing knowledge base using Amazon Bedrock evaluations
You can use computed metrics to evaluate how effective a knowledge base retrieves relevant information from your data sources, and how effective the generated responses are in answering questions. The results of a knowledge base evaluation allow you to compare different knowledge bases, and then choose the best knowledge base suited for your AI application.
You can set up two different types of knowledge base evaluation jobs.
-
Retrieval only – In a Retrieval only model evaluation job the evaluator model is used to perform inference against your knowledge base. The report is based on the data retrieved by from your knowledge base, and the metrics you select.
-
Retrieval and response generation – In a Retrieval and response generation model evaluation job the evaluator model is used to perform inference against your knowledge based. The report is based on the data retrieved from your knowledge base and the summaries generated by the evaluator model.
Use the following topics to see how to create and manage knowledge base evaluation jobs, and the kinds of performance metrics you can use.
Topics
- Prerequisites for creating knowledge base evaluations in Amazon Bedrock
- Use a prompt dataset for a knowledge base evaluation in Amazon Bedrock
- Evaluator prompts used in a knowledge base evaluation job
- Creating a knowledge base evaluation job in Amazon Bedrock
- List evaluation jobs that use a Amazon Bedrock Knowledge Bases in Amazon Bedrock
- Stop a knowledge base evaluation job in Amazon Bedrock
- Knowledge base evaluation of retrieval or response generation
- Review knowledge base evaluation job reports and metrics
- Delete a knowledge base evaluation job in Amazon Bedrock
