Prerequisites for Amazon Bedrock Model Distillation

Choose a teacher model

Choose a teacher model that is significantly larger and more capable than the student model, and whose accuracy you want to achieve for your use case. To make the distillation job more effective, select a model that is already trained on task similar to your use case. For information on the teacher models supported by Amazon Bedrock see Supported models and Regions for Amazon Bedrock Model Distillation.

Choose a student model

Choose a student model that is significantly smaller in size. For information on the student models that Amazon Bedrock supports, see Supported models and Regions for Amazon Bedrock Model Distillation.

Prepare your input dataset

Provide the input data as prompts. Amazon Bedrock uses the input data to generate responses from the teacher model and uses the generated responses to fine-tune the student model. For more information about inputs Amazon Bedrock uses, and for choosing an option that works best for your use case, see How Amazon Bedrock Model Distillation works.

Choose the option that works best for your use case for instructions on preparing your input dataset:

Option 1: Provide your own prompts

Collect your prompts and store them in a jsonl file, each line of which is a JSON object corresponding to a record. Then, upload the file to an Amazon Simple Storage Service (Amazon S3) bucket. The following is an example of a jsonl file that has prompts.

{"prompt": "<prompt1>"}
{"prompt": "<prompt2>"}
{"prompt": "<prompt3>"}      

You can optionally provide labeled input data as prompt-response pairs. If provided, the prompt-response pairs might be used as golden examples to instruct the teacher to generate similar high-quality responses. The following is an example of a jsonl file that has prompt-response pairs.

{"role1": "<prompt-tester1>"}
{"role2": "<prompt-tester2>"}
{"role3": "<prompt-tester3>"}      

Option 2: Use invocation logs

To use invocation logs for model distillation, set the model invocation logging on, use one of the model invocation operations, and make sure that you've set up an Amazon S3 bucket as the destination for the logs. Before you can start the model distillation job, you must provide Amazon Bedrock permissions to access the logs. For more information about setting up the invocation logs, see Monitor model invocation using Amazon CloudWatch Logs.

With this option, you can specify if you want Amazon Bedrock to use only the prompts, or to use prompt-response pairs from the invocation log. If you want Amazon Bedrock to use only prompts, then Amazon Bedrock might add proprietary data synthesis techniques to generate diverse and higher-quality responses from the teacher model. If you want Amazon Bedrock to use prompt-response pairs, then Amazon Bedrock won't re-generate responses from the teacher model. Amazon Bedrock will directly use the responses from the invocation log to fine-tune the student model.

You can optionally add request metadata to the prompt-response pairs in the invocation log using one of the model invocation operations and then later use it to filter the logs. Amazon Bedrock can use the filtered logs to fine-tune the student model.

To filter the logs using multiple request metadata, use a single operation Boolean operator AND, OR, or NOT. You cannot combine operations. For single request metadata filtering, use the Boolean operator NOT.