Prerequisites for Amazon Bedrock Model Distillation

Choose a teacher model

Choose a teacher model that is significantly larger and more capable than the student model, and whose accuracy you want to achieve for your use case. To make the distillation job more effective, select a model that is already trained on task similar to your use case. For information on the teacher models supported by Amazon Bedrock see Supported models and Regions for Amazon Bedrock Model Distillation.

Choose a student model

Choose a student model that is significantly smaller in size. For information on the student models that Amazon Bedrock supports, see Supported models and Regions for Amazon Bedrock Model Distillation.

Prepare your input dataset

Provide the input data as prompts. Amazon Bedrock uses the input data to generate responses from the teacher model and uses the generated responses to fine-tune the student model. For more information about inputs Amazon Bedrock uses, and for choosing an option that works best for your use case, see How Amazon Bedrock Model Distillation works.

Choose the option that works best for your use case for instructions on preparing your input dataset:

Option 1: Provide your own prompts

Collect your prompts and store them in a JSON Line (JSONL) format. Each record in the JSONL must use the following structure.

{
    "schemaVersion": "bedrock-conversation-2024",
    "system": [
        {
            "text": "A chat between a curious User and an artificial intelligence Bot. The Bot gives helpful, detailed, and polite answers to the User's questions."
        }
    ],    
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "text": "why is the sky blue"
                }
            ]
        },
        {
            "role": "assistant"
            "content": [
               {
                   "text": "The sky is blue because molecules in the air scatter blue light from the Sun more than other colors."
               }
            ]
        }
    ]
}

The structure must include the required field schemaVersion that must have the value bedrock-conversion-2024. You can optionally include a system prompt that indicates the role assigned to the model. In the messages field, the user role is required, containing the input prompt provided to the model, while the assistant role, containing the desired response, is optional.

For the preview release Anthropic and Meta models support only single -turn conversation prompts, meaning you can only have one user prompt. The Amazon Nova models support multi-turn conversations, allowing you to provide multiple user and assistant exchanges within one record.

After you have added all your prompts, upload the file to an Amazon Simple Storage Service (Amazon S3) bucket.

Option 2: Use invocation logs

To use invocation logs for model distillation, set the model invocation logging on, use one of the model invocation operations, and make sure that you've set up an Amazon S3 bucket as the destination for the logs. Before you can start the model distillation job, you must provide Amazon Bedrock permissions to access the logs. For more information about setting up the invocation logs, see Monitor model invocation using Amazon CloudWatch Logs.

With this option, you can specify if you want Amazon Bedrock to use only the prompts, or to use prompt-response pairs from the invocation log. If you want Amazon Bedrock to use only prompts, then Amazon Bedrock might add proprietary data synthesis techniques to generate diverse and higher-quality responses from the teacher model. If you want Amazon Bedrock to use prompt-response pairs, then Amazon Bedrock won't re-generate responses from the teacher model. Amazon Bedrock will directly use the responses from the invocation log to fine-tune the student model.

You can optionally add request metadata to the prompt-response pairs in the invocation log using one of the model invocation operations and then later use it to filter the logs. Amazon Bedrock can use the filtered logs to fine-tune the student model.

To filter the logs using multiple request metadata, use a single operation Boolean operator AND, OR, or NOT. You cannot combine operations. For single request metadata filtering, use the Boolean operator NOT.