Submit a model distillation job in Amazon Bedrock

Sign in to the AWS Management Console using an IAM role with Amazon Bedrock permissions, and open the Amazon Bedrock console at https://console.aws.amazon.com/bedrock/.

From the left navigation pane, choose Custom models under Foundation models.

Choose Create distillation job.

For Distilled model details, do the following:

1. For Distilled model name, enter a name for your distilled model.

2. (Optional) For Model encryption, select the checkbox if you want to provide a KMS key for encrypting your job and its related artifacts.

   For more information, see Encryption of model customization jobs and artifacts.

3. (Optional) Apply Tags to your distilled model.

For Job configuration, do the following:

1. For Job name, enter a name for your distillation job.

2. (Optional) For Model encryption, select the checkbox if you want to provide a KMS key for encrypting your job and its related artifacts.

   For more information, see Encryption of model customization jobs and artifacts.

3. (Optional) Apply Tags to your job.

For Teacher model - Student model details, choose the teacher and student models for creating your distilled model.

For more information, see Choose teacher and student models for distillation.

For Synthetic data generation, do the following:

1. For Max response length, specify the maximum length of the synthetic responses generated by the teacher model.

2. For Distillation input dataset, choose one of the following options:

   • Directly upload to S3 location – Specify the S3 location where you're storing the input dataset (prompts) that'll be used for distillation. For more information, see Option 1: Provide your own prompts for data preparation.

   • Provide access to invocation logs – Specify the S3 location where you're storing the invocation logs with the input dataset (prompts) that'll be used for distillation. For more information, see Option 2: Use invocation logs for data preparation.

     • (Optional) For Request Metadata Filters, specify filters if you want Amazon Bedrock to only use certain prompts in your logs for distillation.

     • Choose Read prompts or Read prompt-response pairs depending on what you want Amazon Bedrock to access from your logs. Keep in mind that responses are read only if your teacher model matches the model in your logs.

For Distillation output, specify the S3 location where you want to upload the metrics and reports about your distillation job.

For more information, see Analyze the results of a model customization job.

For VPC settings, choose a VPC configuration for accessing the S3 bucket with your training data.

For more information, see (Optional) Protect your model customization jobs using a VPC.

For Service access, specify the IAM role for accessing the S3 bucket with your training data. Unless you use a Cross Region inference profile or VPC configurations, you can create the role in the Amazon Bedrock console with the correct permissions automatically configured. Or you can use an existing service role.

For a job that has Amazon VPC configurations or uses a Cross Region inference profile, you must create a new service role in IAM that has the required permissions.

For more information, see Create an IAM service role for model customization.

Choose Create distillation job to start the distillation job. After you customize a model, you can share it or copy it to a different region. To run inference using a custom model (including copied models), you must purchase Provisioned Throughput for it. See Increase model invocation capacity with Provisioned Throughput in Amazon Bedrock.