Prepare the datasets

For text-to-text models, prepare a training and optional validation dataset. Each JSON object is a sample containing both a prompt and completion field. Use 6 characters per token as an approximation for the number of tokens. The format is as follows.

{"prompt": "<prompt1>", "completion": "<expected generated text>"}
{"prompt": "<prompt2>", "completion": "<expected generated text>"}
{"prompt": "<prompt3>", "completion": "<expected generated text>"}      

The following is an example item for a question-answer task:

{"prompt": "what is AWS", "completion": "it's Amazon Web Services"}

For text-to-image or image-to-embedding models, prepare a training dataset. Validation datasets are not supported. Each JSON object is a sample containing an image-ref, the Amazon S3 URI for an image, and a caption that could be a prompt for the image.

The images must be in JPEG or PNG format.

{"image-ref": "s3://bucket/path/to/image001.png", "caption": "<prompt text>"}
{"image-ref": "s3://bucket/path/to/image002.png", "caption": "<prompt text>"}
{"image-ref": "s3://bucket/path/to/image003.png", "caption": "<prompt text>"}    

The following is an example item:

{"image-ref": "s3://amzn-s3-demo-bucket/my-pets/cat.png", "caption": "an orange cat with white spots"}

To allow Amazon Bedrock access to the image files, add an IAM policy similar to the one in Permissions to access training and validation files and to write output files in S3 to the Amazon Bedrock model customization service role that you set up or that was automatically set up for you in the console. The Amazon S3 paths you provide in the training dataset must be in folders that you specify in the policy.

To carry out Continued Pre-training on a text-to-text model, prepare a training and optional validation dataset. Because Continued Pre-training involves unlabeled data, each JSON line is a sample containing only an input field. Use 6 characters per token as an approximation for the number of tokens. The format is as follows.

{"input": "<input text>"}
{"input": "<input text>"}
{"input": "<input text>"}

The following is an example item that could be in the training data.

{"input": "AWS stands for Amazon Web Services"}

To fine-tune a text-to-text model using the single-turn messaging format, prepare a training and optional validation dataset. Both data files must be in the JSONL format. Each line specifies a complete data sample in json format; and each data sample must be formatted to 1 line (remove all the ‘\n’ within each sample). One line with multiple data samples or splitting a data sample over multiple lines won’t work.

Fields

Rules

{"system": "<system message>","messages":[{"role": "user", "content": "<user query>"},{"role": "assistant", "content": "<expected generated text>"}]}

Example

{"system": "You are an helpful assistant.","messages":[{"role": "user", "content": "what is AWS"},{"role": "assistant", "content": "it's Amazon Web Services."}]}

To fine-tune a text-to-text model using the multi-turn messaging format, prepare a training and optional validation dataset. Both data files must be in the JSONL format. Each line specifies a complete data sample in json format; and each data sample must be formatted to 1 line (remove all the ‘\n’ within each sample). One line with multiple data samples or splitting a data sample over multiple lines won’t work.

Fields

Rules

{"system": "<system message>","messages":[{"role": "user", "content": "<user query 1>"},{"role": "assistant", "content": "<expected generated text 1>"}, {"role": "user", "content": "<user query 2>"},{"role": "assistant", "content": "<expected generated text 2>"}]}

Example

{"system": "system message","messages":[{"role": "user", "content": "Hello there."},{"role": "assistant", "content": "Hi, how can I help you?"},{"role": "user", "content": "what are LLMs?"},{"role": "assistant", "content": "LLM means large language model."},]}  

To prepare training and validation datasets for a model distillation job, see Prerequisites for Amazon Bedrock Model Distillation.

Image and video constraints