Preparing data for RFT on Amazon Nova 2

RFT on Amazon Nova 2 currently supports text-based training data. This page describes the data format, supported features, constraints, and best practices for preparing RFT training data for Amazon Nova 2 Understanding models.

Data format

RFT training data follows the OpenAI Reinforcement Fine-Tuning format. Each line in your JSONL training file is a JSON object with the following top-level fields. Expand a section to learn more:

"messages": [
    {
        "role": "system",
        "content": "instructions"
    },
    {
        "role": "user",
        "content": "prompt"
    }
]

"reference_answer": {
    "field": "value"
}

"tools": [
    {
        "type": "function",
        "function": {
            "name": "tool-name",
            "description": "tool-description",
            "parameters": { ... }
        }
    }
]

Validating your data

Before submitting your training job, validate your dataset to catch formatting issues early. For available validation tools, see Validation tools.

Sample inputs

The following are complete example JSON objects showing how to combine the fields for different RFT use cases.

Supported features

The following table summarizes feature support for RFT on Amazon Nova 2.

General/Text understanding

This section summarizes the general constraints and best practices for preparing RFT on Amazon Nova 2 training data.

Constraints

Best practices

Sample input

{
    "id": "math-001",
    "messages": [
        {
            "role": "system",
            "content": "You are a math tutor"
        },
        {
            "role": "user",
            "content": "Solve: 2x + 5 = 13"
        }
    ],
    "reference_answer": {
        "solution": "x = 4",
        "steps": ["2x = 13 - 5", "2x = 8", "x = 4"]
    }
}

Tool calling

RFT supports training models on tool calling patterns, enabling your model to learn when and how to invoke external tools or functions.

Constraints

Best practices

Sample input

{
    "id": "tool-001",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful game master assistant"
        },
        {
            "role": "user",
            "content": "Generate a strength stat for a warrior character. Apply a +2 racial bonus modifier."
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "StatRollAPI",
                "description": "Generates character stats by rolling 4d6, dropping the lowest die result, and applying a modifier.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "modifier": {
                            "description": "An integer representing the modifier to apply to the total of the stat roll.",
                            "type": "integer"
                        }
                    },
                    "required": ["modifier"]
                }
            }
        }
    ],
    "reference_answer": {
        "tool_called": "StatRollAPI",
        "tool_parameters": {
            "modifier": 2
        },
        "expected_behavior": "Call StatRollAPI with modifier=2 and return the calculated stat value"
    }
}

Reasoning

RFT on Amazon Nova 2 supports reasoning mode, where the model generates explicit thinking tokens before producing a final answer. You control reasoning behavior during training with the reasoning_effort training configuration field.

Constraints

When to use each mode

Use high reasoning for:

Use none (omit reasoning_effort) or low reasoning for:

Cost and performance trade-offs

Higher reasoning modes increase:

Characteristics of effective training data

Clarity and consistency

Good RFT examples require clear, unambiguous input data that enables accurate reward calculation across different model outputs. Avoid noise in your data, including:

Any ambiguity will mislead the training process and cause the model to learn unintended behaviors.

Diversity

Your dataset should capture the full diversity of production use cases to ensure robust real-world performance. Include:

Reward function considerations

Design your reward function for efficient training:

Fast, scalable reward functions enable rapid iteration and cost-effective experimentation.

RFT training using LLM as a judge

Overview

Large language models (LLMs) are increasingly being used as judges in reinforcement fine-tuning (RFT) workflows, providing automated reward signals that guide model optimization. In this approach, an LLM evaluates model outputs against specified criteria—whether assessing correctness, quality, style adherence, or semantic equivalence—and assigns rewards that drive the reinforcement learning process.

This is particularly valuable for tasks where traditional reward functions are difficult to define programmatically, such as determining whether different representations (like "1/3", "0.333", and "one-third") are semantically equivalent, or evaluating nuanced qualities like coherence and relevance. By using LLM-based judges as reward functions, you can scale RFT to complex domains without requiring extensive human annotation, enabling rapid iteration and continuous improvement of your models across diverse use cases beyond traditional alignment problems.

Validating your LLM judge

Before deploying an LLM-as-a-judge in production, validate that the judge model's evaluations align with human judgment. This involves:

This validation step helps ensure the automated evaluation process will produce models that meet your production quality criteria.

Lambda configuration for LLM judge

Using an LLM as a judge is an extension of using Lambda functions for Reinforcement Learning with Verifiable Rewards (RLVR). Inside the Lambda function, you make a call to one of the models hosted in Amazon Bedrock.

Important configuration requirements: