# Reasoning model evaluation
<a name="nova-reasoning-model-evaluation"></a>

## Overview
<a name="nova-reasoning-overview"></a>

Reasoning model support enables evaluation with reasoning-capable Nova models that perform explicit internal reasoning before generating final responses. This feature uses API-level control via the `reasoning_effort` parameter to dynamically enable or disable reasoning functionality, potentially improving response quality for complex analytical tasks.

**Supported models**
+ amazon.nova-2-lite-v1:0:256k

## Recipe configuration
<a name="nova-reasoning-recipe"></a>

Enable reasoning by adding the `reasoning_effort` parameter to the `inference` section of your recipe:

```
run:  
  name: reasoning-eval-job-name                          # [MODIFIABLE] Unique identifier for your evaluation job  
  model_type: amazon.nova-2-lite-v1:0:256k               # [FIXED] Must be a reasoning-supported model  
  model_name_or_path: nova-lite-2/prod                   # [FIXED] Path to model checkpoint or identifier  
  replicas: 1                                            # [MODIFIABLE] Number of replicas for SageMaker Training job  
  data_s3_path: ""                                       # [MODIFIABLE] Leave empty for SageMaker Training job; optional for SageMaker HyperPod job  
  output_s3_path: ""                                     # [MODIFIABLE] Output path for SageMaker HyperPod job (not compatible with SageMaker Training jobs)  
  
evaluation:  
  task: mmlu                                             # [MODIFIABLE] Evaluation task  
  strategy: zs_cot                                       # [MODIFIABLE] Evaluation strategy  
  metric: accuracy                                       # [MODIFIABLE] Metric calculation method  
  
inference:  
  reasoning_effort: high                                 # [MODIFIABLE] Enables reasoning mode; options: low/high or null to disable  
  max_new_tokens: 32768                                  # [MODIFIABLE] Maximum tokens to generate, recommended value when reasoning_effort set to high  
  top_k: -1                                              # [MODIFIABLE] Top-k sampling parameter  
  top_p: 1.0                                             # [MODIFIABLE] Nucleus sampling parameter  
  temperature: 0                                         # [MODIFIABLE] Sampling temperature (0 = deterministic)
```

## Using the reasoning\$1effort parameter
<a name="nova-reasoning-parameter"></a>

The `reasoning_effort` parameter controls the reasoning behavior for reasoning-capable models.

### Prerequisites
<a name="nova-reasoning-prerequisites"></a>
+ **Model compatibility** – Set `reasoning_effort` only when `model_type` specifies a reasoning-capable model (currently `amazon.nova-2-lite-v1:0:256k`)
+ **Error handling** – Using `reasoning_effort` with unsupported models will fail with `ConfigValidationError: "Reasoning mode is enabled but model '{model_type}' does not support reasoning. Please use a reasoning-capable model or disable reasoning mode."`

### Available options
<a name="nova-reasoning-options"></a>


| Option | Behavior | Token limit | Use case | 
| --- | --- | --- | --- | 
| null (default) | Disables reasoning mode | N/A | Standard evaluation without reasoning overhead | 
| low | Enables reasoning with constraints | 4,000 tokens for internal reasoning | Scenarios requiring concise reasoning; optimizes for speed and cost | 
| high | Enables reasoning without constraints | No token limit on internal reasoning | Complex problems requiring extensive analysis and step-by-step reasoning | 


| Training method | Available options | How to configure | 
| --- | --- | --- | 
| SFT (Supervised Fine-Tuning) | High or Off only | Use reasoning\$1enabled: true (high) or reasoning\$1enabled: false (off) | 
| RFT (Reinforcement Fine-Tuning) | Low, High, or Off | Use reasoning\$1effort: low or reasoning\$1effort: high. Omit field to disable. | 
| Evaluation | Low, High, or Off | Use reasoning\$1effort: low or reasoning\$1effort: high. Use null to disable. | 

### When to enable reasoning
<a name="nova-reasoning-when-to-enable"></a>

**Use reasoning mode (`low` or `high`) for**
+ Complex problem-solving tasks (mathematics, logic puzzles, coding)
+ Multi-step analytical questions requiring intermediate reasoning
+ Tasks where detailed explanations or step-by-step thinking improve accuracy
+ Scenarios where response quality is prioritized over speed

**Use non-reasoning mode (`null` or omit parameter) for**
+ Simple Q&A or factual queries
+ Creative writing tasks
+ When faster response times are critical
+ Performance benchmarking where reasoning overhead should be excluded
+ Cost optimization when reasoning doesn't improve task performance

### Troubleshooting
<a name="nova-reasoning-troubleshooting"></a>

**Error: "Reasoning mode is enabled but model does not support reasoning"**

**Cause**: The `reasoning_effort` parameter is set to a non-null value, but the specified `model_type` doesn't support reasoning.

**Resolution**:
+ Verify your model type is `amazon.nova-2-lite-v1:0:256k`
+ If using a different model, either switch to a reasoning-capable model or remove the `reasoning_effort` parameter from your recipe