Nemotron-Cascade
Free while signed in. Answers cite the passages they came from.
Nemotron-Cascade introduces cascaded domain-wise reinforcement learning (Cascade RL) to build general-purpose reasoning models capable of operating in both instruct and deep thinking modes. Rather than blending heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL stages that reduce engineering complexity while delivering state-of-the-art performance. - **Sequential domain-wise RL:** The approach chains RLHF, instruction-following RL, math RL, code RL, and SWE RL in sequence. Subsequent stages rarely degrade earlier domain performance and may even improve it, avoiding catastrophic forgetting. - **RLHF as reasoning booster:** RLHF for alignment, when used as a pre-step, boosts reasoning ability far beyond mere preference optimization, serving as a foundation for subsequent domain-specific RL stages. - **Strong competitive coding results:** The 14B model outperforms its SFT teacher DeepSeek-R1-0528 on LiveCodeBench v5/v6/Pro and achieves silver-medal performance at the 2025 International Olympiad in Informatics (IOI). - **Cross-domain excellence:** The 8B model achieves 71.1% on LiveCodeBench V6, 37.2% on SWE-bench Verified, and 80.1% on AIME 2025, outperforming larger models like Qwen3-8B and matching or exceeding frontier reasoning models. - **Transparent recipes:** NVIDIA shares complete training and data recipes, including multi-stage SFT, reward modeling, and domain-specific RL configurations for reproducibility.
Sequential domain-wise RL: The approach chains RLHF, instruction-following RL, math RL, code RL, and SWE RL in sequence. Subsequent stages rarely degrade earlier domain performance and may even improve it, avoiding catastrophic forgetting.
RLHF as reasoning booster: RLHF for alignment, when used as a pre-step, boosts reasoning ability far beyond mere preference optimization, serving as a foundation for subsequent domain-specific RL stages.
Strong competitive coding results: The 14B model outperforms its SFT teacher DeepSeek-R1-0528 on LiveCodeBench v5/v6/Pro and achieves silver-medal performance at the 2025 International Olympiad in Informatics (IOI).
Cross-domain excellence: The 8B model achieves 71.1% on LiveCodeBench V6, 37.2% on SWE-bench Verified, and 80.1% on AIME 2025, outperforming larger models like Qwen3-8B and matching or exceeding frontier reasoning models.
Transparent recipes: NVIDIA shares complete training and data recipes, including multi-stage SFT, reward modeling, and domain-specific RL configurations for reproducibility.
Abstract
Building general-purpose reasoning models with reinforcement learning (RL) entails substantial cross-domain heterogeneity, including large variation in inference-time response lengths and verification latency. Such variability complicates the RL infrastructure, slows training, and makes training curriculum (e.g., response length extension) and hyperparameter selection challenging. In this work, we propose cascaded domain-wise reinforcement learning (Cascade RL) to develop general-purpose reasoning models, Nemotron-Cascade, capable of operating in both instruct and deep thinking modes. Departing from conventional approaches that blend heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL, reducing engineering complexity and delivering state-of-the-art performance across a wide range of benchmarks. Notably, RLHF for alignment, when used as a pre-step, boosts the model's reasoning ability far beyond mere preference optimization, and subsequent domain-wise RLVR stages rarely degrade the benchmark performance attained in earlier domains and may even improve it (see an illustration in Figure 1). Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6/Pro and achieves silver-medal performance in the 2025 International Olympiad in Informatics (IOI). We transparently share our training and data recipes.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack