SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale
Md Tahmid Rahman Laskar, Xue-Yong Fu and Shashi Bhushan TN at Dialpad compare LoRA SFT, GRPO and SFT followed by GRPO for tool calling across six Qwen3 models from 0.6B to 32B, measuring both in-distribution accuracy and cross-dataset transfer.
Ask this paper
SFT wins in distribution: LoRA SFT is the strongest in-distribution method at every scale and best in 15 of 18 settings.
Transfer is close: GRPO wins 29 of 54 cross-dataset settings, but its average margin over SFT is under one point.
SFT then GRPO rarely helps: The two-stage recipe is seldom the best method in either the in-distribution or transfer comparison.
Data mixing and LoRA: Mixing datasets gives consistently strong transfer while staying near specialized training, and LoRA outperforms full fine-tuning, which the authors attribute to better preservation of pretrained agentic behavior.
Abstract
Limited controlled evidence exists on how training data, adaptation method, and model scale jointly affect tool-calling performance in language-model agents. We evaluate supervised fine-tuning (SFT) with LoRA, reinforcement learning (RL) via Group Relative Policy Optimization (GRPO), and SFT followed by GRPO across six Qwen3 models from 0.6B to 32B parameters, covering both in-distribution performance and cross-dataset transfer. SFT with LoRA is the strongest in-distribution method throughout the 0.6B-32B range and best in 15 out of 18 experimental settings. On cross-dataset transfer, the methods are closer: GRPO wins 29 out of 54 settings where training and test datasets differ, but its margin over SFT averages under one point, and SFT->GRPO is rarely strongest in either comparison. Dataset mixing gives consistently strong transfer while staying close to specialized in-distribution training, regardless of method. Additional analysis further confirms that LoRA outperforms full-parameter fine-tuning, demonstrating that LoRA better preserves pretrained agentic behavior.