MATCH: Model-Aware Tool Learning with Curriculum Scheduling and Hierarchically Gated Rewards
Shihao Liu and colleagues address two failures in RL for tool use, curricula with fixed difficulty thresholds and additive rewards that leak argument credit when the tool itself is wrong, with MATCH.
Ask this paper
The curriculum tracks the policy. Model-Aware Curriculum Learning maintains reward-derived sample difficulty that co-evolves with the policy, selecting samples near the current capability boundary plus a top-k pool of harder cases.
Rewards are gated, not summed. Hierarchical Tool-call Gated Reward scores tool name, argument key and argument value as a chain, granting credit at a level only when its prerequisites hold, so a wrong tool cannot earn argument credit.
One reward signal closes the loop. The same HTGR scores drive both the GRPO update and the curriculum's difficulty refresh.
72.19 and 62.87 percent overall accuracy. On API-Bank and BFCL V3, above the main supervised and RL baselines, with consistent gains across four backbones from two model families.
Abstract
Tool learning enables large language models (LLMs) to use external tools for tasks beyond parametric knowledge. Reinforcement learning can optimize tool-call behavior from feedback, but current methods still face two problems: fixed-threshold curricula can become misaligned with the policy's evolving capability boundary, and additive rewards can leak argument-level credit when the predicted tool is wrong. To address these problems, we propose MATCH, a closed-loop framework for model-aware tool learning with curriculum scheduling and hierarchically gated rewards. Model-Aware Curriculum Learning (MACL) maintains reward-derived sample difficulty that co-evolves with the policy, and each epoch selects samples near the current capability boundary together with a top-k pool of harder cases. Hierarchical Tool-call Gated Reward (HTGR) scores tool name, argument key, and argument value as a gated chain, granting credit at each level only when prerequisites hold. The same HTGR rewards drive both GRPO updates and MACL's difficulty refresh, closing the loop between policy optimization and sample scheduling. On API-Bank and BFCL V3, MATCH reaches 72.19% and 62.87% overall accuracy, outperforming the main supervised and RL-based baselines. Backbone experiments further show consistent improvements across four backbones from two model families.