DE-Venus: A Data-Efficient RLVR Framework for Large Language Models
Shenzhi Yang and colleagues at Zhejiang University with Ant Group, HKBU, NTU and Southeast University present DE-Venus, a framework that treats RLVR supervision as evolving state across data preparation and policy optimization, so that sample selection, weak supervision and label correction can be compared inside one system rather than as separate papers.
Ask this paper
Three modules covering the supervision lifecycle: Active Data Selection allocates training and annotation budgets, Weak Supervision Construction derives signals from unlabeled examples, and Training-Time Supervision Refinement filters or corrects unreliable supervision.
Seven existing methods reimplemented inside it, expressed either as offline dataset transitions or as online transformations of targets, rewards, batches and advantages, which is what makes controlled comparison possible.
Preserves verl's distributed execution contracts, so the supervision logic is decoupled from the training infrastructure instead of entangled with it.
Label efficiency: configurations preserve or improve model quality with only 10 percent of labels, or as little as 13 percent of relevant data.
Deployment result: selected business configurations reduce observed convergence steps by 63 to 75 percent across three production scenarios.
Abstract
Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost of obtaining reliable targets at scale. Existing methods address sample selection, incomplete supervision, or noisy labels separately, often entangling supervision logic with distributed training and hindering controlled comparison and reuse. We present DE-Venus, a unified framework for data-efficient RLVR that treats supervision as evolving state across data preparation and policy optimization. It organizes this lifecycle into three modules: Active Data Selection allocates training and annotation budgets; Weak Supervision Construction derives learning signals from unlabeled examples; and Training-Time Supervision Refinement filters or corrects unreliable supervision. DE-Venus supports seven representative methods and a data-selection pipeline by expressing method-specific decisions as offline dataset transitions or online transformations of targets, rewards, batches, and advantages while preserving verl's distributed execution contracts. Across public benchmarks and three business scenarios, separate configurations preserve or improve model quality with only 10% of labels or as little as 13% of relevant data; selected business configurations also reduce observed convergence steps by 63%--75%. DE-Venus thus reduces annotation and training costs without sacrificing scalable RL execution.