Calibration-Aware Uncertainty Cascades for Efficient Heterogeneous Model Collaboration
Yilin Zhang, Han Jiang, Cai Xu, Ying Liu and Wei Zhao propose Calibration-Aware Uncertainty Cascades, which calibrate each model's confidence separately so that one threshold rule can decide, across any pool of models, when to accept a cheap model's answer and when to call a stronger one.
Ask this paper
Problem: Trained routers are tied to a fixed task and model pool, and raw-confidence cascades use thresholds whose meaning differs from model to model.
Method: Each model's confidence is calibrated post hoc, and deployment policies are chosen on validation data. The calibrated scores give a common scale for accepting an early prediction, invoking a stronger model, or combining outputs.
Theory: Calibration gives confidence thresholds an explicit selective-risk interpretation, and uncalibrated scores have no comparable guarantee.
Results: Across six language benchmarks, accuracy improves by 1.9% relative to strong-model-only inference while about 47% of strong-model calls are avoided. On image classification, GFLOPs drop by up to 57% at equal or better accuracy.
Abstract
Heterogeneous model collaboration seeks to exploit the complementary strengths of different models to balance predictive performance and inference cost. Existing approaches typically rely either on trained routers, which tie routing decisions to a fixed task and model pool, or on raw-confidence cascades, whose thresholds lack consistent reliability semantics across heterogeneous models. Consequently, these approaches adapt poorly to changing model pools and deployment budgets. We propose Calibration-Aware Uncertainty Cascades (CAUC), a simple post-hoc framework that independently calibrates each model's confidence and selects deployment policies using validation data. The resulting calibrated confidence scores establish a common reliability scale for accepting an early prediction, invoking a stronger model, or selectively combining model outputs. This unified decision criterion decouples deployment policies from any particular model pool or operating budget. We further show theoretically that calibration gives confidence thresholds an explicit selective-risk interpretation, whereas uncalibrated scores offer no comparable reliability guarantee. Extensive experiments demonstrate that, across six language benchmarks, CAUC achieves an average relative accuracy improvement of 1.9% over strong-model-only inference while avoiding approximately 47% of strong-model calls. On image classification benchmarks, it maintains or improves predictive performance while reducing measured GFLOPs by up to 57%.