ACE: Adaptive Calibration-Free Expert Skipping for MoE-based LLMs
Zukang Xu and colleagues skip Mixture-of-Experts expert slots per token at inference without calibration data, training, or a modified checkpoint, by estimating each expert's actual contribution rather than trusting the router's confidence.
Ask this paper
Two estimators, both offline: A Global Spectral Proxy reads global transformation capacity from the coupled gate, up and down projections plus RMSNorm scaling. Router-Conditioned Refinement builds expert-specific direction prototypes from centered router weights and scores expert responses along routing-preferred directions.
Conservative skip rule: An expert slot is dropped only when both views call it low-contribution, and the top-1 expert is always retained.
Runtime cost: All expert statistics are precomputed, so inference adds table lookups and scalar operations only.
Measured effect: At a 50% skipping ratio on Qwen3.6-35B-A3B, ACE lowers WikiText-2 perplexity by 7.96% and raises average downstream accuracy by 4.15 points over the strongest competing method, with the advantage widening as skipping gets more aggressive.
Abstract
Mixture-of-Experts (MoE) architectures provide an efficient paradigm for scaling large language models (LLMs), yet fixed top-k routing activates the same number of expert slots for every token, causing substantial redundant computation. Existing expert-skipping methods often rely on router confidence, calibration data, or additional training, and therefore cannot reliably estimate the actual contribution of routed experts. To this end, we propose ACE, a training-free, calibration-free, and checkpoint-preserving framework for token-adaptive expert skipping in MoE-based LLMs. ACE contains two complementary components: 1) Global Spectral Proxy (GSP), which estimates global transformation capacity from the coupled gate, up, and down projections together with RMSNorm scaling; and 2) Router-Conditioned Refinement (RCR), which constructs expert-specific direction prototypes from centered router weights and evaluates expert responses along routing-preferred directions. During inference, ACE combines both estimates with runtime router gates and skips an expert slot only when both views identify it as low-contribution, while always retaining the top-1 expert. All expert statistics are computed offline, leaving only table lookups and lightweight scalar operations online. Extensive experiments across three MoE-based LLMs and eight benchmarks demonstrate that ACE consistently outperforms existing static and dynamic baselines, with increasingly pronounced advantages under aggressive expert skipping. For instance, at a 50% skipping ratio on Qwen3.6-35B-A3B, ACE reduces WikiText-2 perplexity by 7.96% and improves average downstream accuracy by 4.15 percentage points over the strongest competing method.