RankZephyr
RankZephyr is an open-source LLM for listwise zero-shot reranking that bridges the effectiveness gap with GPT-4.
Ask this paper
Listwise zero-shot: Reranks a full candidate list in a single shot rather than doing pairwise or pointwise scoring, matching the paradigm GPT-4 uses most effectively.
Open-source: Based on the open Zephyr chat model, releasing a fully reproducible stack for high-quality reranking.
Matches/beats GPT-4: Competitive with GPT-4 on standard reranking benchmarks and outperforms GPT-4 on NovelEval, a post-training-cutoff benchmark resistant to contamination.
Contamination-free win: The NovelEval advantage is particularly meaningful because it addresses the concern that GPT-4's strong reranking numbers are partly driven by memorization of benchmark queries.