Retrieve In-Context Examples for LLMs

Iterative training: Trains retrievers using LLM feedback in an iterative loop - retrieved examples that help the LLM answer correctly are used as positive signals.

30-task evaluation: Evaluated across 30 NLP tasks showing consistent improvements over random or similarity-based retrieval.

Pattern-similar examples: Confirms that examples sharing abstract patterns (not just surface similarity) are most useful for ICL.

Scale-invariant gains: Improvements are consistent across model sizes, suggesting dense retrieval is a robust ICL enhancement that transfers across model scales.