Symbol tuning
Free while signed in. Answers cite the passages they came from.
Fine-tunes LMs on in-context input-label pairs with natural-language labels replaced by arbitrary symbols.
Symbolic abstraction: Replacing semantic labels with random symbols forces the model to rely on the demonstrations rather than label priors.
ICL improvements: Boosts performance on unseen in-context learning tasks where the model must infer label semantics from examples.
Algorithmic reasoning: Particularly improves algorithmic reasoning tasks that require following abstract patterns.
ICL mechanism insight: Provides evidence about how ICL works and how to train models that better generalize the mechanism.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack