🚀NEW LABGetting Started with Claude AgentsStart lab
Evaluation

GenBench

Paper preview
GenBench
Paper summary

A Nature Machine Intelligence paper framework for characterizing and understanding generalization research in NLP.

Ask this paper

Key points
01

Meta-analysis: Reviews 543 papers on generalization in NLP, mapping what "generalization" actually means across different research threads.

02

Generalization taxonomy: Organizes generalization into compositional, structural, cross-lingual, cross-task, and cross-domain generalization types.

03

Evaluation taxonomy: Provides tools for classifying generalization studies by the kind of distribution shift and evaluation protocol they test.

04

Research infrastructure: Ships with tools to help researchers classify and compare generalization work, aiming to reduce conceptual fragmentation in the field.

Every Monday
Get next week’s papers.
Subscribe on Substack