LegalBench
First page
Paper summary
A collaboratively constructed benchmark for measuring legal reasoning in LLMs.
Ask this paper
01
162 tasks: Covers 162 legal-reasoning tasks designed by legal experts, significantly broader than prior legal benchmarks.
02
Six reasoning categories: Categorizes tasks across rule-recall, rule-application, rule-conclusion, interpretation, rhetorical-analysis, and issue-spotting.
03
Collaborative construction: Built through collaboration with legal practitioners to ensure tasks reflect real legal reasoning rather than generic NLP tasks dressed in legal vocabulary.
04
LLM-lawyer evaluation: Provides the first rigorous benchmark for systematically evaluating LLM legal capability - essential for responsible deployment in legal workflows.