Can LLMs Reason and Plan?
Free while signed in. Answers cite the passages they came from.
Kambhampati's position paper argues that what looks like reasoning and planning in LLMs is better understood as "universal approximate retrieval" powered by web-scale training.
Core claim: "Nothing that I have read, verified, or done gives me any compelling reason to believe that LLMs do reasoning/planning, as normally understood" - LLMs interpolate over memorized patterns rather than search solution spaces.
Planning evaluations: Cites benchmarks like PlanBench where LLM performance collapses under obfuscation or natural adversarial perturbations, arguing that true planning would be more robust.
Self-critique skepticism: Questions claims that LLMs can reliably self-critique, noting that current systems hallucinate evaluations as readily as they hallucinate answers.
LLM-Modulo framework: Proposes using LLMs as components ("idea generators") alongside sound external verifiers (planners, theorem provers) rather than trusting them as standalone reasoners.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack