Teaching LLMs to Plan
Free while signed in. Answers cite the passages they came from.
A training recipe that teaches LLMs to plan in Planning Domain Definition Language (PDDL) by making them write explicit state–action–state chains and checking each step with an external verifier (VAL). The result: big jumps in plan validity on PlanBench domains, especially when feedback explains why an action failed rather than just saying it failed.
Method in a nutshell: Two stages: (1) instruction tuning on correct and intentionally broken plans with explanations of preconditions and effects, then (2) CoT instruction tuning, where the model outputs ⟨s₀,a₁,s₁⟩… chains that VAL validates step-by-step. Training alternates between optimizing the reasoning chains and the final plan success.
Why it works: The verifier enforces logical coherence at each step, so the model learns to check preconditions, apply effects, and preserve invariants rather than pattern-match. This reduces unfaithful or hand-wavy CoT because every transition is externally validated.
Results: With Llama-3, detailed feedback and 15 iterations reach 94% plan validity on Blocksworld, 79% on Logistics, and 64% on Mystery Blocksworld. GPT-4 shows similar trends, peaking at 91%, 78%, and 59% respectively. Absolute improvements vs. baselines are large, e.g., +66% on some settings.
Feedback matters: Detailed feedback (which precondition failed or which effect was misapplied) consistently beats binary valid/invalid and benefits more from extra iterations (η from 10 to 15).
Scope and limits: Trained and tested on three PlanBench domains; performance drops on the obfuscated-predicate variant (Mystery Blocksworld), highlighting harder generalization. The method targets satisficing plans, not optimality, and currently assumes a PDDL subset without duratives or conditionals.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack