🚀NEW LABGetting Started with Claude AgentsStart lab
← All papers  /  Oct 23, 2026
Agents

PLE: Physical-Engineering's Last Exam

First page
PLE: Physical-Engineering's Last Exam
The curator’s take

Rujun Han, Chen-Yu Lee, Tomas Pfister and colleagues at Google Cloud AI Research introduce PLE, 85 long-horizon agent tasks in which solving a task means producing a billable electrical, civil, structural or mechanical engineering deliverable.

Ask this paper

Key points
01

Task design. Each task gives a multi-format project corpus and requires modelling physics and building-code requirements, optimizing material and installation cost, and issuing 2D/3D design packages a licensed reviewer would accept.

02

Two curation paths. Expert-Authored tasks are written end to end by practising engineers. Co-Authored tasks start from an engineer's problem statement, then an adversarial Creator-Solver agent pipeline builds the environment, executable grader and reference deliverables before expert validation.

03

Result. Across frontier LLMs under seven public agent harnesses, the best configuration averages 50.6%, while spending 27M tokens and 63 minutes per task.

04

Why it matters. Most engineering benchmarks are textbook QA or toy design problems that leading agents already saturate; PLE measures full professional deliverables instead.

Abstract

Frontier language models now score above 90% on popular knowledge benchmarks and are increasingly deployed as agents that carry out multi-hour professional work. Yet no existing benchmark tests whether they can perform the real-world, economically valuable work of a professional physical engineer, such as modelling physics and legal code requirements across a multi-format project corpus, optimising material and installation costs, and issuing 2D/3D engineering design packages that a licensed reviewer would sign off on. Existing engineering benchmarks focus on textbook question answering, isolated tool calls or self-contained toy design problems. As a result, most of these benchmarks are already saturated by leading agents and solving them is far away from delivering real engineering design. We introduce PLE, a benchmark of 85 long-horizon, agentic tasks in professional physical engineering consisting of electrical, civil, structural and mechanical engineering, where solving a task means completing a realistic, billable engineering deliverable. PLE combines two complementary curation paths. Expert-Authored tasks are created end to end by practising engineers from a professional data provider's expert pool. Co-Authored tasks start from problem statements written by practising engineers; an adversarial Creator-Solver agent pipeline constructs each task's realistic project environment, executable grader and reference deliverables before sending them to expert engineers for validation and difficulty intensification. We evaluate a variety of frontier LLMs under seven publicly available agent harnesses. Even the strongest configuration reaches a mean score of only 50.6%, while spending 27M tokens and 63 minutes per task on average, demonstrating the substantial gap between current agents and autonomous physical engineering design.

Every Monday
Get next week’s papers.
Subscribe on Substack