Metacognitive Scaffolds Create Productive Friction for AI-Assisted Knowledge Work
Viktor Kewenig, Lev Tankelevitch, Sean Rintel, Advait Sarkar and Abigail Sellen (Microsoft Research Cambridge) run a randomized experiment showing that AI-generated metacognitive scaffolds improve AI-assisted writing while users rate the AI as less helpful.
Ask this paper
Design. A between-subjects experiment with 297 workers writing 594 essays, testing dynamically generated scaffolds that prompt goal specification and task decomposition before and during AI-assisted writing.
Quality. Scaffolds improve essay quality, and the improvement persists in a short-lag transfer task after the scaffolds are removed.
Learning. Scaffolded participants show better topic recall three days later, and their prompting behavior changes in ways consistent with having learned to plan their requests.
Perception gap. Scaffolded participants rate the AI as less helpful and more cognitively demanding even though they perform better, so satisfaction ratings understate the value of the support.
Abstract
Generative AI imposes new metacognitive demands on knowledge workers as their work shifts from production to planning, evaluation, and orchestration. In a randomized, between-subjects experiment on AI-assisted writing (297 workers, 594 essays), we test dynamically-generated metacognitive scaffolds supporting goal specification and task decomposition. We find that scaffolds improve writing quality, with this boost persisting in short-lag transfer when scaffolds are removed. We also find evidence of improved topic recall three days later. These effects are consistent with observed changes in how people learn to prompt AI systems after exposure to scaffolding. However, scaffolded participants rate the AI as less helpful and more cognitively demanding even as they perform better, suggesting that subjective ease can understate the value of support. Our findings make the case for metacognitive scaffolds in AI-assisted knowledge work, revealing a productive friction worth designing for as workers increasingly delegate work to agentic AI.