Recurrent Harness for Agentic Continual Learning
Yuxin Xiao (MIT, work done at Meta), Bailin Wang and colleagues (Meta MRS Research) propose ReHarness, which revises the reflections an agent retrieved for an instance based on how that instance turned out, so harness state gets backward feedback instead of only accumulating.
Ask this paper
Problem. Memory and reflection harnesses write experience forward and rarely update an entry based on whether it helped later. Add and delete operations keep the bank coherent but do not use downstream outcomes.
Backward revision. After each instance, the trajectory is used as feedback to rewrite the reflections that were retrieved for it.
Results. Across three continual-learning benchmarks (16 tasks) and three backbones, ReHarness outperforms raw-context, memory, reflection and prompt-optimization baselines, with statistically significant gains after multiple-comparison correction.
Distribution shift. Revisions are grounded in source evidence and correct overgeneralized or outdated experience, improving both stability and plasticity.
Retrieval size. Performance rises up to k=5 retrieved reflections and then levels off at higher cost.
Abstract
Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.