From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs
Kunal Jha, Max Kleiman-Weiner and Natasha Jaques at the University of Washington ask whether self-improving LLM agents that each pursue their own reward can learn from one another well enough to improve the whole population, a capability they call recursive social improvement.
Ask this paper
Setup. Agents revise skill files and decide whether, when and whom to copy, with private search, peer observation and acting all drawn from one token budget.
Bandit environments. Classic social-learning algorithms benefit from peers, but GPT-OSS-20B, Qwen and Ministral populations earn less reward per token than solo learners, exploring too narrowly or running out of tokens before acting.
Self-written skills. With GPT-OSS-120B and GLM-5 writing their own skills, peer observation helps one model find useful skills sooner and lets another spend less on private search, but neither beats independent learners at equal cost.
Diversity loss. Copying concentrates the population on few discoveries; in GPT-OSS social runs the executed skills descend from about one founding lineage on average versus about four for solo populations.
Conclusion. Current LLMs use peers to make learning cheaper, not to make it better.
Abstract
Large language models (LLMs) can now improve themselves by revising the instructions they follow, and LLM agents are increasingly orchestrated to work together on complex problems. However, self-improvement methods typically optimize one system at a time, and multi-agent frameworks often have every model work toward a shared goal. We ask a different question. When each agent pursues its own reward, can self-improving LLMs learn from one another well enough to improve the whole population? We call this capability recursive social improvement. We study populations that revise skill files and choose whether, when, and whom to copy from. Independent search, learning from peers, and acting all share one token budget. In controlled environments, established social-learning algorithms benefit from peers, but three LLMs do not. They earn less reward per token than solo learners, and explore too narrowly or run out of tokens before acting. We then let the models write and revise their own skills. Observing peers changes how they improve, helping one model find useful skills sooner and another spend less on private search. Neither, however, outperforms independent learners at the same cost. Skills are copied, revised, and passed on, so one discovery can seed further search. Yet these exchanges concentrate the population around fewer independent discoveries. Together, these results show that LLMs can make learning more efficient by copying from peers, but not yet more effective.