Adversarial Program Evolution with LLMs
Free while signed in. Answers cite the passages they came from.
Digital Red Queen (DRQ) introduces an algorithm where LLMs evolve assembly-like programs called "warriors" that compete for control of a virtual machine in the game of Core War. Rather than optimizing toward static objectives, DRQ embraces "Red Queen" dynamics where goals continually shift based on competition, demonstrating how adversarial self-play can drive the evolution of increasingly sophisticated programs. - **Core War as testbed:** The classic programming game serves as an ideal environment for studying adversarial adaptation, where programs must simultaneously attack opponents and defend themselves in shared memory space. - **Emergent generalization:** Evolved warriors become increasingly effective against unseen opponents, suggesting that competitive dynamics produce more robust solutions than static optimization objectives. - **Behavioral convergence:** Despite independent evolutionary runs, warriors show paradoxical behavioral convergence, indicating that competitive pressure discovers similar successful strategies across different lineages. - **Dynamic objectives outperform static:** The research demonstrates that continually shifting competitive objectives can outperform traditional static optimization for evolving general-purpose solutions. - **Broad applications:** The approach has implications for cybersecurity (evolving attack/defense strategies), evolutionary biology (modeling arms races), and AI safety (understanding adversarial dynamics in multi-agent systems).
Core War as testbed: The classic programming game serves as an ideal environment for studying adversarial adaptation, where programs must simultaneously attack opponents and defend themselves in shared memory space.
Emergent generalization: Evolved warriors become increasingly effective against unseen opponents, suggesting that competitive dynamics produce more robust solutions than static optimization objectives.
Behavioral convergence: Despite independent evolutionary runs, warriors show paradoxical behavioral convergence, indicating that competitive pressure discovers similar successful strategies across different lineages.
Dynamic objectives outperform static: The research demonstrates that continually shifting competitive objectives can outperform traditional static optimization for evolving general-purpose solutions.
Broad applications: The approach has implications for cybersecurity (evolving attack/defense strategies), evolutionary biology (modeling arms races), and AI safety (understanding adversarial dynamics in multi-agent systems).
Abstract
Large language models (LLMs) are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological evolution, most LLM-evolution frameworks are formulated as static optimization problems, overlooking the open-ended adversarial dynamics that characterize real-world evolutionary processes. Here, we study Digital Red Queen (DRQ), a simple self-play algorithm that embraces these so-called "Red Queen" dynamics via continual adaptation to a changing objective. DRQ uses an LLM to evolve assembly-like programs, called warriors, which compete against each other for control of a virtual machine in the game of Core War, a Turing-complete environment studied in artificial life and connected to cybersecurity. In each round of DRQ, the model evolves a new warrior to defeat all previous ones, producing a sequence of adapted warriors. Over many rounds, we observe that warriors become increasingly general (relative to a set of held-out human warriors). Interestingly, warriors also become less behaviorally diverse across independent runs, indicating a convergence pressure toward a general-purpose behavioral strategy, much like convergent evolution in nature. This result highlights a potential value of shifting from static objectives to dynamic Red Queen objectives. Our work positions Core War as a rich, controllable sandbox for studying adversarial adaptation in artificial systems and for evaluating LLM-based evolution methods. More broadly, the simplicity and effectiveness of DRQ suggest that similarly minimal self-play approaches could prove useful in other more practical multi-agent adversarial domains, like real-world cybersecurity or combating drug resistance.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack