For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances
Alexander Shirnin and Aleksey Kudelya build a cooperative signalling game in which a Sender describes two words and an isolated Receiver, sharing only pretraining and task instructions, must identify the hidden target.
Ask this paper
Seven models, four architecture families, 300 word pairs. Pairs come from established psycholinguistic corpora, and the Double-Pass Success Rate controls for output biases that would otherwise inflate coordination scores.
Filtering for detectability breaks most models. Once required to avoid detectable signals, most models lose coordination; one frontier model keeps near-perfect performance after the same filtering.
The capability can be pointed at deception. Models can use the same channel for deliberate misdirection, which is the property that matters for automated pipelines consuming model output.
Cross-family coordination is weaker than within-family. Coordination is consistently better between instances of the same model than across architectures, which bounds how far the effect generalises.
Abstract
As model-generated content is increasingly consumed by other model instances in automated workflows, a practically important question arises: can a model embed a signal in natural language that an independent instance of the same model can detect, relying only on shared pre-training and task instructions, without any shared memory or coordination-specific training? We introduce For Your Eyes Only, a cooperative signalling game designed to evaluate this directly. A Sender produces free-form descriptions for two words, one of which is a hidden target; an isolated Receiver must identify it. We evaluate seven contemporary models from four architectural families on 300 word pairs from established psycholinguistic corpora, using the Double-Pass Success Rate to control for output biases. We find that most models struggle to maintain coordination once they are required to avoid detectable signals, while one frontier model retains near-perfect performance even after such filtering. We further show that models can direct this capability toward deliberate misdirection, and that coordination is consistently weaker across architectures than within them.