Paritok-4B, context compression that pays for itself
Free while signed in. Answers cite the passages they came from.

Jiayu Shi and Luzhuo Chen release Paritok-4B, a 4B LoRA compressor that shrinks coding-agent context to a quarter of its size by extracting spans rather than paraphrasing them, with weights and data open under Apache 2.0.
Extractive by design: 96.0% of emitted identifiers, paths and numbers already appear in the input (96.2% on held-out SWE-bench Lite), which is exactly the property general prose compressors violate when they rename variables.
Intent-conditioned selection, not variable retention: Told the agent's current task, it changes which lines survive rather than how many; retained lines score +0.067 more intent-relevant than removed ones, 95% CI [+0.056, +0.078].
Compresses 2.0x harder than gpt-4.1-mini: 25.7% of original size versus 50.2% for gpt-4.1-mini and 61.9% for gpt-5, retaining 86.5% of uncompressed single-shot solve quality across all 300 SWE-bench Lite instances.
The economics are the argument: A 264 MB adapter self-hosting on one 24 GB GPU has no per-token compressor fee; at list prices gpt-5 as a compressor is net-negative, costing more than the downstream tokens it saves.
Abstract
Coding agents re-send large file reads and tool outputs to a frontier LLM every turn, and this context dominates their token bill. General-purpose prompt compressors are trained on prose and suit code poorly: they paraphrase identifiers and drop the exact spans an agent needs to edit. We present Paritok-4B, a 4B LoRA compressor for coding-agent trajectories built on two commitments. It is extractive: it selects spans rather than rewriting them, and 96.0% of the identifiers, paths, and numbers it emits already appear in its input, holding at 96.2% on held-out SWE-bench Lite output. It is intent-conditioned: told the agent's current task, it acts chiefly inside a retained segment, selecting which lines survive (retained lines are +0.067 more intent-relevant than removed ones, paired 95% CI [+0.056, +0.078]) rather than changing how much is retained. We distil a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories into 40,606 validated examples and fine-tune Qwen3-4B. On all 300 SWE-bench Lite instances, Paritok-4B compresses agent context to 25.7% of its size, 2.0x harder than a gpt-4.1-mini compressor (50.2%) and 2.4x harder than gpt-5 (61.9%), while retaining 86.5% of uncompressed single-shot solve quality. Fed the cat -n line-numbered input real agents produce, it compresses slightly less (27.8%) and retains more (89.3%); there the paired test is informative, with 30 instances solved only uncompressed and 17 only compressed, an exact McNemar p=0.079, so at this sample size compressing context to roughly a quarter of its size does not significantly reduce the solve rate. The model is a 264 MB adapter that self-hosts on one 24 GB GPU with no per-token compressor fee, which at list prices decides the economics: gpt-5 as a compressor is net-negative, costing more than the downstream tokens it saves. Weights, data, and evaluation scripts are open (Apache 2.0).
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack