Evolutionary Model Merge

Two search spaces: Evolution operates in parameter space (weight mixing coefficients) and in data-flow space (which layers of which models to route activations through), giving a richer merge vocabulary than hand-crafted recipes.

Cross-domain transfer: The method produces a Japanese LLM with math reasoning by merging unrelated parents - showing that evolution can graft capabilities from one model onto another even across languages.

New SoTA with less compute: The resulting Japanese Math LLM achieves state-of-the-art on Japanese LLM benchmarks, beating models with many more parameters that were never explicitly trained for either language or math.

Collective intelligence: Positions evolutionary merge as an automated composition paradigm that leverages the open-source community's collective work instead of training every capability from scratch.