Phi-3
Microsoft's Phi-3 is a family of small language models (3.8B, 7B, 14B) trained on 3.3-4.8T tokens of heavily filtered web data combined with synthetic data. The flagship phi-3-mini rivals Mixtral 8x7B and GPT-3.5 while being small enough to run locally on a phone.
OpenELM
Apple's OpenELM is a fully-open small language model family (270M, 450M, 1.1B, 3B) that uses layer-wise parameter scaling instead of uniform layer widths. At ~1B parameters it improves on OLMo by 2.36% accuracy while using half the pre-training tokens.
Arctic
Snowflake's Arctic is an Apache 2.0 open LLM with a Dense-MoE Hybrid transformer (480B total / 17B active) that matches Llama 3 70B on enterprise metrics while using under 3K GPU weeks (~$2M) of training compute - roughly 17x less than Llama 3 70B.
Make Your LLM Fully Utilize the Context (FILM-7B)
FILM-7B targets the lost-in-the-middle problem where long-context LLMs fail to retrieve information buried between the start and end of their input. The authors apply an information-intensive (IN2) training recipe to Mistral-7B that forces uniform attention across the full 32K window.
FineWeb
HuggingFace's FineWeb is a 15 trillion token English web dataset built from 96 CommonCrawl snapshots (2013-2024). In 1.8B-parameter ablations, models trained on FineWeb beat C4, RefinedWeb, Dolma, The Pile, SlimPajama, and RedPajama2 across aggregated benchmarks.
AI-powered Gene Editors
Profluent's OpenCRISPR-1 paper demonstrates that a large protein language model trained on biological diversity at scale can design programmable gene editors from scratch. The AI-designed editors successfully perform precision editing in the human genome.
AutoCrawler
AutoCrawler is a two-stage framework that combines LLMs with the hierarchical structure of HTML to auto-generate reusable web scrapers. Wrapper-based scrapers break on new sites and pure LLM agents don't reuse well across pages; AutoCrawler addresses both limitations.
Graph Machine Learning in the Era of LLMs
This survey maps the intersection of Graph ML and LLMs, covering both how LLMs enhance graph learning and how graphs (especially knowledge graphs) strengthen LLMs. The authors organize the literature into a taxonomy and highlight where open problems remain.
Self-Evolution of LLMs
This survey organizes the emerging literature on self-evolving LLMs - models that improve through their own generated experience rather than additional human supervision. The authors propose a unified four-phase cycle and taxonomize existing methods across both standalone models and agent systems.
Naturalized Execution Tuning (NExT)
NExT teaches LLMs to reason about program runtime behavior by generating synthetic chain-of-thought rationales over execution traces. The approach bootstraps training data through self-training rather than manual annotation, and the learned reasoning transfers to scenarios where traces are unavailable at inference.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack