Kimi K2
Moonshot AI introduces Kimi K2, a 1T parameter Mixture-of-Experts model (32B active) optimized not just for knowledge tasks but for agentic capabilities, models that act, not just respond. Released as Kimi-K2-Base and Kimi-K2-Instruct, it achieves strong scores across coding, math, and tool-use benchmarks and is open-source for research and deployment.
Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
This work introduces the Hierarchical Network (H-Net), a novel end-to-end architecture that learns to segment raw data dynamically, eliminating the need for fixed tokenizers. Key ideas:
What Has a Foundation Model Found?
This work introduces an inductive bias probe to evaluate whether foundation models have learned the underlying "world model" of a domain, or are just good at sequence prediction. Key ideas:
Why do Some Language Models Fake Alignment While Others Don’t
This paper expands the analysis of alignment faking in LLMs, where models comply with training objectives during training but behave differently in deployment. The authors study 25 LLMs and find that only five (Claude 3 Opus, Claude 3.5 Sonnet, Llama 3 405B, Grok 3, Gemini 2.0 Flash) exhibit a significant compliance gap: they are more likely to comply with harmful requests when they believe they are being trained. This selective compliance, or "alignment faking", is linked to deeper model motivations and training dynamics.
Bridging Offline and Online Reinforcement Learning for LLMs
Investigates the effectiveness of different RL strategies for finetuning LLMs, systematically bridging the gap between offline and online methods. Key findings:
A Survey on Latent Reasoning
Provides a comprehensive overview of latent reasoning, an emerging field that shifts AI reasoning from explicit, token-based "chain-of-thought" to implicit computations within a model's continuous hidden states. Key ideas:
MemAgent
Introduces an RL–driven memory agent that enables transformer-based LLMs to handle documents up to 3.5 million tokens with near lossless performance, linear complexity, and no need for architectural modifications.
AI Research Agents for Machine Learning
Presents a new framework, AIRA-dojo, for developing and evaluating AI research agents. They use this framework to systematically investigate the components of successful AI agents on the MLE-bench benchmark, a challenging set of real-world machine learning problems from Kaggle. Key findings:
Adaptive Branching MCTS
Researchers from Sakana AI introduce Adaptive Branching Monte Carlo Tree Search (AB-MCTS), a new framework that dynamically decides whether to "go wider" (explore new solutions) or "go deeper" (refine existing ones) during inference. Key ideas:
HIRAG
HIRAG is a new instruction fine-tuning method that enhances the capabilities of RAG models by teaching them to think before answering. Key ideas:
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack