OLMo
Allen AI releases OLMo, a truly open 7B-parameter LLM shipped with training code, pretraining data, full weights, evaluation tooling, and fine-tuning recipes - an answer to the "open-weights but closed-pipeline" releases dominating the space.
Advances in Multimodal LLMs
A comprehensive survey mapping design choices for architecture and training pipeline around multimodal large language models (MLLMs).
Corrective RAG (CRAG)
CRAG adds a self-correcting loop around retrieval so a RAG system can detect and repair bad retrievals instead of feeding them straight into generation.
LLMs for Mathematical Reasoning
A survey of the fast-growing literature on using LLMs for mathematical reasoning, from arithmetic word problems to theorem proving.
Compression Algorithms for LLMs
A survey covering the main families of LLM compression techniques and when each one is appropriate.
MoE-LLaVA
MoE-LLaVA applies Mixture-of-Experts tuning to the LLaVA vision-language architecture, getting a sparse model with dramatically fewer active parameters at the same compute cost.
Rephrasing the Web (WRAP)
WRAP uses an off-the-shelf instruction-tuned model to paraphrase web documents into styles like "Wikipedia" or "question-answer format" and trains on the mixture of real + synthetic rephrases.
The Power of Noise: Redefining Retrieval in RAG
A study stress-testing the retriever component of RAG systems with surprising results about what actually helps generation.
Hallucination in LVLMs
A survey specifically scoped to hallucination in Large Vision-Language Models, a phenomenon that differs substantially from text-only LLM hallucination.
SliceGPT
Microsoft's SliceGPT is a post-training LLM compression technique that literally slices rows and columns out of weight matrices while preserving zero-shot quality.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack