AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Discover and explore top AI papers with Claude Code or Codex
npx @dair-ai/mcp setup
Efficient Inference of LLMs
proposes a layer-condensed KV cache to achieve efficient inference in LLMs; only computes and caches the key-values (KVs) of a small number of layers which leads to saving memory consumption and improved inference throughput; can achieve up to 26x higher throughput than baseline transformers while maintaining satisfactory performance.
You Only Cache Once
a decoder-decoder LLM architecture that only caches key-value pairs once; it involves a cross-decoder stacked upon a self-decoder which efficiently encodes global key-value caches and the cross-encoder reuses the cache via cross-attention; this leads to a significant reduction in GPU memory use without sacrificing capabilities; achieves comparable performance to Transformer in various settings of scaling up model size and number of training token.
Med-Gemini
presents a family of multimodal models specialized in medicines and based on the strong multimodal and long-context reasoning capabilities of Gemini; achieves state-of-the-art performance on 10/14 benchmarks surpassing GPT-4 models; it achieves 91% accuracy on MedQA (USMLE) benchmark using an uncertainty-guided search strategy.
In-Context Learning with Long-Context Models
studies the behavior in-context learning of LLMs at extreme context lengths with long-context models; shows that performance increases as hundreds or thousands of demonstrations are used; demonstrates that long-context ICL is less sensitive to random input shuffling than short-context ICL; concludes that the effectiveness of long-context LLMs is not due to task learning but from attending to similar examples.
Make Your LLM Fully Utilize the Context (FILM-7B)
FILM-7B targets the lost-in-the-middle problem where long-context LLMs fail to retrieve information buried between the start and end of their input. The authors apply an information-intensive (IN2) training recipe to Mistral-7B that forces uniform attention across the full 32K window.
LM In-Context Recall is Prompt Dependent
Using needle-in-a-haystack tests across multiple models, this paper shows that in-context recall is highly sensitive to prompt wording and that training data biases can silently degrade a model's ability to retrieve from its own context.
Leave No Context Behind (Infini-attention)
Google's Infini-attention extends Transformer LLMs to effectively infinite context with bounded memory and compute. It blends a compressive memory module with both masked local attention and linear long-term attention inside a single Transformer block.
Long-context LLMs Struggle with Long In-Context Learning
LongICLBench stress-tests 13 long-context LLMs on extreme-label classification with up to 174 classes and 50K-token prompts, exposing sharp quality cliffs beyond 20K tokens.
Grok-1.5
xAI's Grok-1.5 is the successor to the open-weight Grok-1, emphasizing long-context understanding and substantially stronger math, code, and reasoning performance.
AIOS
AIOS treats the LLM as the "brain" of an operating-system kernel for agents, providing scheduling, memory, storage, tool, and access-control services so agent apps can share resources safely.
Claude 3
Anthropic releases the Claude 3 family (Haiku, Sonnet, Opus), with Opus leapfrogging GPT-4 on many standard benchmarks and bringing frontier multimodal capability plus a much larger context window.
GaLore
GaLore (Gradient Low-Rank Projection) reduces optimizer-state memory during LLM training while still permitting full-parameter updates, unlike LoRA-style adapters that restrict learning to a low-rank subspace.
Recurrent Memory Finds What LLMs Miss
Introduces BABILong, a new long-context benchmark, and shows that transformers with recurrent memory can handle sequences far beyond vanilla LLMs.
Gemini 1.5
Google DeepMind's Gemini 1.5 is a multimodal MoE LLM that scales context to 1M tokens (10M in research settings) while matching or surpassing Gemini 1.0 Ultra on standard benchmarks.
Large World Model (LWM)
UC Berkeley's LWM is an open 7B multimodal model trained on long videos and books that handles context windows up to 1M tokens via RingAttention.
RAISE
RAISE is an advanced agent architecture that adds a dual-memory system on top of a ReAct-style backbone to better support long-running conversational agents.
Advancing Long-Context LLMs
A survey of methodologies for improving Transformer long-context capability across pretraining, fine-tuning, and inference stages.
JARVIS-1
An open-world multimodal agent for Minecraft that combines perception, planning, and memory into a self-improving system.
S-LoRA
S-LoRA enables serving thousands of LoRA adapters concurrently on a single GPU through memory-paging and custom CUDA kernels.
YaRN (Efficient Context Extension)
YaRN is a compute-efficient method for extending the context window of LLMs well beyond their pretrained length.
Ring Attention
UC Berkeley's Ring Attention scales transformer context to 100M+ tokens by distributing blockwise self-attention across devices in a ring topology.
MemWalker
MemWalker treats the LLM as an interactive agent that traverses a tree-structured summary of long text.
Retrieval Meets Long-Context LLMs
NVIDIA's study comparing RAG and long-context LLMs, with the punchline that the two are complementary rather than substitutes.
StreamingLLM
MIT's StreamingLLM enables efficient streaming inference by preserving "attention sinks" - early-sequence tokens that most attention mass flows to.