Ring Attention
UC Berkeley's Ring Attention scales transformer context to 100M+ tokens by distributing blockwise self-attention across devices in a ring topology.
UniSim (Universal Simulator)
Google's UniSim learns a universal generative simulator of real-world interactions from diverse video + action data.
Survey on Factuality in LLMs
A survey covering evaluation and enhancement techniques for LLM factuality.
Hypothesis Search (LLMs Can Learn Rules)
A two-stage framework where the LLM learns a rule library for reasoning.
Meta Chain-of-Thought Prompting (Meta-CoT)
A generalizable CoT framework that selects domain-appropriate reasoning patterns for the task at hand.
LLMs for Healthcare Survey
A comprehensive overview of LLMs applied to the healthcare domain.
RECOMP (Retrieval-Augmented LMs with Compressors)
Proposes two compression approaches to shrink retrieved documents before in-context use.
InstructRetro
NVIDIA introduces Retro 48B, the largest LLM pretrained with retrieval at the time.
MemWalker
MemWalker treats the LLM as an interactive agent that traverses a tree-structured summary of long text.
FireAct (Language Agent Fine-tuning)
Explores fine-tuning LLMs specifically for language-agent use, demonstrating consistent gains over prompting alone.