AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
LLM Self-Explanations
Investigates whether LLMs can generate useful feature-attribution explanations for their own outputs.
Hypothesis Search (LLMs Can Learn Rules)
A two-stage framework where the LLM learns a rule library for reasoning.
Meta Chain-of-Thought Prompting (Meta-CoT)
A generalizable CoT framework that selects domain-appropriate reasoning patterns for the task at hand.
MemWalker
MemWalker treats the LLM as an interactive agent that traverses a tree-structured summary of long text.
LLMs Represent Space and Time
MIT researchers find that LLMs internally encode linear representations of space and time across multiple scales.
The Dawn of LMMs (GPT-4V Deep Dive)
Microsoft's exhaustive 166-page analysis of GPT-4V's capabilities and limitations.
Training LLMs with Pause Tokens
CMU shows that adding a learnable `<pause>` token during both pretraining and fine-tuning gives the model extra "thinking time" and improves reasoning.
Analogical Prompting
Google's Analogical Prompting guides LLM reasoning by having the model self-generate relevant exemplars on the fly.
Effective Long-Context Scaling (Meta)
Meta proposes a 70B long-context LLM that surpasses GPT-3.5-turbo-16k on long-context benchmarks.
Boolformer
The first Transformer trained to perform end-to-end symbolic regression of Boolean functions.
Logical Chain-of-Thought (LogiCoT)
A neurosymbolic framework that verifies and revises zero-shot CoT reasoning using symbolic-logic principles.
Chain-of-Verification (CoVe)
Meta's Chain-of-Verification adds a "deliberation" step where the LLM fact-checks its own draft before finalizing.
Contrastive Decoding for Reasoning
Shows that contrastive decoding, a simple inference-time technique, substantially improves reasoning in large LLMs.
Textbooks Are All You Need II (phi-1.5)
Microsoft's phi-1.5 demonstrates that a 1.3B model trained on "textbook-quality" synthetic data rivals much larger models on reasoning.
MAmmoTH
An open-source LLM family specialized for general mathematical problem solving.
RLAIF (Scaling RLHF with AI Feedback)
Google compares RLHF with RLAIF (Reinforcement Learning from AI Feedback) to test whether AI preferences can replace human preferences.
GPT Solves Math Problems Without a Calculator
Demonstrates that with sufficient training data, even a small language model can perform accurate multi-digit arithmetic.
OPRO (LLMs as Optimizers)
DeepMind's OPRO uses LLMs as general-purpose optimizers over natural-language-described problems.
Cognitive Architectures for Language Agents (CoALA)
Princeton proposes CoALA, a systematic framework for understanding and building language agents.
Graph of Thoughts (GoT)
Generalizes Chain-of-Thought and Tree-of-Thought by modeling LLM reasoning as an arbitrary graph.
FacTool
A tool-augmented framework for detecting factual errors in LLM-generated text.
LegalBench
A collaboratively constructed benchmark for measuring legal reasoning in LLMs.
Model Compression for LLMs Survey
A survey of recent model-compression techniques applied specifically to LLMs.
GPT-4 Code Interpreter for Math
A zero-shot prompting technique for GPT-4 Code Interpreter that dramatically boosts math-reasoning accuracy via code self-verification.