AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
D-Bot (LLMs as Database Administrators)
Introduces D-Bot, an LLM-based framework that continuously acquires database-administration knowledge from textual sources.
AgentBench
Tsinghua's AgentBench is a multidimensional benchmark for LLM-as-Agent reasoning and decision-making across 8 environments.
Trustworthy LLMs
Presents a comprehensive framework of categories for assessing LLM trustworthiness.
Skeleton-of-Thought (SoT)
Microsoft's Skeleton-of-Thought parallelizes LLM generation by first producing an answer skeleton then filling it in concurrently.
Self-Check
Explores LLM capacity for self-checking on complex reasoning tasks requiring multi-step and non-linear thinking.
RT-2
Google DeepMind's end-to-end vision-language-action model that learns from both web and robotics data to control robots.
FacTool
A task- and domain-agnostic framework for factuality detection of LLM-generated text.
How is ChatGPT's Behavior Changing Over Time?
Evaluates GPT-3.5 and GPT-4 over months to show significant behavioral drift in deployed systems.
Measuring Faithfulness in Chain-of-Thought Reasoning
Anthropic's investigation into whether CoT reasoning actually reflects the model's internal decision process.
FLASK
Proposes fine-grained evaluation of LLMs decomposed into 12 alignment skill sets.
Claude 2
Anthropic's second-generation LLM with a detailed model card on safety, alignment, and capabilities.
LLMs as General Pattern Machines
Demonstrates LLMs serve as general sequence modelers without additional training.
Teaching Arithmetic to Small Transformers
Trains small transformers on chain-of-thought style data for arithmetic with large gains.
LLMs as Effective Text Rankers
A prompting technique that enables open-source LLMs to perform SOTA text ranking.
Multimodal Generation with Frozen LLMs
Maps images to LLM token space enabling models like PaLM and GPT-4 to handle visual tasks without parameter updates.
LeanDojo
An open-source Lean playground consisting of toolkits, data, models, and benchmarks for theorem proving.
Computer Vision Through the Lens of Natural Language
A modular approach solving CV problems by routing through LLM reasoning.
Understanding Theory-of-Mind in LLMs with LLMs
A framework for procedurally generating ToM evaluations using LLMs themselves.
SequenceMatch
Formulates sequence generation as imitation learning, enabling backtracking via a backspace action.
Unifying LLMs & Knowledge Graphs
A roadmap for combining LLMs with knowledge graphs for stronger reasoning.
Augmenting LLMs with Databases (ChatDB)
Combines an LLM with SQL databases as a symbolic memory framework.
Imitating Reasoning Process of Larger LLMs (Orca)
Microsoft's 13B model that imitates GPT-4's reasoning traces.
Let's Verify Step by Step
OpenAI's landmark paper on process reward models for mathematical reasoning.
Thought Cloning
Imitation learning framework that learns to think as well as act.