AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
The Larger They Are, the Harder They Fail
Reveals inverse-scaling failures in LLM code generation.
LLM Research Directions
A list of research directions for students entering LLM research.
Evidence of Meaning in Language Models Trained on Programs
Argues LMs learn meaning despite only next-token prediction.
Towards Expert-Level Medical Question Answering (Med-PaLM 2)
Google's second-generation medical LLM.
StructGPT
A general framework for LLM reasoning over structured data.
TinyStories
Explores how small LMs can be and still speak coherent English.
Symbol tuning
Fine-tunes LMs on in-context input-label pairs with natural-language labels replaced by arbitrary symbols.
PaLM 2
Google's second-generation PaLM powering Bard and Google products.
Unfaithful Explanations in Chain-of-Thought Prompting
Demonstrates CoT explanations can misrepresent the true reason for a model's prediction.
StarCoder
An open-access 15.5B code LLM with 8K context and 80+ programming languages.
Distilling Step-by-Step!
A mechanism to train smaller models that outperform larger LLMs using fewer examples.
Learning to Reason and Memorize with Self-Notes
LLMs that deviate from input to explicitly "think" and memorize.
Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models
A framework inferring tool sequences for compositional reasoning.
Teaching Large Language Models to Self-Debug
Teaches LLMs to debug their own code via few-shot demonstrations.