AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Voyager
An LLM-powered embodied lifelong learning agent in Minecraft exploring autonomously.
Gorilla
A fine-tuned LLaMA-based model that surpasses GPT-4 on API call generation.
The False Promise of Imitating Proprietary LLMs
Berkeley's critical analysis of open-source imitation of proprietary LLMs.
Sophia
A simple, scalable second-order optimizer with negligible per-step overhead.
The Larger They Are, the Harder They Fail
Reveals inverse-scaling failures in LLM code generation.
Model Evaluation for Extreme Risks
DeepMind's framework for evaluating models for catastrophic-risk capabilities.
LLM Research Directions
A list of research directions for students entering LLM research.
Reinventing RNNs for the Transformer Era (RWKV)
Combines parallelizable training of Transformers with efficient RNN inference.
Drag Your GAN (DragGAN)
Interactive point-based image manipulation on the generative image manifold.
Evidence of Meaning in Language Models Trained on Programs
Argues LMs learn meaning despite only next-token prediction.
Towards Expert-Level Medical Question Answering (Med-PaLM 2)
Google's second-generation medical LLM.
MEGABYTE
Multiscale Transformers for predicting million-byte sequences.
StructGPT
A general framework for LLM reasoning over structured data.
TinyStories
Explores how small LMs can be and still speak coherent English.
DoReMi
Optimizes data mixtures for faster language model pretraining.
CodeT5+
An open code LLM family for code understanding and generation.
Symbol tuning
Fine-tunes LMs on in-context input-label pairs with natural-language labels replaced by arbitrary symbols.
Incidental Bilingualism in PaLM's Translation Capability
Explores where PaLM's translation ability actually comes from.
LLM Explains Neurons in LLMs
OpenAI's automated interpretability pipeline using GPT-4 to explain GPT-2 neurons.
PaLM 2
Google's second-generation PaLM powering Bard and Google products.
ImageBind
Meta's joint embedding across six modalities at once.
TidyBot
Combines LLM-based planning and perception with few-shot summarization to infer user preferences.
Unfaithful Explanations in Chain-of-Thought Prompting
Demonstrates CoT explanations can misrepresent the true reason for a model's prediction.
InstructBLIP
Visual-language instruction tuning built on BLIP-2.