AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
Challenges & Application of LLMs
A comprehensive enumeration of open challenges and application domains for LLMs.
Retentive Network (RetNet)
Microsoft's proposed foundation architecture aiming to replace Transformer attention for LLMs.
Meta-Transformer
A unified framework performing learning across 12 different modalities with a shared backbone.
Retrieve In-Context Examples for LLMs
A framework to iteratively train dense retrievers that identify high-quality in-context examples.
FLASK
Proposes fine-grained evaluation of LLMs decomposed into 12 alignment skill sets.
CM3Leon
Meta's retrieval-augmented multi-modal language model that generates both text and images.
Claude 2
Anthropic's second-generation LLM with a detailed model card on safety, alignment, and capabilities.
Secrets of RLHF in LLMs
A deep investigation into RLHF with a focus on the inner workings of PPO, including open-source code.
LongLLaMA
Extends LLaMA's context length using a contrastive training process that reshapes the (key, value) space.
Patch n' Pack: NaViT
A vision transformer handling any aspect ratio and resolution through sequence packing.
LLMs as General Pattern Machines
Demonstrates LLMs serve as general sequence modelers without additional training.
HyperDreamBooth
A smaller, faster, and more efficient version of DreamBooth for personalizing text-to-image models.
Teaching Arithmetic to Small Transformers
Trains small transformers on chain-of-thought style data for arithmetic with large gains.
AnimateDiff
Animates frozen text-to-image diffusion models via a plug-in motion modeling module.
Generative Pretraining in Multimodality (Emu)
A transformer-based multimodal foundation model for generating images and text.
A Survey on Evaluation of LLMs
A comprehensive overview of evaluation methods covering what, where, and how to evaluate LLMs.
How Language Models Use Long Contexts (Lost-in-the-Middle)
Shows LLM performance drops when relevant information is in the middle of a long context.
LLMs as Effective Text Rankers
A prompting technique that enables open-source LLMs to perform SOTA text ranking.
Multimodal Generation with Frozen LLMs
Maps images to LLM token space enabling models like PaLM and GPT-4 to handle visual tasks without parameter updates.
CodeGen2.5
Salesforce's new 7B code LLM trained on 1.5T tokens and optimized for fast sampling.
Elastic Decision Transformer
An advance over Decision Transformers that enables trajectory stitching at inference time.
Robots That Ask for Help
A framework for calibrating LLM-based robot planners so they ask for help when uncertain.
Physics-based Motion Retargeting in Real-Time
Uses RL to retarget motions from sparse human sensor data to characters of various morphologies.
Scaling Transformer to 1 Billion Tokens (LongNet)
Microsoft's Transformer variant scaling sequence length past 1B tokens.