AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
GRIT
GRIT (Generative Representational Instruction Tuning) trains a single LLM to handle both generative and embedding tasks, switching behavior based on instructions.
AnyTool
AnyTool is a training-free LLM agent that scales tool-use to 16K+ Rapid APIs through a hierarchical retriever and a self-reflective solver.
Corrective RAG (CRAG)
CRAG adds a self-correcting loop around retrieval so a RAG system can detect and repair bad retrievals instead of feeding them straight into generation.
The Power of Noise: Redefining Retrieval in RAG
A study stress-testing the retriever component of RAG systems with surprising results about what actually helps generation.
RAG vs. Finetuning
Microsoft researchers systematically compare RAG and fine-tuning (and their combination) on LLMs like Llama 2 and GPT-4 using an agricultural domain dataset.
Mitigating Hallucination in LLMs
A survey cataloging 32 hallucination-mitigation techniques and organizing them into a practical taxonomy.
Exploiting Novel GPT-4 APIs
A red-team study of three newer GPT-4 API surfaces - fine-tuning, function calling, and knowledge retrieval - that reveals each introduces new attack vectors.
LLaRA
LLaRA adapts a decoder-only LLM for dense retrieval via two tailored pretext tasks that leverage text embeddings from the LLM itself.
RAG for LLMs
A broad survey of Retrieval-Augmented Generation research, organizing the rapidly growing literature into a coherent map.
UniIR
UniIR is a unified instruction-guided multimodal retriever that handles eight retrieval tasks across modalities with a single model.
Chain-of-Note (CoN)
Tencent's Chain-of-Note adds an explicit note-taking step to RAG so the model can evaluate retrieved evidence before answering.
Learning to Filter Context for RAG (FILCO)
CMU's FILCO improves RAG by training a dedicated model to filter retrieved contexts before they reach the generator.
FreshLLMs (FreshQA)
Introduces FreshQA, a dynamic benchmark designed to stress-test LLMs on time-sensitive knowledge.
ChipNeMo (LLMs for Chip Design)
NVIDIA's ChipNeMo applies domain-adapted LLMs to industrial chip design workflows.
Fact-Checking with LLMs
Investigates the fact-checking capabilities of frontier LLMs across multiple languages and claim types.
LLMs Meet New Knowledge
A benchmark that evaluates how well LLMs handle new knowledge beyond their training cutoff.
Self-RAG
Self-RAG trains an LM to adaptively retrieve, generate, and self-critique using special reflection tokens.
RAG for Long-Form QA
Explores retrieval-augmented LMs specifically on long-form question answering, where RAG failures are more subtle.
RECOMP (Retrieval-Augmented LMs with Compressors)
Proposes two compression approaches to shrink retrieved documents before in-context use.
InstructRetro
NVIDIA introduces Retro 48B, the largest LLM pretrained with retrieval at the time.
Retrieval Meets Long-Context LLMs
NVIDIA's study comparing RAG and long-context LLMs, with the punchline that the two are complementary rather than substitutes.
RA-DIT (Retrieval-Augmented Dual Instruction Tuning)
Meta's RA-DIT is a lightweight recipe that retrofits LLMs with retrieval capabilities through dual fine-tuning.
Vector Search with OpenAI Embeddings
Argues, via empirical analysis, that dedicated vector databases aren't necessarily required for modern AI-stack search applications.
FacTool
A tool-augmented framework for detecting factual errors in LLM-generated text.