AI Papers of the Week
Every paper worth reading in AI, hand-picked one week at a time.
EfficientSAM
Meta's EfficientSAM is a lightweight Segment Anything variant that preserves most of SAM's zero-shot quality at a fraction of the compute.
Magicoder
Magicoder is a fully open-source code LLM that closes the gap with top commercial code models at only 7B parameters via high-quality synthetic instruction data.
Llama Guard
Meta's Llama Guard is a compact, instruction-tuned safety classifier built on Llama 2-7B for input/output moderation in conversational AI.
KTO (Kahneman-Tversky Optimization)
Contextual AI introduces KTO, an alignment objective derived from prospect theory that works with binary "good/bad" signals instead of preference pairs.
Data Management for LLMs
A survey of data-management research for LLM pretraining and supervised fine-tuning stages.
RankZephyr
RankZephyr is an open-source LLM for listwise zero-shot reranking that bridges the effectiveness gap with GPT-4.
Adversarial Diffusion Distillation (SDXL Turbo)
Stability AI's ADD trains a student diffusion model that produces high-quality images in just 1-4 sampling steps.
MEDITRON-70B
EPFL's MEDITRON is an open-source family of medical LLMs at 7B and 70B parameters, continually pretrained on curated medical corpora.
Medprompt
Microsoft researchers show that careful prompt engineering can push general-purpose GPT-4 to state-of-the-art on medical benchmarks, no domain fine-tuning required.
Advancing Long-Context LLMs
A survey of methodologies for improving Transformer long-context capability across pretraining, fine-tuning, and inference stages.
TÜLU 2
Allen AI's TÜLU 2 is a suite of improved open instruction-tuned LLMs and an accompanying study of adaptation best practices.
Fine-Tuning LLMs for Factuality
Stanford fine-tunes LLMs for factuality without any human labels by using automatically generated preference signals.
Survey on Language Models for Code
A comprehensive survey of LLMs for code covering 50+ models, 30+ evaluation tasks, and 500 related works.
Hallucination in LLMs Survey
A comprehensive survey of hallucination in LLMs, covering taxonomy, causes, evaluation, and mitigation.
Simplifying Transformer Blocks
Researchers show that many components of the standard transformer block can be removed with no loss in training speed or quality.
In-Context Learning Generalization Limits
Investigates whether transformers' in-context learning can generalize beyond the distribution of their pretraining data.
Rephrase and Respond (RaR)
An effective prompting method where the LLM rephrases and expands the user's question before answering it.
GPT4All Technical Report
The GPT4All technical report documents the model family and the open ecosystem built around democratizing local LLMs.
S-LoRA
S-LoRA enables serving thousands of LoRA adapters concurrently on a single GPU through memory-paging and custom CUDA kernels.
Battle of the Backbones
A large-scale benchmarking framework that compares vision backbones across a diverse suite of computer vision tasks.
ChipNeMo (LLMs for Chip Design)
NVIDIA's ChipNeMo applies domain-adapted LLMs to industrial chip design workflows.
YaRN (Efficient Context Extension)
YaRN is a compute-efficient method for extending the context window of LLMs well beyond their pretrained length.
FP8-LM
Microsoft's FP8-LM demonstrates that most LLM training variables - gradients, optimizer states - can use FP8 without sacrificing accuracy.
Zephyr
Hugging Face's Zephyr-7B is a 7B parameter LLM whose chat performance rivals much larger chat models aligned with human feedback.