Survey of LLMs
A survey that maps the landscape of the three dominant LLM families - GPT, Llama, and PaLM - and the shared toolbox used to build and augment them.
Ask this paper
Three-family framing: Organizes the field around GPT, Llama, and PaLM lineages, tracing how each family evolved since ChatGPT's November 2022 release.
Capabilities and techniques: Summarizes training techniques (pretraining, fine-tuning, RLHF) and augmentation methods (RAG, tool use, chain-of-thought) used across the three families.
Datasets and metrics: Catalogs the datasets used for training, fine-tuning, and evaluation, and compares popular evaluation benchmarks and metrics.
Open challenges: Closes with concrete research directions including efficiency, alignment, multilinguality, and evaluation that remain open for the next wave of LLM work.