Survey of LLMs
Free while signed in. Answers cite the passages they came from.
A survey that maps the landscape of the three dominant LLM families - GPT, Llama, and PaLM - and the shared toolbox used to build and augment them.
Three-family framing: Organizes the field around GPT, Llama, and PaLM lineages, tracing how each family evolved since ChatGPT's November 2022 release.
Capabilities and techniques: Summarizes training techniques (pretraining, fine-tuning, RLHF) and augmentation methods (RAG, tool use, chain-of-thought) used across the three families.
Datasets and metrics: Catalogs the datasets used for training, fine-tuning, and evaluation, and compares popular evaluation benchmarks and metrics.
Open challenges: Closes with concrete research directions including efficiency, alignment, multilinguality, and evaluation that remain open for the next wave of LLM work.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack