Platypus
First page
Paper summary
Platypus is a family of fine-tuned and merged LLMs that topped the Open LLM Leaderboard in August 2023.
Ask this paper
01
LoRA fine-tuning + merging: Describes an efficient process for fine-tuning and merging LoRA modules, demonstrating that careful composition beats monolithic fine-tuning.
02
Open-Platypus dataset: Releases a small, highly curated fine-tuning dataset that delivers strong performance with short and cheap training - quality over quantity.
03
5 hours on one A100: A 13B Platypus can be trained on a single A100 GPU using 25K curated questions in roughly 5 hours.
04
Leaderboard-topping: Demonstrates that careful data curation and LoRA merging can produce leaderboard-topping open models without massive compute.