Platypus
Free while signed in. Answers cite the passages they came from.
Platypus is a family of fine-tuned and merged LLMs that topped the Open LLM Leaderboard in August 2023.
LoRA fine-tuning + merging: Describes an efficient process for fine-tuning and merging LoRA modules, demonstrating that careful composition beats monolithic fine-tuning.
Open-Platypus dataset: Releases a small, highly curated fine-tuning dataset that delivers strong performance with short and cheap training - quality over quantity.
5 hours on one A100: A 13B Platypus can be trained on a single A100 GPU using 25K curated questions in roughly 5 hours.
Leaderboard-topping: Demonstrates that careful data curation and LoRA merging can produce leaderboard-topping open models without massive compute.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack