LoraHub
Free while signed in. Answers cite the passages they came from.
Enables efficient cross-task generalization via dynamic LoRA composition.
Dynamic composition: Combines pre-trained LoRA modules via learned weights without human expertise or additional parameters/gradient updates.
Gradient-free optimization: Uses gradient-free algorithms like Nelder-Mead to find optimal LoRA weightings on a handful of examples.
ICL-matching performance: Matches the performance of in-context learning in few-shot settings while using much less inference compute.
Modular LLMs vision: Part of the broader push toward modular, composable adapter ecosystems - a direction still actively developed in 2024's MoE-of-LoRAs work.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack