Self-Adaptive LLMs

introduces Transformer^2, a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting singular components of their weight matrices; it’s built with two key phases: 1) a dispatch system that analyzes and identifies the properties of the incoming task, and 2) a step that combines "expert" vectors (trained via reinforcement learning) to create task-specific behaviors; claims to be more efficient than LoRA with fewer parameters and can works across different LLM architectures.