Model Compression for LLMs Survey

Core technique families: Covers quantization, pruning, knowledge distillation, and architectural compression across training-time and post-training approaches.

LLM-specific concerns: Addresses unique LLM concerns including long-sequence compression, KV-cache optimization, and retaining reasoning capability under compression.

Evaluation metrics: Reviews benchmark strategies and evaluation metrics for measuring compressed-LLM effectiveness - not just perplexity but downstream capability preservation.

Practitioner reference: Functions as a compact reference for teams deciding which compression technique matches their deployment constraints.