Giraffe

Extended contexts: Fine-tuned models with 4K, 16K, and 32K context windows, providing ready-to-use open long-context variants.

Technique comparison: Systematically compares context-extension methods including positional interpolation, truncation strategies, and attention scaling.

Practitioner insights: Reports practical findings on which techniques preserve downstream quality at extended contexts - useful for anyone building long-context applications.

Context-extension recipe: The lessons from Giraffe fed directly into the recipes that would culminate in YaRN and similar approaches later that year.