Code Llama
Paper preview
Paper summary
Meta releases Code Llama, a family of code-specialized LLMs built on top of Llama 2.
Ask this paper
01
Three-tier release: Foundation base models, Python-specialist variants, and instruction-following Code Llama - Instruct models, all in 7B/13B/34B sizes.
02
Long context: Supports input contexts up to 100K tokens, enabling whole-repository or long-file code completion and analysis - unusual for open code LLMs at the time.
03
Fill-in-the-middle: Includes fill-in-the-middle support, a key capability for editor-integrated use cases like code completion and gap filling.
04
Strong HumanEval results: Code Llama - Python 34B reaches ~53% on HumanEval, establishing a strong open baseline for code models that persisted into 2024.