Code Llama
Free while signed in. Answers cite the passages they came from.
Meta releases Code Llama, a family of code-specialized LLMs built on top of Llama 2.
Three-tier release: Foundation base models, Python-specialist variants, and instruction-following Code Llama - Instruct models, all in 7B/13B/34B sizes.
Long context: Supports input contexts up to 100K tokens, enabling whole-repository or long-file code completion and analysis - unusual for open code LLMs at the time.
Fill-in-the-middle: Includes fill-in-the-middle support, a key capability for editor-integrated use cases like code completion and gap filling.
Strong HumanEval results: Code Llama - Python 34B reaches ~53% on HumanEval, establishing a strong open baseline for code models that persisted into 2024.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack