Code · Training

CodeGen2.5

First page

Paper summary

Salesforce's new 7B code LLM trained on 1.5T tokens and optimized for fast sampling.

Ask this paper

Key points

Small-but-competitive: 7B model matches or beats prior >15B code-generation models, demonstrating data quality can substitute for model scale.

Fast-sampling optimization: Architecturally tuned for inference speed, making it practical for IDE integration use cases.

Multilingual code: Handles multiple programming languages with strong Python, JavaScript, and TypeScript performance.

Open code LLM: Part of the 2023 open-source code LLM wave (CodeGen, StarCoder, CodeLlama) that made private code assistants viable for enterprise.

Every Monday