OctoPack
First page
Paper summary
Hugging Face releases OctoPack, a 4TB dataset of Git commits across 350 programming languages for instruction-tuning code LLMs.
Ask this paper
01
4TB commit dataset: Curated dataset of 4 terabytes of Git commits across 350 programming languages, using commit messages as implicit instructions.
02
Natural code instructions: Commit messages provide real-world, naturally occurring instructions for code changes - far more authentic than synthetically generated code instructions.
03
SoTA without OpenAI outputs: Achieves state-of-the-art performance on HumanEval Python among models not trained on OpenAI outputs.
04
HumanEval extension: Extends HumanEval beyond Python generation to include code explanation and code repair tasks, providing richer evaluation coverage.