OctoPack
Free while signed in. Answers cite the passages they came from.
Hugging Face releases OctoPack, a 4TB dataset of Git commits across 350 programming languages for instruction-tuning code LLMs.
4TB commit dataset: Curated dataset of 4 terabytes of Git commits across 350 programming languages, using commit messages as implicit instructions.
Natural code instructions: Commit messages provide real-world, naturally occurring instructions for code changes - far more authentic than synthetically generated code instructions.
SoTA without OpenAI outputs: Achieves state-of-the-art performance on HumanEval Python among models not trained on OpenAI outputs.
HumanEval extension: Extends HumanEval beyond Python generation to include code explanation and code repair tasks, providing richer evaluation coverage.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack