Code · Evaluation

CodeTF

First page

Paper summary

An open-source Transformer library for state-of-the-art code LLMs.

Ask this paper

Key points

Code-LLM infrastructure: Provides pretrained code LLMs, popular code benchmarks, and standard methods for training and serving them efficiently.

Unified interface: Consistent API across different code LLMs makes comparison and swapping straightforward.

Benchmark-driven: Built-in evaluation on HumanEval, MBPP, and other code benchmarks enables easy empirical comparisons.

Open-source code AI: Part of the 2023 expansion of open-source code LLM tooling that made private coding assistants practical for enterprise.

Every Monday