Benchmarking NN Training Algorithms (AlgoPerf)

Realistic workloads: Tests optimizers on actual production-scale tasks (ImageNet, language modeling, translation) rather than toy problems.

Wall-clock benchmarking: Evaluates optimizers on time-to-target-accuracy rather than just step counts, reflecting real training budgets.

Hyperparameter rules: Standardizes hyperparameter tuning budgets for fair cross-optimizer comparisons.

Optimizer research infrastructure: Enabled credible claims about new optimizers versus Adam and SGD - raising the bar for optimizer papers going forward.