PromptBench
Free while signed in. Answers cite the passages they came from.
A unified library for comprehensive evaluation and analysis of LLMs that consolidates multiple evaluation concerns under one roof.
Prompt-construction tooling: Ships with utilities for prompt construction, prompt engineering, and dataset/model loading, covering the end-to-end LLM evaluation workflow.
Adversarial prompt attacks: Built-in adversarial prompt-attack capabilities let users stress-test LLMs against perturbations rather than just measuring clean accuracy.
Dynamic evaluation: Supports dynamic evaluation protocols to detect dataset contamination and measure robustness beyond static benchmark numbers.
Unified interface: Replaces the ad-hoc evaluation scripts many teams maintain with a consistent API, reducing friction when comparing across models and prompt variants.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack