PromptBench
A unified library for comprehensive evaluation and analysis of LLMs that consolidates multiple evaluation concerns under one roof.
Ask this paper
Prompt-construction tooling: Ships with utilities for prompt construction, prompt engineering, and dataset/model loading, covering the end-to-end LLM evaluation workflow.
Adversarial prompt attacks: Built-in adversarial prompt-attack capabilities let users stress-test LLMs against perturbations rather than just measuring clean accuracy.
Dynamic evaluation: Supports dynamic evaluation protocols to detect dataset contamination and measure robustness beyond static benchmark numbers.
Unified interface: Replaces the ad-hoc evaluation scripts many teams maintain with a consistent API, reducing friction when comparing across models and prompt variants.