Struc-Bench (LLMs for Structured Data)

Structured data challenge: Tests LLMs on generating complex structured data (HTML tables, JSON, LaTeX) where surface-form correctness matters.

Structure-aware fine-tuning: Proposes a fine-tuning recipe specifically designed to teach small models the syntactic constraints of structured outputs.

7B beats GPT-4: A fine-tuned Llama 7B significantly outperforms GPT-3.5/4 and Vicuna-13B on structured-data generation benchmarks.

Deployment relevance: Demonstrates that for production structured-output applications, small specialized models can beat frontier general-purpose models at a fraction of the cost.