Hypothesis Search (LLMs Can Learn Rules)

Rule induction phase: In the first stage, the LLM induces general rules from a small set of examples, producing an explicit rule library rather than implicit pattern matching.

Rule application phase: In the second stage, the model applies rules from its library to new problems, with explicit rule-lookup rather than end-to-end inference.

Improves reasoning: The explicit rule library improves reasoning performance on tasks where generalization from examples beats pure in-context learning.

Interpretability bonus: The learned rule library is human-readable and auditable, providing a window into what the model actually learned from its examples.