SequenceMatch

Imitation learning framing: Views autoregressive generation as imitation learning with expert data, opening the door to standard IL techniques.

Backspace action: Introduces a "backspace" action that lets the model undo tokens that led to out-of-distribution sequences.

Compounding error mitigation: Addresses the classical autoregressive problem where small early errors compound catastrophically.

Training innovation: An interesting precursor to later work on self-correcting LLMs and reasoning with error recovery.