Harness-Aware Distillation for Small Language Model Agents
Moonseok Choi, Taehong Moon, Giung Nam and Juho Lee (KAIST AI and an independent researcher) propose Harness-Aware Distillation (HAD), which distills a harness-equipped agent by teaching the student the decisions the teacher makes differently because of the harness.
Ask this paper
Problem. When the harness stays in place after distillation, the student mostly needs the teacher's ability to act on harness information. Adding the harness to on-policy distillation raises harness use on ALFWorld from 65.7% to 73.1% but leaves success flat (43.1% to 43.5%).
Method. HAD queries the same teacher with and without harness information and trains the student to prefer the action chosen with it. A validity filter drops pairs whose preferred action contradicts the harness records. No task rewards or success labels are used.
Results. HAD reaches 63.4% on unseen and 51.4% on seen ALFWorld tasks against 47.0% and 41.4% for the best baseline, and exceeds its 8B teacher. Removing the validity check lowers success from 57.4% to 50.8%.
Recovery behavior. HAD escapes 59.7% of stalls, while no baseline clearly improves on the untrained student's 46.8%. Hiding the harness's state tracking cuts unseen-task success from 63.4% to 37.3%, so the student relies on the harness for state.
Abstract
Language model agents are deployed with a harness, the software around the model that manages its context, tools, and feedback. When such an agent is distilled into a smaller one, the harness stays in place, so the student mainly needs the teacher-specific abilities that the harness cannot provide, such as acting correctly on harness information. Standard distillation, however, imitates the teacher's full outputs and treats the harness as part of the input. We propose Harness-Aware Distillation (HAD), which focuses distillation on what the teacher adds beyond the harness. HAD complements on-policy distillation with two components: an action preference that contrasts the same teacher's actions with and without the harness information, scored after the student's own reasoning, and a validity check that drops preference pairs whose preferred action contradicts the harness records. We show that the contrast gives the student information that imitating the teacher alone cannot provide, and HAD needs no task rewards, success labels, or future information. Across multiple long-horizon agent benchmarks and models, HAD outperforms on-policy distillation baselines with the same fixed harness. Our analysis shows that HAD enters fewer unproductive loops and recovers from errors more often than the baselines, and suggests that it adaptively keeps learnable feedback in its weights while reading state information from the harness.